Bug #7977
Updated by Dan Mick about 10 years ago
Calculation of the original session key is byte-order-dependent; cephx_calc_client_server_challenge gets a message digest of the random server and client challenges, and then steps through that digest 64 bits at a time and XORs each chunk to come up with a 64-bit key. But it does that by casting to uint64_t, which means the answer is different (byteswapped) if client and server have different byte orders.
Adding an mswab64() solves the problem:
<pre>
--- a/src/auth/cephx/CephxProtocol.cc
+++ b/src/auth/cephx/CephxProtocol.cc
@@ -45,7 +45,7 @@ void cephx_calc_client_server_challenge(CephContext *cct, CryptoKey& secret, uin
uint64_t k = 0;
const uint64_t *p = (const uint64_t *)enc.c_str();
for (int pos = 0; pos + sizeof(k) <= enc.length(); pos+=sizeof(k), p++)
- k ^= *p;
+ k ^= mswab64(*p);
*key = k;
ldout(cct, 0) << "cephx_calc_client_server_challenge: final key: " << std::hex << k << dendl;
}
</pre>