Project

General

Profile

Bug #16122

msg: upper 32-bits of message sequence get lost

Added by Zheng Yan 12 months ago. Updated 6 months ago.

Status:
Pending Backport
Priority:
High
Assignee:
-
Category:
msgr
Target version:
-
Start date:
06/02/2016
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
jewel, hammer
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Release:
Needs Doc:
No

Description

struct ceph_msg_header {
        __le64 seq;       /* message seq# for this session */
...
};

class Message : public RefCountedObject {
...
  uint64_t get_seq() const { return header.seq; }
  void set_seq(uint64_t s) { header.seq = s; }
...
}

Message sequence is 64-bits number, but the upper 32-bits get lost when setting/getting message's sequence.

void Pipe::reader()
{
...
      if (m->get_seq() <= in_seq) {
        ldout(msgr->cct,0) << "reader got old message " 
                << m->get_seq() << " <= " << in_seq << " " << m << " " << *m
                << ", discarding" << dendl;
        msgr->dispatch_throttle_release(m->get_dispatch_throttle_size());
        m->put();
        if (connection_state->has_feature(CEPH_FEATURE_RECONNECT_SEQ) &&
            msgr->cct->_conf->ms_die_on_old_message)
          assert(0 == "old msgs despite reconnect_seq feature");
        continue;
      }
...
}

When in_seq is 0xffffffff, Pipe::reader expects the incoming message's sequence to be 0x100000000. But m->get_seq() return 0. So the incoming message gets dropped.


Related issues

Related to Bug #17910: rgw: threads were stuck by discarded osd_op_reply message Duplicate 11/15/2016
Copied to Backport #18219: jewel: msg: upper 32-bits of message sequence get lost Resolved
Copied to Backport #18220: hammer: msg: upper 32-bits of message sequence get lost Need More Info

History

#1 Updated by Zheng Yan 12 months ago

  • Status changed from New to Need Review

#2 Updated by Yehuda Sadeh 6 months ago

  • Status changed from Need Review to Pending Backport
  • Backport set to jewel, hammer
  • Needs Doc set to No

#3 Updated by Yehuda Sadeh 6 months ago

  • Related to Bug #17910: rgw: threads were stuck by discarded osd_op_reply message added

#4 Updated by Nathan Cutler 6 months ago

  • Copied to Backport #18219: jewel: msg: upper 32-bits of message sequence get lost added

#5 Updated by Nathan Cutler 6 months ago

  • Copied to Backport #18220: hammer: msg: upper 32-bits of message sequence get lost added

Also available in: Atom PDF