Ceph » rbd

Given a large journal that has been trimmed, without starting at a specific (valid) commit position, it's likely you will encounter replay errors due to missing records that have been trimmed from one object but not another:

Example journal positions:

rbd --cluster master journal status 10612ae8944a 
minimum_set: 28
active_set: 34
registered clients: 
    [id=, commit_position=[positions=[[object_number=131, tag_tid=3, entry_tid=6511], [object_number=138, tag_tid=3, entry_tid=6510], [object_number=133, tag_tid=3, entry_tid=6509], [object_number=132, tag_tid=3, entry_tid=6508]]], state=connected]
    [id=e62a7d53-cb6a-4a62-ad50-d1d7642391bc, commit_position=[positions=[[object_number=117, tag_tid=2, entry_tid=64937], [object_number=112, tag_tid=2, entry_tid=64936], [object_number=115, tag_tid=2, entry_tid=64935], [object_number=118, tag_tid=2, entry_tid=64934]]], state=connected]

Result:

# rbd --cluster master journal inspect 10612ae8944a
2016-05-05 21:24:26.857870 7fb3a6a76700 -1 JournalPlayer: missing prior journal entry: Entry[tag_tid=2, entry_tid=50517, data size=7198]
rbd: failed to process journal: (42) No message of desired type
Summary:
  1 entries inspected, 0 errors
rbd: journal inspect: (42) No message of desired type

In this case, tag_tid=2, entry_tid=64937 should have been the starting point