Bug #45133: BlueStore asserting on fs upgrade tests - bluestore - Ceph

Actions

Copy link

Bug #45133

closed

BlueStore asserting on fs upgrade tests

Added by Greg Farnum about 4 years ago. Updated almost 4 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Igor Fedotov

Target version:

% Done:

Source:

Tags:

Backport:

octopus

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

34616

Crash signature (v1):

Crash signature (v2):

Description

See for instance http://pulpito.front.sepia.ceph.com/gregf-2020-04-15_20:54:50-fs-wip-greg-testing-415-1-distro-basic-smithi/4957531

2020-04-15T21:27:04.353 INFO:tasks.ceph.osd.0.smithi095.stderr:/build/ceph-16.0.0-779-g9da325c/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_prepare_ondisk_format_super(KeyValueDB::Transaction&)' thread 7f45ba1aed80 time 2020-04-15T21:27:04.351535+0000
2020-04-15T21:27:04.354 INFO:tasks.ceph.osd.0.smithi095.stderr:/build/ceph-16.0.0-779-g9da325c/src/os/bluestore/BlueStore.cc: 10916: FAILED ceph_assert(ondisk_format == latest_ondisk_format)
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: ceph version 16.0.0-779-g9da325c (9da325c9a5f9c40917142095f49236282c250d8e) pacific (dev)
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x154) [0x559c71a93504]
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 2: (ceph::__ceph_assertf_fail(char const*, char const*, int, char const*, char const*, ...)+0) [0x559c71a936df]
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 3: (BlueStore::_prepare_ondisk_format_super(std::shared_ptr<KeyValueDB::TransactionImpl>&)+0x332) [0x559c71fca0a2]
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 4: (BlueStore::_upgrade_super()+0x4fe) [0x559c71fcbfae]
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 5: (BlueStore::_mount(bool, bool)+0x599) [0x559c7201f719]
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 6: (OSD::init()+0x4d1) [0x559c71b30031]
2020-04-15T21:27:04.358 INFO:tasks.ceph.osd.0.smithi095.stderr: 7: (main()+0x3da5) [0x559c71a9a8d5]
2020-04-15T21:27:04.359 INFO:tasks.ceph.osd.0.smithi095.stderr: 8: (__libc_start_main()+0xe7) [0x7f45b6fadb97]
2020-04-15T21:27:04.359 INFO:tasks.ceph.osd.0.smithi095.stderr: 9: (_start()+0x2a) [0x559c71aaf52a]
2020-04-15T21:27:04.359 INFO:tasks.ceph.osd.0.smithi095.stderr:2020-04-15T21:27:04.351+0000 7f45ba1aed80 -1 /build/ceph-16.0.0-779-g9da325c/src/os/bluestore/BlueStore.cc: In function 'void BlueStore::_prepare_ondisk_format_super(KeyValueDB::Transaction&)' thread 7f45ba1aed80 time 2020-04-15T21:27:04.351535+0000
2020-04-15T21:27:04.359 INFO:tasks.ceph.osd.0.smithi095.stderr:/build/ceph-16.0.0-779-g9da325c/src/os/bluestore/BlueStore.cc: 10916: FAILED ceph_assert(ondisk_format == latest_ondisk_format)

As far as I can tell this all looks kosher — we install nautilus, then upgrade to master branch, and this occurs 11 seconds after restarting the OSDs.

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Greg Farnum about 4 years ago

Yes indeed, BlueStore::_upgrade_super() when you are starting with format 2 invokes _prepare_ondisk_format_super() as part of the upgrade to format 3, but latest_ondisk_format is now 4!

Actions

Copy link