Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: *** Caught signal (Aborted) ** Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: in thread 7fc0da01c700 thread_name:md_log_replay Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: gsignal() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: abort() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 12: clone() Feb 27 12:35:14 ceph-mon1 ceph-mds[422129]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-mds[422129]: *** Caught signal (Aborted) ** in thread 7fc0da01c700 thread_name:md_log_replay ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] 12: clone() NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2024-02-27T20:35:14.548+0000 7fc0da01c700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2024-02-27T20:35:14.549+0000 7fc0da01c700 -1 *** Caught signal (Aborted) ** Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: in thread 7fc0da01c700 thread_name:md_log_replay Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: gsignal() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: abort() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 12: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-mds[422129]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-mds[422129]: *** Caught signal (Aborted) ** in thread 7fc0da01c700 thread_name:md_log_replay ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] 12: clone() NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: -9999> 2024-02-27T20:35:14.548+0000 7fc0da01c700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: -9998> 2024-02-27T20:35:14.549+0000 7fc0da01c700 -1 *** Caught signal (Aborted) ** Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: in thread 7fc0da01c700 thread_name:md_log_replay Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: gsignal() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: abort() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 12: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-mds[422129]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-mds[422129]: *** Caught signal (Aborted) ** in thread 7fc0da01c700 thread_name:md_log_replay ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] 2: gsignal() 3: abort() 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] 12: clone() NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: -1> 2024-02-27T20:35:14.548+0000 7fc0da01c700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: In function 'void interval_set::erase(T, T, std::function) [with T = inodeno_t; C = std::map]' thread 7fc0da01c700 time 2024-02-27T20:35:14.548959+0000 Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.2.7/rpm/el8/BUILD/ceph-17.2.7/src/include/interval_set.h: 568: FAILED ceph_assert(p->first <= start) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x135) [0x7fc0ea4674a3] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 0> 2024-02-27T20:35:14.549+0000 7fc0da01c700 -1 *** Caught signal (Aborted) ** Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: in thread 7fc0da01c700 thread_name:md_log_replay Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable) Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fc0e9460cf0] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 2: gsignal() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 3: abort() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x18f) [0x7fc0ea4674fd] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 5: /usr/lib64/ceph/libceph-common.so.2(+0x269669) [0x7fc0ea467669] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 6: (interval_set::erase(inodeno_t, inodeno_t, std::function)+0x2e5) [0x5631002c4885] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 7: (EMetaBlob::replay(MDSRank*, LogSegment*, int, MDPeerUpdate*)+0x4377) [0x5631005c97b7] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 8: (EUpdate::replay(MDSRank*)+0x61) [0x5631005cdbd1] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 9: (MDLog::_replay_thread()+0x7bb) [0x56310055454b] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 10: (MDLog::ReplayThread::entry()+0x11) [0x563100207041] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 11: /lib64/libpthread.so.0(+0x81ca) [0x7fc0e94561ca] Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: 12: clone() Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Feb 27 12:35:14 ceph-mon1 ceph-5e958f94-22fb-11eb-934f-0c42a12ce4f3-mds-our-cephfs-ceph-mon1-hpysoq[422106]: