Project

General

Profile

Actions
Copy link

Bug #56706

closed

_do_thrash:test_bluestore_reshard:Exception: ceph-bluestore-tool resharding failed

Added by Kamoltat (Junior) Sirivadhna almost 2 years ago. Updated almost 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Igor Fedotov
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6944099

2022-07-22T22:44:50.061 INFO:teuthology.orchestra.run.smithi064.stdout:m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P
2022-07-22T22:44:50.065 INFO:tasks.thrashosds.thrasher:applying new sharding to bluestore on osd.8
2022-07-22T22:44:50.066 INFO:tasks.thrashosds.thrasher:generate random sharding
2022-07-22T22:44:50.067 INFO:tasks.thrashosds.thrasher:applying new sharding: m(2) P(4,5-21) L(5)
2022-07-22T22:44:50.068 DEBUG:teuthology.orchestra.run.smithi064:> sudo ceph-bluestore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/bluestore_tool.$pid.log' --log-level=10 --path /var/lib/ceph/osd/ceph-8 --sharding 'm(2) P(4,5-21) L(5)' reshard
2022-07-22T22:44:50.467 INFO:teuthology.orchestra.run.smithi064.stdout:reshard success
2022-07-22T22:44:50.469 INFO:teuthology.orchestra.run.smithi064.stderr:*** Caught signal (Segmentation fault) **
2022-07-22T22:44:50.470 INFO:teuthology.orchestra.run.smithi064.stderr: in thread 7f8b57de7740 thread_name:ceph-bluestore-
2022-07-22T22:44:50.471 INFO:teuthology.orchestra.run.smithi064.stderr: ceph version 17.0.0-13750-g19acc2f7 (19acc2f7e3edb197f028ffa801e28f62f3698c79) quincy (dev)
2022-07-22T22:44:50.471 INFO:teuthology.orchestra.run.smithi064.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8b54ff2ce0]
2022-07-22T22:44:50.472 INFO:teuthology.orchestra.run.smithi064.stderr: 2: (RocksDBStore::submit_common(rocksdb::WriteOptions&, std::shared_ptr<KeyValueDB::TransactionImpl>)+0x7b) [0x55ddb0db841b]
2022-07-22T22:44:50.475 INFO:teuthology.orchestra.run.smithi064.stderr: 3: (RocksDBStore::submit_transaction_sync(std::shared_ptr<KeyValueDB::TransactionImpl>)+0x9a) [0x55ddb0db8e2a]
2022-07-22T22:44:50.476 INFO:teuthology.orchestra.run.smithi064.stderr: 4: (BlueStore::_close_db()+0x4e5) [0x55ddb0b8cbc5]
2022-07-22T22:44:50.477 INFO:teuthology.orchestra.run.smithi064.stderr: 5: (BlueStore::_close_db_and_around()+0x17) [0x55ddb0b8d507]
2022-07-22T22:44:50.477 INFO:teuthology.orchestra.run.smithi064.stderr: 6: (BlueStore::close_db_environment()+0xd) [0x55ddb0b8d55d]
2022-07-22T22:44:50.478 INFO:teuthology.orchestra.run.smithi064.stderr: 7: main()
2022-07-22T22:44:50.478 INFO:teuthology.orchestra.run.smithi064.stderr: 8: __libc_start_main()
2022-07-22T22:44:50.479 INFO:teuthology.orchestra.run.smithi064.stderr: 9: _start()
2022-07-22T22:44:50.480 INFO:teuthology.orchestra.run.smithi064.stderr:2022-07-22T22:44:50.463+0000 7f8b57de7740 -1 *** Caught signal (Segmentation fault) **
2022-07-22T22:44:50.481 INFO:teuthology.orchestra.run.smithi064.stderr: in thread 7f8b57de7740 thread_name:ceph-bluestore-
2022-07-22T22:44:50.481 INFO:teuthology.orchestra.run.smithi064.stderr:
2022-07-22T22:44:50.482 INFO:teuthology.orchestra.run.smithi064.stderr: ceph version 17.0.0-13750-g19acc2f7 (19acc2f7e3edb197f028ffa801e28f62f3698c79) quincy (dev)
2022-07-22T22:44:50.482 INFO:teuthology.orchestra.run.smithi064.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8b54ff2ce0]
2022-07-22T22:44:50.483 INFO:teuthology.orchestra.run.smithi064.stderr: 2: (RocksDBStore::submit_common(rocksdb::WriteOptions&, std::shared_ptr<KeyValueDB::TransactionImpl>)+0x7b) [0x55ddb0db841b]
2022-07-22T22:44:50.483 INFO:teuthology.orchestra.run.smithi064.stderr: 3: (RocksDBStore::submit_transaction_sync(std::shared_ptr<KeyValueDB::TransactionImpl>)+0x9a) [0x55ddb0db8e2a]
2022-07-22T22:44:50.484 INFO:teuthology.orchestra.run.smithi064.stderr: 4: (BlueStore::_close_db()+0x4e5) [0x55ddb0b8cbc5]
2022-07-22T22:44:50.484 INFO:teuthology.orchestra.run.smithi064.stderr: 5: (BlueStore::_close_db_and_around()+0x17) [0x55ddb0b8d507]
2022-07-22T22:44:50.485 INFO:teuthology.orchestra.run.smithi064.stderr: 6: (BlueStore::close_db_environment()+0xd) [0x55ddb0b8d55d]
2022-07-22T22:44:50.498 INFO:teuthology.orchestra.run.smithi064.stderr: 7: main()
2022-07-22T22:44:50.498 INFO:teuthology.orchestra.run.smithi064.stderr: 8: __libc_start_main()
2022-07-22T22:44:50.499 INFO:teuthology.orchestra.run.smithi064.stderr: 9: _start()
2022-07-22T22:44:50.508 INFO:teuthology.orchestra.run.smithi064.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2022-07-22T22:44:50.508 INFO:teuthology.orchestra.run.smithi064.stderr:
2022-07-22T22:44:50.514 INFO:teuthology.orchestra.run.smithi064.stderr:     0> 2022-07-22T22:44:50.463+0000 7f8b57de7740 -1 *** Caught signal (Segmentation fault) **
2022-07-22T22:44:50.521 INFO:teuthology.orchestra.run.smithi064.stderr: in thread 7f8b57de7740 thread_name:ceph-bluestore-
2022-07-22T22:44:50.522 INFO:teuthology.orchestra.run.smithi064.stderr:
2022-07-22T22:44:50.522 INFO:teuthology.orchestra.run.smithi064.stderr: ceph version 17.0.0-13750-g19acc2f7 (19acc2f7e3edb197f028ffa801e28f62f3698c79) quincy (dev)
2022-07-22T22:44:50.524 INFO:teuthology.orchestra.run.smithi064.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8b54ff2ce0]
2022-07-22T22:44:50.524 INFO:teuthology.orchestra.run.smithi064.stderr: 2: (RocksDBStore::submit_common(rocksdb::WriteOptions&, std::shared_ptr<KeyValueDB::TransactionImpl>)+0x7b) [0x55ddb0db841b]
2022-07-22T22:44:50.525 INFO:teuthology.orchestra.run.smithi064.stderr: 3: (RocksDBStore::submit_transaction_sync(std::shared_ptr<KeyValueDB::TransactionImpl>)+0x9a) [0x55ddb0db8e2a]
2022-07-22T22:44:50.525 INFO:teuthology.orchestra.run.smithi064.stderr: 4: (BlueStore::_close_db()+0x4e5) [0x55ddb0b8cbc5]
2022-07-22T22:44:50.525 INFO:teuthology.orchestra.run.smithi064.stderr: 5: (BlueStore::_close_db_and_around()+0x17) [0x55ddb0b8d507]
2022-07-22T22:44:50.531 INFO:teuthology.orchestra.run.smithi064.stderr: 6: (BlueStore::close_db_environment()+0xd) [0x55ddb0b8d55d]
2022-07-22T22:44:50.532 INFO:teuthology.orchestra.run.smithi064.stderr: 7: main()
2022-07-22T22:44:50.532 INFO:teuthology.orchestra.run.smithi064.stderr: 8: __libc_start_main()
2022-07-22T22:44:50.532 INFO:teuthology.orchestra.run.smithi064.stderr: 9: _start()
2022-07-22T22:44:50.532 INFO:teuthology.orchestra.run.smithi064.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2022-07-22T22:44:50.533 INFO:teuthology.orchestra.run.smithi064.stderr:
2022-07-22T22:44:50.533 INFO:teuthology.orchestra.run.smithi064.stderr:     0> 2022-07-22T22:44:50.463+0000 7f8b57de7740 -1 *** Caught signal (Segmentation fault) **
2022-07-22T22:44:50.533 INFO:teuthology.orchestra.run.smithi064.stderr: in thread 7f8b57de7740 thread_name:ceph-bluestore-
2022-07-22T22:44:50.533 INFO:teuthology.orchestra.run.smithi064.stderr:
2022-07-22T22:44:50.534 INFO:teuthology.orchestra.run.smithi064.stderr: ceph version 17.0.0-13750-g19acc2f7 (19acc2f7e3edb197f028ffa801e28f62f3698c79) quincy (dev)
2022-07-22T22:44:50.534 INFO:teuthology.orchestra.run.smithi064.stderr: 1: /lib64/libpthread.so.0(+0x12ce0) [0x7f8b54ff2ce0]
2022-07-22T22:44:50.534 INFO:teuthology.orchestra.run.smithi064.stderr: 2: (RocksDBStore::submit_common(rocksdb::WriteOptions&, std::shared_ptr<KeyValueDB::TransactionImpl>)+0x7b) [0x55ddb0db841b]
2022-07-22T22:44:50.534 INFO:teuthology.orchestra.run.smithi064.stderr: 3: (RocksDBStore::submit_transaction_sync(std::shared_ptr<KeyValueDB::TransactionImpl>)+0x9a) [0x55ddb0db8e2a]
2022-07-22T22:44:50.535 INFO:teuthology.orchestra.run.smithi064.stderr: 4: (BlueStore::_close_db()+0x4e5) [0x55ddb0b8cbc5]
2022-07-22T22:44:50.535 INFO:teuthology.orchestra.run.smithi064.stderr: 5: (BlueStore::_close_db_and_around()+0x17) [0x55ddb0b8d507]
2022-07-22T22:44:50.535 INFO:teuthology.orchestra.run.smithi064.stderr: 6: (BlueStore::close_db_environment()+0xd) [0x55ddb0b8d55d]
2022-07-22T22:44:50.535 INFO:teuthology.orchestra.run.smithi064.stderr: 7: main()
2022-07-22T22:44:50.536 INFO:teuthology.orchestra.run.smithi064.stderr: 8: __libc_start_main()
2022-07-22T22:44:50.536 INFO:teuthology.orchestra.run.smithi064.stderr: 9: _start()
2022-07-22T22:44:50.536 INFO:teuthology.orchestra.run.smithi064.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2022-07-22T22:44:50.536 INFO:teuthology.orchestra.run.smithi064.stderr:
2022-07-22T22:44:50.634 DEBUG:teuthology.orchestra.run:got remote process result: None
2022-07-22T22:44:50.636 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 190, in wrapper
    return func(self)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1422, in _do_thrash
    self.choose_action()()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1162, in test_bluestore_reshard
    self.test_bluestore_reshard_action()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1142, in test_bluestore_reshard_action
    raise Exception("ceph-bluestore-tool resharding failed.")
Exception: ceph-bluestore-tool resharding failed.

2022-07-22T22:44:50.636 ERROR:tasks.thrashosds.thrasher:exception:
Traceback (most recent call last):
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1290, in do_thrash
    self._do_thrash()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 190, in wrapper
    return func(self)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1422, in _do_thrash
    self.choose_action()()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1162, in test_bluestore_reshard
    self.test_bluestore_reshard_action()
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/ceph_manager.py", line 1142, in test_bluestore_reshard_action
    raise Exception("ceph-bluestore-tool resharding failed.")
Exception: ceph-bluestore-tool resharding failed.

Actions
Copy link
 #1

Updated by Kamoltat (Junior) Sirivadhna almost 2 years ago

  • Assignee set to Igor Fedotov
Actions
Copy link
 #2

Updated by Kamoltat (Junior) Sirivadhna almost 2 years ago

Dead job:
/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6943775

Actions
Copy link
 #3

Updated by Kamoltat (Junior) Sirivadhna almost 2 years ago

  • Status changed from New to Closed

closed because this is yet merged to main

Actions
Copy link

Also available in: Atom PDF