Ceph : Issues
https://tracker.ceph.com/
https://tracker.ceph.com/favicon.ico
2023-11-03T19:36:39Z
Ceph
Redmine
bluestore - Bug #63436 (Pending Backport): Typo in reshard example
https://tracker.ceph.com/issues/63436
2023-11-03T19:36:39Z
Adam Kupczyk
<p>See <a class="external" href="https://tracker.ceph.com/issues/63353">https://tracker.ceph.com/issues/63353</a>.<br />I missed the fact that "o"->"O" should be done too.</p>
bluestore - Bug #63353 (Pending Backport): resharding RocksDB after upgrade to Pacific breaks OSDs
https://tracker.ceph.com/issues/63353
2023-10-30T13:14:14Z
Denis Polom
<p>Hi</p>
<p>we upgraded our Ceph cluster from latest Octopus to Pacific 16.2.14 and then we followed the docs (<a class="external" href="https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#rocksdb-sharding">https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#rocksdb-sharding</a>) to reshard RocksDB on our OSDs.</p>
<p>Despite resharding reports operation as successful, OSD fails to start.</p>
<pre>
# ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-5/ --sharding="m(3) p(3,0-12) o(3,0-13)=block_cache={type=binned_lru} l p" reshard
reshard success
</pre>
<pre>
Oct 30 12:44:17 octopus2 ceph-osd[4521]: /build/ceph-16.2.14/src/kv/RocksDBStore.cc: 1223: FAILED ceph_assert(recreate_mode)
Oct 30 12:44:17 octopus2 ceph-osd[4521]: ceph version 16.2.14 (238ba602515df21ea7ffc75c88db29f9e5ef12c9) pacific (stable)
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14b) [0x564047cb92b2]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 2: /usr/bin/ceph-osd(+0xaa948a) [0x564047cb948a]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 3: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1609) [0x564048794829]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 4: (BlueStore::_open_db(bool, bool, bool)+0x601) [0x564048240421]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 5: (BlueStore::_open_db_and_around(bool, bool)+0x26b) [0x5640482a5f8b]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 6: (BlueStore::_mount()+0x9c) [0x5640482a896c]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 7: (OSD::init()+0x38a) [0x564047daacea]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 8: main()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 9: __libc_start_main()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 10: _start()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 0> 2023-10-30T12:44:17.088+0000 7f4971ed2100 -1 *** Caught signal (Aborted) **
Oct 30 12:44:17 octopus2 ceph-osd[4521]: in thread 7f4971ed2100 thread_name:ceph-osd
Oct 30 12:44:17 octopus2 ceph-osd[4521]: ceph version 16.2.14 (238ba602515df21ea7ffc75c88db29f9e5ef12c9) pacific (stable)
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 1: /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730) [0x7f4972921730]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 2: gsignal()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 3: abort()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19c) [0x564047cb9303]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 5: /usr/bin/ceph-osd(+0xaa948a) [0x564047cb948a]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 6: (RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1609) [0x564048794829]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 7: (BlueStore::_open_db(bool, bool, bool)+0x601) [0x564048240421]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 8: (BlueStore::_open_db_and_around(bool, bool)+0x26b) [0x5640482a5f8b]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 9: (BlueStore::_mount()+0x9c) [0x5640482a896c]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 10: (OSD::init()+0x38a) [0x564047daacea]
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 11: main()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 12: __libc_start_main()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: 13: _start()
Oct 30 12:44:17 octopus2 ceph-osd[4521]: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Oct 30 12:44:17 octopus2 ceph-osd[4521]: -1> 2023-10-30T12:44:17.084+0000 7f4971ed2100 -1 /build/ceph-16.2.14/src/kv/RocksDBStore.cc: In function 'int RocksDBStore::do_open(std::ostream&, bool, bool, const string&)' thread 7f4971ed2100 time 2023-10-30T12:44:17.087172+0000
</pre>
<p>Any advice will be appreciated.</p>
<p>thx</p>
bluestore - Backport #61465 (New): reef: Fragmentation score rising by seemingly stuck thread
https://tracker.ceph.com/issues/61465
2023-05-26T10:44:59Z
Backport Bot
bluestore - Backport #61463 (New): quincy: Fragmentation score rising by seemingly stuck thread
https://tracker.ceph.com/issues/61463
2023-05-26T10:44:45Z
Backport Bot
RADOS - Bug #59099 (New): PG move causes data duplication
https://tracker.ceph.com/issues/59099
2023-03-17T13:51:03Z
Adam Kupczyk
<p>Lets imagine we have a pool TEST.<br />In the PG we have object OBJ of size 1M.</p>
<p>We create snap SNAP-1 and write some 4K to OBJ.<br />As result we get OBJ.1 that takes 1M and OBJ.head that reuses all but 4K.<br />The total data usage is 1M + 4K.</p>
<p>Now we move PG to other OSD.<br />In some cases OBJ.head + OBJ.1 will take 2M.</p>
<p>The example of this happening is in attachment snap-pg-move-history.sh.<br />When data is on original PG on OSD.0:</p>
<p>ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS<br /> 0 ssd 0.09859 1.00000 101 GiB 1.1 GiB 101 MiB 0 B 21 MiB 100 GiB 1.09 1.05 2 up<br /> 1 ssd 0.09859 1.00000 101 GiB 1.0 GiB 740 KiB 0 B 20 MiB 100 GiB 0.99 0.95 1 up<br /> TOTAL 202 GiB 2.1 GiB 101 MiB 0 B 41 MiB 200 GiB 1.04 <br />MIN/MAX VAR: 0.95/1.05 STDDEV: 0.05</p>
<p>And after forcibly moving PG to OSD.</p>
<p>ID CLASS WEIGHT REWEIGHT SIZE RAW USE DATA OMAP META AVAIL %USE VAR PGS STATUS<br /> 0 ssd 0.09859 1.00000 101 GiB 1.0 GiB 756 KiB 0 B 21 MiB 100 GiB 0.99 0.91 1 up<br /> 1 ssd 0.09859 1.00000 101 GiB 1.2 GiB 201 MiB 0 B 21 MiB 100 GiB 1.18 1.09 2 up<br /> TOTAL 202 GiB 2.2 GiB 201 MiB 0 B 42 MiB 200 GiB 1.09 <br />MIN/MAX VAR: 0.91/1.09 STDDEV: 0.10</p>
<p>The script was tested on Reef, but I do not believe it is limited to it.</p>
Ceph - Bug #58596 (New): rocksdb: rm_range_keys() (message with 'enter') logs binary data
https://tracker.ceph.com/issues/58596
2023-01-29T07:19:21Z
Ronen Friedman
rfriedma@redhat.com
<p>that log message contains keys in their binary format, causing<br />a problem for grep(1) and editors (and might create a security<br />issue).</p>
bluestore - Backport #55517 (New): quincy: test_cls_rbd.sh: multiple TestClsRbd failures during u...
https://tracker.ceph.com/issues/55517
2022-05-02T17:20:08Z
Backport Bot
bluestore - Bug #55444 (Pending Backport): test_cls_rbd.sh: multiple TestClsRbd failures during u...
https://tracker.ceph.com/issues/55444
2022-04-26T01:14:27Z
Laura Flores
<p>Description: rados/upgrade/nautilus-x-singleton/{0-cluster/{openstack start} 1-install/nautilus 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{rbd-cls rbd-import-export readwrite snaps-few-objects} 5-workload/{radosbench rbd_api} 6-finish-upgrade 7-pacific 8-workload/{rbd-python snaps-many-objects} bluestore-bitmap mon_election/classic thrashosds-health ubuntu_18.04}</p>
<p>/a/lflores-2022-04-22_20:48:19-rados-wip-55324-pacific-backport-distro-default-smithi/6801098<br /><pre><code class="text syntaxhl"><span class="CodeRay">2022-04-23T08:54:27.447 INFO:tasks.workunit.client.0.smithi084.stdout:[ RUN ] TestClsRbd.directory_methods
2022-04-23T08:54:27.465 INFO:tasks.workunit.client.0.smithi084.stdout:/build/ceph-14.2.22/src/test/cls_rbd/test_cls_rbd.cc:297: Failure
2022-04-23T08:54:27.465 INFO:tasks.workunit.client.0.smithi084.stdout: Expected: -16
2022-04-23T08:54:27.465 INFO:tasks.workunit.client.0.smithi084.stdout:To be equal to: dir_state_set(&ioctx, oid, cls::rbd::DIRECTORY_STATE_ADD_DISABLED)
2022-04-23T08:54:27.465 INFO:tasks.workunit.client.0.smithi084.stdout: Which is: 0
2022-04-23T08:54:27.466 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.directory_methods (18 ms)
...
2022-04-23T08:54:27.633 INFO:tasks.workunit.client.0.smithi084.stdout:/build/ceph-14.2.22/src/test/cls_rbd/test_cls_rbd.cc:750: Failure
2022-04-23T08:54:27.633 INFO:tasks.workunit.client.0.smithi084.stdout: Expected: 0
2022-04-23T08:54:27.633 INFO:tasks.workunit.client.0.smithi084.stdout:To be equal to: get_parent(&ioctx, oid, 10, &pspec, &size)
2022-04-23T08:54:27.634 INFO:tasks.workunit.client.0.smithi084.stdout: Which is: -22
2022-04-23T08:54:27.634 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.parents_v1 (45 ms)
...
2022-04-23T08:54:27.729 INFO:tasks.workunit.client.0.smithi084.stdout:/build/ceph-14.2.22/src/test/cls_rbd/test_cls_rbd.cc:1008: Failure
2022-04-23T08:54:27.730 INFO:tasks.workunit.client.0.smithi084.stdout: Expected: 1u
2022-04-23T08:54:27.730 INFO:tasks.workunit.client.0.smithi084.stdout: Which is: 1
2022-04-23T08:54:27.730 INFO:tasks.workunit.client.0.smithi084.stdout:To be equal to: snapc.snaps.size()
2022-04-23T08:54:27.730 INFO:tasks.workunit.client.0.smithi084.stdout: Which is: 0
2022-04-23T08:54:27.730 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.snapshots (6 ms)
...
2022-04-23T08:54:27.778 INFO:tasks.workunit.client.0.smithi084.stdout:/build/ceph-14.2.22/src/test/cls_rbd/test_cls_rbd.cc:1437: Failure
2022-04-23T08:54:27.778 INFO:tasks.workunit.client.0.smithi084.stdout: Expected: 2U
2022-04-23T08:54:27.778 INFO:tasks.workunit.client.0.smithi084.stdout: Which is: 2
2022-04-23T08:54:27.778 INFO:tasks.workunit.client.0.smithi084.stdout:To be equal to: pairs.size()
2022-04-23T08:54:27.778 INFO:tasks.workunit.client.0.smithi084.stdout: Which is: 0
2022-04-23T08:54:27.779 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.metadata (6 ms)
... + 22 more failed tests
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[==========] 67 tests from 1 test case ran. (22012 ms total)
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ PASSED ] 41 tests.
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] 26 tests, listed below:
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.directory_methods
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.parents_v1
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.snapshots
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.metadata
2022-04-23T08:54:39.861 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.mirror
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.mirror_image
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.mirror_image_status
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.mirror_image_map
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_dir_list
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_dir_add
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.dir_add_already_existing
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_dir_rename
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_dir_remove
2022-04-23T08:54:39.862 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_dir_remove_missing
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_image_add
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_image_remove
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_image_list
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_image_clean
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.image_group_add
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_snap_set_duplicate_name
2022-04-23T08:54:39.863 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_snap_set
2022-04-23T08:54:39.864 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_snap_list
2022-04-23T08:54:39.864 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.group_snap_remove
2022-04-23T08:54:39.864 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.trash_methods
2022-04-23T08:54:39.864 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.clone_child
2022-04-23T08:54:39.864 INFO:tasks.workunit.client.0.smithi084.stdout:[ FAILED ] TestClsRbd.namespace_methods
</span></code></pre></p>
bluestore - Bug #55187 (Need More Info): ceph_abort_msg(\"bluefs enospc\")
https://tracker.ceph.com/issues/55187
2022-04-05T16:10:38Z
Aishwarya Mathuria
<p>from osd crash info in gibba cluster: <br /><pre>
$ sudo ceph crash info 2022-04-05T02:08:50.176782Z_e45030ee-e34b-46a4-bdde-3bcb3e8005fa
{
"assert_condition": "abort",
"assert_file": "/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.1.0-163-g4e244311/rpm/el8/BUILD/ceph-17.1.0-163-g4e244311/src/os/bluestore/BlueFS.cc",
"assert_func": "int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t)",
"assert_line": 3137,
"assert_msg": "/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.1.0-163-g4e244311/rpm/el8/BUILD/ceph-17.1.0-163-g4e244311/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range_F(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f10c8ce23c0 time 2022-04-05T02:08:50.161549+0000\n/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.1.0-163-g4e244311/rpm/el8/BUILD/ceph-17.1.0-163-g4e244311/src/os/bluestore/BlueFS.cc: 3137: ceph_abort_msg(\"bluefs enospc\")\n",
"assert_thread_name": "ceph-osd",
"backtrace": [
"/lib64/libpthread.so.0(+0x12ce0) [0x7f10c6ee7ce0]",
"gsignal()",
"abort()",
"(ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b8) [0x56488be01c6c]",
"(BlueFS::_flush_range_F(BlueFS::FileWriter*, unsigned long, unsigned long)+0x943) [0x56488c5697f3]",
"(BlueFS::_flush_F(BlueFS::FileWriter*, bool, bool*)+0xa9) [0x56488c5699d9]",
"(BlueFS::fsync(BlueFS::FileWriter*)+0x19e) [0x56488c58620e]",
"(BlueRocksWritableFile::Sync()+0x18) [0x56488c595f88]",
"(rocksdb::LegacyWritableFileWrapper::Sync(rocksdb::IOOptions const&, rocksdb::IODebugContext*)+0x1f) [0x56488cab96bf]",
"(rocksdb::WritableFileWriter::SyncInternal(bool)+0x662) [0x56488cbe9e92]",
"(rocksdb::WritableFileWriter::Sync(bool)+0xf8) [0x56488cbeb858]",
"(rocksdb::SyncManifest(rocksdb::Env*, rocksdb::ImmutableDBOptions const*, rocksdb::WritableFileWriter*)+0x11d) [0x56488cbe2d5d]",
"(rocksdb::VersionSet::ProcessManifestWrites(std::deque<rocksdb::VersionSet::ManifestWriter, std::allocator<rocksdb::VersionSet::ManifestWriter> >&, rocksdb::InstrumentedMutex*, rocksdb::FSDirectory*, bool, rocksdb::ColumnFamilyOptions const*)+0x181c) [0x56488cba9aac]",
"(rocksdb::VersionSet::LogAndApply(rocksdb::autovector<rocksdb::ColumnFamilyData*, 8ul> const&, rocksdb::autovector<rocksdb::MutableCFOptions const*, 8ul> const&, rocksdb::autovector<rocksdb::autovector<rocksdb::VersionEdit*, 8ul>, 8ul> const&, rocksdb::InstrumentedMutex*, rocksdb::FSDirectory*, bool, rocksdb::ColumnFamilyOptions const*, std::vector<std::function<void (rocksdb::Status const&)>, std::allocator<std::function<void (rocksdb::Status const&)> > > const&)+0xad1) [0x56488cbab711]",
"(rocksdb::VersionSet::LogAndApply(rocksdb::ColumnFamilyData*, rocksdb::MutableCFOptions const&, rocksdb::VersionEdit*, rocksdb::InstrumentedMutex*, rocksdb::FSDirectory*, bool, rocksdb::ColumnFamilyOptions const*)+0x1c6) [0x56488cac4ba6]",
"(rocksdb::DBImpl::DeleteUnreferencedSstFiles()+0x99a) [0x56488caf4d1a]",
"(rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)+0x1269) [0x56488cb035d9]",
"(rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)+0x5a3) [0x56488cafc3c3]",
"(rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)+0x15) [0x56488cafda75]",
"(RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x10a1) [0x56488ca70541]",
"(BlueStore::_open_db(bool, bool, bool)+0x68c) [0x56488c47100c]",
"(BlueStore::_open_db_and_around(bool, bool)+0x34b) [0x56488c4ba23b]",
"(BlueStore::_mount()+0x1ae) [0x56488c4bd38e]",
"(OSD::init()+0x403) [0x56488bf3f513]",
"main()",
"__libc_start_main()",
"_start()"
],
"ceph_version": "17.1.0-163-g4e244311",
"crash_id": "2022-04-05T02:08:50.176782Z_e45030ee-e34b-46a4-bdde-3bcb3e8005fa",
"entity_name": "osd.652",
"os_id": "centos",
"os_name": "CentOS Stream",
"os_version": "8",
"os_version_id": "8",
"process_name": "ceph-osd",
"stack_sig": "8399067597cb57a8aae13abe0f112a83b5cf7cc7d53b85d56fd1afb99a6c5bed",
"timestamp": "2022-04-05T02:08:50.176782Z",
"utsname_hostname": "gibba024",
"utsname_machine": "x86_64",
"utsname_release": "4.18.0-301.1.el8.x86_64",
"utsname_sysname": "Linux",
"utsname_version": "#1 SMP Tue Apr 13 16:24:22 UTC 2021"
}
</pre></p>
bluestore - Bug #53359 (New): bluestore: missing block.db symlinks leads to confusing crash
https://tracker.ceph.com/issues/53359
2021-11-22T16:09:44Z
Sage Weil
sage@newdream.net
<p>A regression in ceph-volume (master branch) led to the block.db symlink not getting created. This leads to OSDs that crash like so:</p>
<pre>
"backtrace": [
"/lib64/libpthread.so.0(+0x12c20) [0x7f3573347c20]",
"gsignal()",
"abort()",
"/lib64/libstdc++.so.6(+0x9009b) [0x7f357295e09b]",
"/lib64/libstdc++.so.6(+0x9653c) [0x7f357296453c]",
"/lib64/libstdc++.so.6(+0x96597) [0x7f3572964597]",
"/lib64/libstdc++.so.6(+0x967f8) [0x7f35729647f8]",
"/usr/bin/ceph-osd(+0x5c7203) [0x55cc53713203]",
"(BlueFS::_open_super()+0x18f) [0x55cc53e66cff]",
"(BlueFS::mount()+0xeb) [0x55cc53e88ddb]",
"(BlueStore::_open_bluefs(bool, bool)+0x94) [0x55cc53d4bad4]",
"(BlueStore::_prepare_db_environment(bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)+0x6d9) [0x55cc53d4cc29]",
"(BlueStore::_open_db(bool, bool, bool)+0x15c) [0x55cc53d4df4c]",
"(BlueStore::_open_db_and_around(bool, bool)+0x2b4) [0x55cc53dc68d4]",
"(BlueStore::_mount()+0x1ae) [0x55cc53dc971e]",
"(OSD::init()+0x3ba) [0x55cc5385711a]",
"main()",
"__libc_start_main()",
"_start()"
],
"ceph_version": "17.0.0-9073-g6e528ed7",
</pre>
<p>The on-disk block that we are trying to decode is all zeros.</p>
<p>I thought we had a flag somewhere indicating whether a db and/or wal was expected so that we could provide a meaningful/informative error message, but maybe not?</p>
<p>(ceph-volume fix is here: <a class="external" href="https://github.com/ceph/ceph/pull/44030">https://github.com/ceph/ceph/pull/44030</a>)</p>
bluestore - Bug #50844 (Triaged): ceph_assert(r == 0) in BlueFS::_rewrite_log_and_layout_sync()
https://tracker.ceph.com/issues/50844
2021-05-17T16:36:35Z
Neha Ojha
nojha@redhat.com
<pre>
2021-05-17T12:08:17.044 INFO:tasks.workunit.client.0.smithi104.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:179: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --dev-target td/osd-bluefs-volume-ops/1/db --command bluefs-bdev-new-db
2021-05-17T12:08:17.053 INFO:tasks.workunit.client.0.smithi104.stdout:inferring bluefs devices from bluestore path
2021-05-17T12:08:24.848 INFO:tasks.workunit.client.0.smithi104.stderr:2021-05-17T12:08:24.846+0000 7f30f9fc5400 -1 bluefs _allocate_without_fallback unable to allocate 0x500000 on bdev 0, allocator name bluefs-wal, allocator type hybrid, capacity 0x20000000, block size 0x100000, free 0xff000, fragmentation 0, allocated 0x0
2021-05-17T12:08:24.848 INFO:tasks.workunit.client.0.smithi104.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4229-gd98b3fc9/rpm/el8/BUILD/ceph-17.0.0-4229-gd98b3fc9/src/os/bluestore/BlueFS.cc: In function 'void BlueFS::_rewrite_log_and_layout_sync(bool, int, int, int, int, std::optional<bluefs_layout_t>)' thread 7f30f9fc5400 time 2021-05-17T12:08:24.846276+0000
2021-05-17T12:08:24.849 INFO:tasks.workunit.client.0.smithi104.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos8/DIST/centos8/MACHINE_SIZE/gigantic/release/17.0.0-4229-gd98b3fc9/rpm/el8/BUILD/ceph-17.0.0-4229-gd98b3fc9/src/os/bluestore/BlueFS.cc: 2241: FAILED ceph_assert(r == 0)
2021-05-17T12:08:24.849 INFO:tasks.workunit.client.0.smithi104.stderr: ceph version 17.0.0-4229-gd98b3fc9 (d98b3fc98cdd22d1e98566aab6a991dad70d1b4d) quincy (dev)
2021-05-17T12:08:24.850 INFO:tasks.workunit.client.0.smithi104.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x152) [0x7f30f8228782]
2021-05-17T12:08:24.850 INFO:tasks.workunit.client.0.smithi104.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x27c98a) [0x7f30f822898a]
2021-05-17T12:08:24.850 INFO:tasks.workunit.client.0.smithi104.stderr: 3: (BlueFS::_rewrite_log_and_layout_sync(bool, int, int, int, int, std::optional<bluefs_layout_t>)+0x108a) [0x564969c2954a]
2021-05-17T12:08:24.851 INFO:tasks.workunit.client.0.smithi104.stderr: 4: (BlueFS::prepare_new_device(int, bluefs_layout_t const&)+0x19f) [0x564969c297bf]
2021-05-17T12:08:24.851 INFO:tasks.workunit.client.0.smithi104.stderr: 5: (BlueStore::add_new_bluefs_device(int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x2f7) [0x564969cde5c7]
2021-05-17T12:08:24.852 INFO:tasks.workunit.client.0.smithi104.stderr: 6: main()
2021-05-17T12:08:24.852 INFO:tasks.workunit.client.0.smithi104.stderr: 7: __libc_start_main()
2021-05-17T12:08:24.852 INFO:tasks.workunit.client.0.smithi104.stderr: 8: _start()
</pre>
<p>/a/sseshasa-2021-05-17_11:08:21-rados-wip-sseshasa-testing-2021-05-17-1504-distro-basic-smithi/6118192</p>
bluestore - Bug #38745 (In Progress): spillover that doesn't make sense
https://tracker.ceph.com/issues/38745
2019-03-14T22:08:56Z
Sage Weil
sage@newdream.net
<pre>
BLUEFS_SPILLOVER BlueFS spillover detected on 3 OSD(s)
osd.50 spilled over 1.3 GiB metadata from 'db' device (20 GiB used of 31 GiB) to slow device
osd.94 spilled over 1.1 GiB metadata from 'db' device (16 GiB used of 31 GiB) to slow device
osd.103 spilled over 1.0 GiB metadata from 'db' device (18 GiB used of 31 GiB) to slow device
</pre><br />this is on the sepia lab cluster.
bluestore - Bug #38363 (Need More Info): Failure in assert when calling: ceph-volume lvm prepare ...
https://tracker.ceph.com/issues/38363
2019-02-18T13:41:08Z
Rainer Krienke
<p>I run Ubuntu 18.04 and and ceph version 13.2.4-1bionic from this repo: <a class="external" href="https://download.ceph.com/debian-mimic">https://download.ceph.com/debian-mimic</a>.</p>
<p>When I try to create a new bluestore osd on several 4TB disks I get an error I first thought was related to <a class="external" href="http://tracker.ceph.com/issues/15386_(read_fsid">http://tracker.ceph.com/issues/15386_(read_fsid</a> unparsable uuid) . However a user cephs user list gave me a hint that in my error log I posted an assertion failure is the real problem not the _read_fsid unparsable uuid message, So I created this new bug report. The same also happens when I omit the --bluestore option.</p>
<p>So here is the complete log for a run of ceph-volume to create an osd which fails reproducebly. I also tried several different devices but the result was always the same:</p>
<ol>
<li>ceph-volume lvm prepare --bluestore --data /dev/sdg</li>
</ol>
<p>Running command: /usr/bin/ceph-authtool --gen-print-key<br />Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new a87c3a87-cf22-41df-af4b-c971ed4c0e1a<br />Running command: /sbin/vgcreate --force --yes ceph-22a3d361-78b5-40b4-8af3-74b1efe1b65a /dev/sdg<br /> stdout: Physical volume "/dev/sdg" successfully created.<br /> stdout: Volume group "ceph-22a3d361-78b5-40b4-8af3-74b1efe1b65a" successfully created<br />Running command: /sbin/lvcreate --yes -l 100%FREE -n osd-block-a87c3a87-cf22-41df-af4b-c971ed4c0e1a ceph-22a3d361-78b5-40b4-8af3-74b1efe1b65a<br /> stdout: Logical volume "osd-block-a87c3a87-cf22-41df-af4b-c971ed4c0e1a" created.<br />Running command: /usr/bin/ceph-authtool --gen-print-key<br />Running command: /bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0<br />--> Absolute path not found for executable: restorecon<br />--> Ensure $PATH environment variable contains common executable locations<br />Running command: /bin/chown -h ceph:ceph /dev/ceph-22a3d361-78b5-40b4-8af3-74b1efe1b65a/osd-block-a87c3a87-cf22-41df-af4b-c971ed4c0e1a<br />Running command: /bin/chown -R ceph:ceph /dev/dm-8<br />Running command: /bin/ln -s /dev/ceph-22a3d361-78b5-40b4-8af3-74b1efe1b65a/osd-block-a87c3a87-cf22-41df-af4b-c971ed4c0e1a /var/lib/ceph/osd/ceph-0/block<br />Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap<br /> stderr: got monmap epoch 1<br />Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQAQtGpcjkxOMxAARlPykBaxHWqIyndvjTMNuQ==<br /> stdout: creating /var/lib/ceph/osd/ceph-0/keyring<br />added entity osd.0 auth auth(auid = 18446744073709551615 key=AQAQtGpcjkxOMxAARlPykBaxHWqIyndvjTMNuQ== with 0 caps)<br />Running command: /bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring<br />Running command: /bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/<br />Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid a87c3a87-cf22-41df-af4b-c971ed4c0e1a --setuser ceph --setgroup ceph<br /> stderr: 2019-02-18 14:33:07.093 7fb9508d5240 -1 bluestore(/var/lib/ceph/osd/ceph-0/) <em>read_fsid unparsable uuid<br /> stderr: /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fb9508d5240 time 2019-02-18 14:33:07.155877<br /> stderr: /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: 821: FAILED assert((uint64_t)r len)<br /> stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)<br /> stderr: 1: (ceph::</em>_ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7fb947cf53e2]<br /> stderr: 2: (()+0x26d5a7) [0x7fb947cf55a7]<br /> stderr: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::list*, IOContext*, bool)+0x4a7) [0x55a21e5d4817]<br /> stderr: 4: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x435) [0x55a21e5945c5]<br /> stderr: 5: (BlueFS::_replay(bool, bool)+0x214) [0x55a21e59a434]<br /> stderr: 6: (BlueFS::mount()+0x1f1) [0x55a21e59ec81]<br /> stderr: 7: (BlueStore::_open_db(bool, bool)+0x17cd) [0x55a21e4c504d]<br /> stderr: 8: (BlueStore::mkfs()+0x805) [0x55a21e4f5fe5]<br /> stderr: 9: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; > const&, uuid_d, int)+0x1b0) [0x55a21e09e480]<br /> stderr: 10: (main()+0x4222) [0x55a21df85462]<br /> stderr: 11: (_<em>libc_start_main()+0xe7) [0x7fb9452b7b97]<br /> stderr: 12: (_start()+0x2a) [0x55a21e04e95a]<br /> stderr: NOTE: a copy of the executable, or `objdump -rdS &lt;executable&gt;` is needed to interpret this.<br /> stderr: 2019-02-18 14:33:07.157 7fb9508d5240 -1 /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fb9508d5240 time 2019-02-18 14:33:07.155877<br /> stderr: /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: 821: FAILED assert((uint64_t)r len)<br /> stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)<br /> stderr: 1: (ceph::</em>_ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7fb947cf53e2]<br /> stderr: 2: (()+0x26d5a7) [0x7fb947cf55a7]<br /> stderr: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::list*, IOContext*, bool)+0x4a7) [0x55a21e5d4817]<br /> stderr: 4: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x435) [0x55a21e5945c5]<br /> stderr: 5: (BlueFS::_replay(bool, bool)+0x214) [0x55a21e59a434]<br /> stderr: 6: (BlueFS::mount()+0x1f1) [0x55a21e59ec81]<br /> stderr: 7: (BlueStore::_open_db(bool, bool)+0x17cd) [0x55a21e4c504d]<br /> stderr: 8: (BlueStore::mkfs()+0x805) [0x55a21e4f5fe5]<br /> stderr: 9: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x55a21e09e480]<br /> stderr: 10: (main()+0x4222) [0x55a21df85462]<br /> stderr: 11: (_<em>libc_start_main()+0xe7) [0x7fb9452b7b97]<br /> stderr: 12: (_start()+0x2a) [0x55a21e04e95a]<br /> stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.<br /> stderr: -25> 2019-02-18 14:33:07.093 7fb9508d5240 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid<br /> stderr: 0> 2019-02-18 14:33:07.157 7fb9508d5240 -1 /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: In function 'virtual int KernelDevice::read(uint64_t, uint64_t, ceph::bufferlist*, IOContext*, bool)' thread 7fb9508d5240 time 2019-02-18 14:33:07.155877<br /> stderr: /build/ceph-13.2.4/src/os/bluestore/KernelDevice.cc: 821: FAILED assert((uint64_t)r == len)<br /> stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)<br /> stderr: 1: (ceph::</em>_ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7fb947cf53e2]<br /> stderr: 2: (()+0x26d5a7) [0x7fb947cf55a7]<br /> stderr: 3: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::list*, IOContext*, bool)+0x4a7) [0x55a21e5d4817]<br /> stderr: 4: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x435) [0x55a21e5945c5]<br /> stderr: 5: (BlueFS::_replay(bool, bool)+0x214) [0x55a21e59a434]<br /> stderr: 6: (BlueFS::mount()+0x1f1) [0x55a21e59ec81]<br /> stderr: 7: (BlueStore::_open_db(bool, bool)+0x17cd) [0x55a21e4c504d]<br /> stderr: 8: (BlueStore::mkfs()+0x805) [0x55a21e4f5fe5]<br /> stderr: 9: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x55a21e09e480]<br /> stderr: 10: (main()+0x4222) [0x55a21df85462]<br /> stderr: 11: (_<em>libc_start_main()+0xe7) [0x7fb9452b7b97]<br /> stderr: 12: (_start()+0x2a) [0x55a21e04e95a]<br /> stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.<br /> stderr: <b>* Caught signal (Aborted) *<strong><br /> stderr: in thread 7fb9508d5240 thread_name:ceph-osd<br /> stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)<br /> stderr: 1: (()+0x92aa40) [0x55a21e5e5a40]<br /> stderr: 2: (()+0x12890) [0x7fb9463f9890]<br /> stderr: 3: (gsignal()+0xc7) [0x7fb9452d4e97]<br /> stderr: 4: (abort()+0x141) [0x7fb9452d6801]<br /> stderr: 5: (ceph::</em>_ceph_assert_fail(char const</strong>, char const*, int, char const*)+0x250) [0x7fb947cf5530]<br /> stderr: 6: (()+0x26d5a7) [0x7fb947cf55a7]<br /> stderr: 7: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::list*, IOContext*, bool)+0x4a7) [0x55a21e5d4817]<br /> stderr: 8: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x435) [0x55a21e5945c5]<br /> stderr: 9: (BlueFS::_replay(bool, bool)+0x214) [0x55a21e59a434]<br /> stderr: 10: (BlueFS::mount()+0x1f1) [0x55a21e59ec81]<br /> stderr: 11: (BlueStore::_open_db(bool, bool)+0x17cd) [0x55a21e4c504d]<br /> stderr: 12: (BlueStore::mkfs()+0x805) [0x55a21e4f5fe5]<br /> stderr: 13: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x55a21e09e480]<br /> stderr: 14: (main()+0x4222) [0x55a21df85462]<br /> stderr: 15: (_<em>libc_start_main()+0xe7) [0x7fb9452b7b97]<br /> stderr: 16: (_start()+0x2a) [0x55a21e04e95a]<br /> stderr: 2019-02-18 14:33:07.157 7fb9508d5240 -1 <strong></b> Caught signal (Aborted) <b><br /> stderr: in thread 7fb9508d5240 thread_name:ceph-osd<br /> stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)<br /> stderr: 1: (()+0x92aa40) [0x55a21e5e5a40]<br /> stderr: 2: (()+0x12890) [0x7fb9463f9890]<br /> stderr: 3: (gsignal()+0xc7) [0x7fb9452d4e97]<br /> stderr: 4: (abort()+0x141) [0x7fb9452d6801]<br /> stderr: 5: (ceph::</em>_ceph_assert_fail(char const</strong>, char const*, int, char const*)+0x250) [0x7fb947cf5530]<br /> stderr: 6: (()+0x26d5a7) [0x7fb947cf55a7]<br /> stderr: 7: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::list*, IOContext*, bool)+0x4a7) [0x55a21e5d4817]<br /> stderr: 8: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x435) [0x55a21e5945c5]<br /> stderr: 9: (BlueFS::_replay(bool, bool)+0x214) [0x55a21e59a434]<br /> stderr: 10: (BlueFS::mount()+0x1f1) [0x55a21e59ec81]<br /> stderr: 11: (BlueStore::_open_db(bool, bool)+0x17cd) [0x55a21e4c504d]<br /> stderr: 12: (BlueStore::mkfs()+0x805) [0x55a21e4f5fe5]<br /> stderr: 13: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x55a21e09e480]<br /> stderr: 14: (main()+0x4222) [0x55a21df85462]<br /> stderr: 15: (_<em>libc_start_main()+0xe7) [0x7fb9452b7b97]<br /> stderr: 16: (_start()+0x2a) [0x55a21e04e95a]<br /> stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.<br /> stderr: 0> 2019-02-18 14:33:07.157 7fb9508d5240 -1 <strong></b> Caught signal (Aborted) *</strong><br /> stderr: in thread 7fb9508d5240 thread_name:ceph-osd<br /> stderr: ceph version 13.2.4 (b10be4d44915a4d78a8e06aa31919e74927b142e) mimic (stable)<br /> stderr: 1: (()+0x92aa40) [0x55a21e5e5a40]<br /> stderr: 2: (()+0x12890) [0x7fb9463f9890]<br /> stderr: 3: (gsignal()+0xc7) [0x7fb9452d4e97]<br /> stderr: 4: (abort()+0x141) [0x7fb9452d6801]<br /> stderr: 5: (ceph::</em>_ceph_assert_fail(char const*, char const*, int, char const*)+0x250) [0x7fb947cf5530]<br /> stderr: 6: (()+0x26d5a7) [0x7fb947cf55a7]<br /> stderr: 7: (KernelDevice::read(unsigned long, unsigned long, ceph::buffer::list*, IOContext*, bool)+0x4a7) [0x55a21e5d4817]<br /> stderr: 8: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0x435) [0x55a21e5945c5]<br /> stderr: 9: (BlueFS::_replay(bool, bool)+0x214) [0x55a21e59a434]<br /> stderr: 10: (BlueFS::mount()+0x1f1) [0x55a21e59ec81]<br /> stderr: 11: (BlueStore::_open_db(bool, bool)+0x17cd) [0x55a21e4c504d]<br /> stderr: 12: (BlueStore::mkfs()+0x805) [0x55a21e4f5fe5]<br /> stderr: 13: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, uuid_d, int)+0x1b0) [0x55a21e09e480]<br /> stderr: 14: (main()+0x4222) [0x55a21df85462]<br /> stderr: 15: (__libc_start_main()+0xe7) [0x7fb9452b7b97]<br /> stderr: 16: (_start()+0x2a) [0x55a21e04e95a]<br /> stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.<br />--> Was unable to complete a new OSD, will rollback changes<br />Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it<br /> stderr: purged osd.0<br />--> RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid a87c3a87-cf22-41df-af4b-c971ed4c0e1a --setuser ceph --setgroup ceph</p>
rgw - Feature #20235 (Fix Under Review): vstart.sh fails to start more than one radosgw process
https://tracker.ceph.com/issues/20235
2017-06-09T12:56:17Z
Jens Harbott
j.harbott@x-ion.de
<p>When running something like</p>
<pre>
MON=1 OSD=3 MDS=0 RGW=2 ../src/vstart.sh -d -n -x
</pre>
<p>all the radosgw processes try to listen on the same port, so only one of them starts successfully. It would be nice if vstart.sh would instead start them on different ports.</p>
rgw - Cleanup #19851 (In Progress): Move AES_256_CTR to auth/Crypto for others to reuse
https://tracker.ceph.com/issues/19851
2017-05-04T02:55:43Z
Jos Collin
<p>The following warning was introduced by Adam Kupczyk. So creating a tracker for implementing the changes suggested by Adam Kupczyk.</p>
<p>ceph/src/rgw/rgw_crypt.cc:38:2: warning: #warning "TODO: move this code to auth/Crypto for others to reuse." [-Wcpp]<br /> #warning "TODO: move this code to auth/Crypto for others to reuse." <br /> ^<sub>~~~</sub>~<br />ceph/src/rgw/rgw_crypt.cc:247:2: warning: #warning "TODO: use auth/Crypto instead of reimplementing." [-Wcpp]<br /> #warning "TODO: use auth/Crypto instead of reimplementing." <br /> ^<sub>~~~</sub>~</p>