Project

General

Profile

Bug #22435

fail to create bluestore osd with ceph-volume command on ubuntu 14.04 with ceph 12.2.2

Added by Bruce Jiang over 3 years ago. Updated over 3 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Target version:
% Done:

0%

Source:
Community (user)
Tags:
Ubuntu14.04;
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-ansible
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Tried to create a bluestore osd manually with ceph-volume tool on a Ubuntu 14.04 system, but with no luck. The Ceph version is Luminous 12.2.2.

Several Linux commands' parameter or output format used by ceph-volume are not compatible with Ubuntu 14.04:

1. lvscreate command doesn't support "--yes" parameter
2. blkid command doesn't show PARTUUID

After modified ceph-volume code to bypass these problem
1. remove "--yes" parameter
2. trucated "sgdisk" output as PARTUUID

The modified version ceph-volume still failed. When ceph-volume tried to run this command, ceph raised a core dump:
sudo ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 96 --monmap /var/lib/ceph/osd/ceph-96/activate.monmap --key ******************************** --bluestore-block-wal-path /dev/nvme0n1p1 --bluestore-block-db-path /dev/nvme0n1p1 --osd-data /var/lib/ceph/osd/ceph-96/ --osd-uuid 0e30f7f8-3fef-4a91-b748-443b55fbb1d6 --setuser ceph --setgroup ceph

The command that trigger the problem is as following, this command was ran on OSD host:
$ sudo ceph-volume lvm prepare --bluestore --data /dev/sdy --block.wal /dev/nvme0n1p1 --block.db /dev/nvme0n1p1

ceph-volume.log output is uploaded in attachment. The following is the start part of error output:

[2017-12-13 20:49:50,735][ceph_volume.process][INFO ] Running command: sudo ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 96 --monmap /var/lib/ceph/osd/ceph-96/activate.monmap --key ******************************** --bluestore-block-wal-path /dev/nvme0n1p1 --bluestore-block-db-path /dev/nvme0n1p1 --osd-data /var/lib/ceph/osd/ceph-96/ --osd-uuid 0e30f7f8-3fef-4a91-b748-443b55fbb1d6 --setuser ceph --setgroup ceph
[2017-12-13 20:49:50,763][ceph_volume.process][INFO ] stderr 2017-12-13 20:49:50.763027 7f2789e87d00 -1 bluestore(/var/lib/ceph/osd/ceph-96//block) read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding
[2017-12-13 20:49:50,763][ceph_volume.process][INFO ] stderr 2017-12-13 20:49:50.763233 7f2789e87d00 -1 bluestore(/var/lib/ceph/osd/ceph-96//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding
[2017-12-13 20:49:50,763][ceph_volume.process][INFO ] stderr 2017-12-13 20:49:50.763393 7f2789e87d00 -1 bluestore(/var/lib/ceph/osd/ceph-96//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::iterator&) decode past end of struct encoding
2017-12-13 20:49:50.763452 7f2789e87d00 -1 bluestore(/var/lib/ceph/osd/ceph-96/) _read_fsid unparsable uuid
[2017-12-13 20:49:51,784][ceph_volume.process][INFO ] stderr /build/ceph-12.2.2/src/os/bluestore/StupidAllocator.cc: In function 'virtual void StupidAllocator::init_rm_free(uint64_t, uint64_t)' thread 7f2789e87d00 time 2017-12-13 20:49:51.784626
/build/ceph-12.2.2/src/os/bluestore/StupidAllocator.cc: 300: FAILED assert(rm.empty())
[2017-12-13 20:49:51,787][ceph_volume.process][INFO ] stderr ceph version 12.2.2 (cf0baeeeeba3b47f9427c6c97e2144b094b7e5ba) luminous (stable)
1: (ceph::
_ceph_assert_fail(char const*, char const*, int, char const*)+0x10e) [0x55cb5ba7d82e]
2: (StupidAllocator::init_rm_free(unsigned long, unsigned long)+0x291a) [0x55cb5ba2e66a]
3: (BlueFS::mount()+0x275) [0x55cb5ba0f9c5]
4: (BlueStore::_open_db(bool)+0x1680) [0x55cb5b92d2a0]
5: (BlueStore::_fsck(bool, bool)+0x3ca) [0x55cb5b95ebba]
6: (BlueStore::mkfs()+0xf20) [0x55cb5b936480]
7: (OSD::mkfs(CephContext*, ObjectStore*, std::string const&, uuid_d, int)+0x2a5) [0x55cb5b5299a5]
8: (main()+0x114e) [0x55cb5b441b1e]
9: (__libc_start_main()+0xf5) [0x7f27873b1f45]
10: (()+0x4b36d6) [0x55cb5b4e16d6]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2017-12-13 20:49:51.787609 7f2789e87d00 -1 /build/ceph-12.2.2/src/os/bluestore/StupidAllocator.cc: In function 'virtual void StupidAllocator::init_rm_free(uint64_t, uint64_t)' thread 7f2789e87d00 time 2017-12-13 20:49:51.784626

ceph-volume.log View (25.1 KB) Bruce Jiang, 12/13/2017 05:01 PM

History

#1 Updated by Alfredo Deza over 3 years ago

  • Status changed from New to Rejected

Unfortunately, there is no way to make this work with ceph-volume. We are testing with ceph-ansible which also doesn't support Trusty.

On top of the problems you found, ceph-volume relies exclusively on systemd. And there is just no systemd capabilities in Trusty, so even if blkid/lsblk were able to work here, the OSD would not come up.

The alternatives here are either to keep using ceph-disk, or to upgrade to Xenial or CentOS7

We can't really fix this correctly for Trusty.

#2 Updated by Bruce Jiang over 3 years ago

Alfredo,

Thanks for you reply. I understand that, but, from documents from http://www.ceph.com, it says Luminous release is tested under Ubuntu 14.04 comprehensively.

http://docs.ceph.com/docs/master/start/os-recommendations/

Ubuntu 14.04 Trusty Tahr linux-3.13.0 B, I, C

If ceph-volume can't work on Ubuntu 14.04, this may be a little misleading.

BR,
Bruce J.

Also available in: Atom PDF