root@ac1:~/mycluster# ceph-deploy osd create --bluestore --data /dev/sdg ceph4 [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy osd create --bluestore --data /dev/sdg ceph4 [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] bluestore : True [ceph_deploy.cli][INFO ] cd_conf : [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] fs_type : xfs [ceph_deploy.cli][INFO ] block_wal : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] journal : None [ceph_deploy.cli][INFO ] subcommand : create [ceph_deploy.cli][INFO ] host : ceph4 [ceph_deploy.cli][INFO ] filestore : None [ceph_deploy.cli][INFO ] func : [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] zap_disk : False [ceph_deploy.cli][INFO ] data : /dev/sdg [ceph_deploy.cli][INFO ] block_db : None [ceph_deploy.cli][INFO ] dmcrypt : False [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] dmcrypt_key_dir : /etc/ceph/dmcrypt-keys [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] debug : False [ceph_deploy.osd][DEBUG ] Creating OSD on cluster ceph with data device /dev/sdg Password: Password: [ceph4][DEBUG ] connected to host: ceph4 [ceph4][DEBUG ] detect platform information from remote host [ceph4][DEBUG ] detect machine type [ceph4][DEBUG ] find the location of an executable [ceph_deploy.osd][INFO ] Distro info: Ubuntu 16.04 xenial [ceph_deploy.osd][DEBUG ] Deploying osd to ceph4 [ceph4][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph4][DEBUG ] find the location of an executable [ceph4][INFO ] Running command: /usr/sbin/ceph-volume --cluster ceph lvm create --bluestore --data /dev/sdg [ceph4][DEBUG ] Running command: /usr/bin/ceph-authtool --gen-print-key [ceph4][DEBUG ] Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 2aef5735-543b-4f0c-9974-bf5efa1a41f2 [ceph4][DEBUG ] Running command: vgcreate --force --yes ceph-b3e24831-a357-436b-a982-16a00cb6c849 /dev/sdg [ceph4][DEBUG ] stdout: Physical volume "/dev/sdg" successfully created [ceph4][DEBUG ] stdout: Volume group "ceph-b3e24831-a357-436b-a982-16a00cb6c849" successfully created [ceph4][DEBUG ] Running command: lvcreate --yes -l 100%FREE -n osd-block-2aef5735-543b-4f0c-9974-bf5efa1a41f2 ceph-b3e24831-a357-436b-a982-16a00cb6c849 [ceph4][DEBUG ] stdout: Wiping VMFS_volume_member signature on /dev/ceph-b3e24831-a357-436b-a982-16a00cb6c849/osd-block-2aef5735-543b-4f0c-9974-bf5efa1a41f2. [ceph4][DEBUG ] stdout: Logical volume "osd-block-2aef5735-543b-4f0c-9974-bf5efa1a41f2" created. [ceph4][DEBUG ] Running command: /usr/bin/ceph-authtool --gen-print-key [ceph4][DEBUG ] Running command: mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 [ceph4][DEBUG ] --> Absolute path not found for executable: restorecon [ceph4][DEBUG ] --> Ensure $PATH environment variable contains common executable locations [ceph4][DEBUG ] Running command: chown -h ceph:ceph /dev/ceph-b3e24831-a357-436b-a982-16a00cb6c849/osd-block-2aef5735-543b-4f0c-9974-bf5efa1a41f2 [ceph4][DEBUG ] Running command: chown -R ceph:ceph /dev/dm-0 [ceph4][DEBUG ] Running command: ln -s /dev/ceph-b3e24831-a357-436b-a982-16a00cb6c849/osd-block-2aef5735-543b-4f0c-9974-bf5efa1a41f2 /var/lib/ceph/osd/ceph-0/block [ceph4][DEBUG ] Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap [ceph4][DEBUG ] stderr: got monmap epoch 2 [ceph4][DEBUG ] Running command: ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQD7Y4ZcCY6WCxAAvHl7o5INk8UPONL7B/d2FA== [ceph4][DEBUG ] stdout: creating /var/lib/ceph/osd/ceph-0/keyring [ceph4][DEBUG ] added entity osd.0 auth auth(auid = 18446744073709551615 key=AQD7Y4ZcCY6WCxAAvHl7o5INk8UPONL7B/d2FA== with 0 caps) [ceph4][DEBUG ] Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring [ceph4][DEBUG ] Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ [ceph4][DEBUG ] Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 2aef5735-543b-4f0c-9974-bf5efa1a41f2 --setuser ceph --setgroup ceph [ceph4][DEBUG ] stderr: 2019-03-11 14:34:54.820641 7f5fb957ce00 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid [ceph4][DEBUG ] stderr: /build/ceph-12.2.11/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f5fb957ce00 time 2019-03-11 14:34:54.884072 [ceph4][DEBUG ] stderr: /build/ceph-12.2.11/src/os/bluestore/BlueFS.cc: 1000: FAILED assert(r == 0) [ceph4][DEBUG ] stderr: ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable) [ceph4][DEBUG ] stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55d9d68a3b32] [ceph4][DEBUG ] stderr: 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xf6a) [0x55d9d680fdfa] [ceph4][DEBUG ] stderr: 3: (BlueFS::_replay(bool)+0x242) [0x55d9d68182b2] [ceph4][DEBUG ] stderr: 4: (BlueFS::mount()+0x209) [0x55d9d681c4a9] [ceph4][DEBUG ] stderr: 5: (BlueStore::_open_db(bool)+0x169c) [0x55d9d672baec] [ceph4][DEBUG ] stderr: 6: (BlueStore::mkfs()+0x106d) [0x55d9d67614bd] [ceph4][DEBUG ] stderr: 7: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string, std::allocator > const&, uuid_d, int)+0x164) [0x55d9d62990d4] [ceph4][DEBUG ] stderr: 8: (main()+0x11aa) [0x55d9d61bba0a] [ceph4][DEBUG ] stderr: 9: (__libc_start_main()+0xf0) [0x7f5fb67dc830] [ceph4][DEBUG ] stderr: 10: (_start()+0x29) [0x55d9d624aeb9] [ceph4][DEBUG ] stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. [ceph4][DEBUG ] stderr: 2019-03-11 14:34:54.889986 7f5fb957ce00 -1 /build/ceph-12.2.11/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f5fb957ce00 time 2019-03-11 14:34:54.884072 [ceph4][DEBUG ] stderr: /build/ceph-12.2.11/src/os/bluestore/BlueFS.cc: 1000: FAILED assert(r == 0) [ceph4][DEBUG ] stderr: ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable) [ceph4][DEBUG ] stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55d9d68a3b32] [ceph4][DEBUG ] stderr: 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xf6a) [0x55d9d680fdfa] [ceph4][DEBUG ] stderr: 3: (BlueFS::_replay(bool)+0x242) [0x55d9d68182b2] [ceph4][DEBUG ] stderr: 4: (BlueFS::mount()+0x209) [0x55d9d681c4a9] [ceph4][DEBUG ] stderr: 5: (BlueStore::_open_db(bool)+0x169c) [0x55d9d672baec] [ceph4][DEBUG ] stderr: 6: (BlueStore::mkfs()+0x106d) [0x55d9d67614bd] [ceph4][DEBUG ] stderr: 7: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string, std::allocator > const&, uuid_d, int)+0x164) [0x55d9d62990d4] [ceph4][DEBUG ] stderr: 8: (main()+0x11aa) [0x55d9d61bba0a] [ceph4][DEBUG ] stderr: 9: (__libc_start_main()+0xf0) [0x7f5fb67dc830] [ceph4][DEBUG ] stderr: 10: (_start()+0x29) [0x55d9d624aeb9] [ceph4][DEBUG ] stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. [ceph4][DEBUG ] stderr: -15> 2019-03-11 14:34:54.820641 7f5fb957ce00 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid [ceph4][DEBUG ] stderr: 0> 2019-03-11 14:34:54.889986 7f5fb957ce00 -1 /build/ceph-12.2.11/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, uint64_t, size_t, ceph::bufferlist*, char*)' thread 7f5fb957ce00 time 2019-03-11 14:34:54.884072 [ceph4][DEBUG ] stderr: /build/ceph-12.2.11/src/os/bluestore/BlueFS.cc: 1000: FAILED assert(r == 0) [ceph4][DEBUG ] stderr: ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable) [ceph4][DEBUG ] stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x55d9d68a3b32] [ceph4][DEBUG ] stderr: 2: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xf6a) [0x55d9d680fdfa] [ceph4][DEBUG ] stderr: 3: (BlueFS::_replay(bool)+0x242) [0x55d9d68182b2] [ceph4][DEBUG ] stderr: 4: (BlueFS::mount()+0x209) [0x55d9d681c4a9] [ceph4][DEBUG ] stderr: 5: (BlueStore::_open_db(bool)+0x169c) [0x55d9d672baec] [ceph4][DEBUG ] stderr: 6: (BlueStore::mkfs()+0x106d) [0x55d9d67614bd] [ceph4][DEBUG ] stderr: 7: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string, std::allocator > const&, uuid_d, int)+0x164) [0x55d9d62990d4] [ceph4][DEBUG ] stderr: 8: (main()+0x11aa) [0x55d9d61bba0a] [ceph4][DEBUG ] stderr: 9: (__libc_start_main()+0xf0) [0x7f5fb67dc830] [ceph4][DEBUG ] stderr: 10: (_start()+0x29) [0x55d9d624aeb9] [ceph4][DEBUG ] stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. [ceph4][DEBUG ] stderr: *** Caught signal (Aborted) ** [ceph4][DEBUG ] stderr: in thread 7f5fb957ce00 thread_name:ceph-osd [ceph4][DEBUG ] stderr: ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable) [ceph4][DEBUG ] stderr: 1: (()+0xaa0214) [0x55d9d6860214] [ceph4][DEBUG ] stderr: 2: (()+0x11390) [0x7f5fb7856390] [ceph4][DEBUG ] stderr: 3: (gsignal()+0x38) [0x7f5fb67f1428] [ceph4][DEBUG ] stderr: 4: (abort()+0x16a) [0x7f5fb67f302a] [ceph4][DEBUG ] stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x55d9d68a3cbe] [ceph4][DEBUG ] stderr: 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xf6a) [0x55d9d680fdfa] [ceph4][DEBUG ] stderr: 7: (BlueFS::_replay(bool)+0x242) [0x55d9d68182b2] [ceph4][DEBUG ] stderr: 8: (BlueFS::mount()+0x209) [0x55d9d681c4a9] [ceph4][DEBUG ] stderr: 9: (BlueStore::_open_db(bool)+0x169c) [0x55d9d672baec] [ceph4][DEBUG ] stderr: 10: (BlueStore::mkfs()+0x106d) [0x55d9d67614bd] [ceph4][DEBUG ] stderr: 11: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string, std::allocator > const&, uuid_d, int)+0x164) [0x55d9d62990d4] [ceph4][DEBUG ] stderr: 12: (main()+0x11aa) [0x55d9d61bba0a] [ceph4][DEBUG ] stderr: 13: (__libc_start_main()+0xf0) [0x7f5fb67dc830] [ceph4][DEBUG ] stderr: 14: (_start()+0x29) [0x55d9d624aeb9] [ceph4][DEBUG ] stderr: 2019-03-11 14:34:54.897043 7f5fb957ce00 -1 *** Caught signal (Aborted) ** [ceph4][DEBUG ] stderr: in thread 7f5fb957ce00 thread_name:ceph-osd [ceph4][DEBUG ] stderr: ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable) [ceph4][DEBUG ] stderr: 1: (()+0xaa0214) [0x55d9d6860214] [ceph4][DEBUG ] stderr: 2: (()+0x11390) [0x7f5fb7856390] [ceph4][DEBUG ] stderr: 3: (gsignal()+0x38) [0x7f5fb67f1428] [ceph4][DEBUG ] stderr: 4: (abort()+0x16a) [0x7f5fb67f302a] [ceph4][DEBUG ] stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x55d9d68a3cbe] [ceph4][WARNIN] --> RuntimeError: Command failed with exit code -6: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 2aef5735-543b-4f0c-9974-bf5efa1a41f2 --setuser ceph --setgroup ceph [ceph4][DEBUG ] stderr: 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xf6a) [0x55d9d680fdfa] [ceph4][DEBUG ] stderr: 7: (BlueFS::_replay(bool)+0x242) [0x55d9d68182b2] [ceph4][DEBUG ] stderr: 8: (BlueFS::mount()+0x209) [0x55d9d681c4a9] [ceph4][DEBUG ] stderr: 9: (BlueStore::_open_db(bool)+0x169c) [0x55d9d672baec] [ceph4][DEBUG ] stderr: 10: (BlueStore::mkfs()+0x106d) [0x55d9d67614bd] [ceph4][DEBUG ] stderr: 11: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string, std::allocator > const&, uuid_d, int)+0x164) [0x55d9d62990d4] [ceph4][DEBUG ] stderr: 12: (main()+0x11aa) [0x55d9d61bba0a] [ceph4][DEBUG ] stderr: 13: (__libc_start_main()+0xf0) [0x7f5fb67dc830] [ceph4][DEBUG ] stderr: 14: (_start()+0x29) [0x55d9d624aeb9] [ceph4][DEBUG ] stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. [ceph4][DEBUG ] stderr: 0> 2019-03-11 14:34:54.897043 7f5fb957ce00 -1 *** Caught signal (Aborted) ** [ceph4][DEBUG ] stderr: in thread 7f5fb957ce00 thread_name:ceph-osd [ceph4][DEBUG ] stderr: ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable) [ceph4][DEBUG ] stderr: 1: (()+0xaa0214) [0x55d9d6860214] [ceph4][DEBUG ] stderr: 2: (()+0x11390) [0x7f5fb7856390] [ceph4][DEBUG ] stderr: 3: (gsignal()+0x38) [0x7f5fb67f1428] [ceph4][DEBUG ] stderr: 4: (abort()+0x16a) [0x7f5fb67f302a] [ceph4][DEBUG ] stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x28e) [0x55d9d68a3cbe] [ceph4][DEBUG ] stderr: 6: (BlueFS::_read(BlueFS::FileReader*, BlueFS::FileReaderBuffer*, unsigned long, unsigned long, ceph::buffer::list*, char*)+0xf6a) [0x55d9d680fdfa] [ceph4][DEBUG ] stderr: 7: (BlueFS::_replay(bool)+0x242) [0x55d9d68182b2] [ceph4][DEBUG ] stderr: 8: (BlueFS::mount()+0x209) [0x55d9d681c4a9] [ceph4][DEBUG ] stderr: 9: (BlueStore::_open_db(bool)+0x169c) [0x55d9d672baec] [ceph4][DEBUG ] stderr: 10: (BlueStore::mkfs()+0x106d) [0x55d9d67614bd] [ceph4][DEBUG ] stderr: 11: (OSD::mkfs(CephContext*, ObjectStore*, std::__cxx11::basic_string, std::allocator > const&, uuid_d, int)+0x164) [0x55d9d62990d4] [ceph4][DEBUG ] stderr: 12: (main()+0x11aa) [0x55d9d61bba0a] [ceph4][DEBUG ] stderr: 13: (__libc_start_main()+0xf0) [0x7f5fb67dc830] [ceph4][DEBUG ] stderr: 14: (_start()+0x29) [0x55d9d624aeb9] [ceph4][DEBUG ] stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. [ceph4][DEBUG ] --> Was unable to complete a new OSD, will rollback changes [ceph4][DEBUG ] --> OSD will be fully purged from the cluster, because the ID was generated [ceph4][DEBUG ] Running command: ceph osd purge osd.0 --yes-i-really-mean-it [ceph4][DEBUG ] stderr: purged osd.0 [ceph4][ERROR ] RuntimeError: command returned non-zero exit status: 1 [ceph_deploy.osd][ERROR ] Failed to execute command: /usr/sbin/ceph-volume --cluster ceph lvm create --bluestore --data /dev/sdg [ceph_deploy][ERROR ] GenericError: Failed to create 1 OSDs