Project

General

Profile

Actions
Copy link

Bug #24480

closed

using a 1GB data device refuses provisioning with misleading error

Added by Alfredo Deza almost 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Igor Fedotov
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The logs (or CLI output) should be able to say that it is unable to complete due to insufficient sizing, not "No such file or directory" which is not accurate (as seen from the output below)

# lvcreate -L 1G -n ceph cah_foo
  Logical volume "ceph" created.

# ceph-volume lvm create --data cah_foo/ceph
Running command: /bin/ceph-authtool --gen-print-key
Running command: /bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new b8016385-e46c-4e93-a334-be4fc92bea85
Running command: /bin/ceph-authtool --gen-print-key
Running command: mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0
Running command: chown -R ceph:ceph /dev/dm-2
Running command: ln -s /dev/cah_foo/ceph /var/lib/ceph/osd/ceph-0/block
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap
 stderr: got monmap epoch 2
Running command: ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQCxuhlbAVylMRAAXsKQpKbau3T1rI66z651ng==
 stdout: creating /var/lib/ceph/osd/ceph-0/keyring
added entity osd.0 auth auth(auid = 18446744073709551615 key=AQCxuhlbAVylMRAAXsKQpKbau3T1rI66z651ng== with 0 caps)
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/
Running command: /bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid b8016385-e46c-4e93-a334-be4fc92bea85 --setuser ceph --setgroup ceph
 stderr: 2018-06-07 16:07:32.804440 7f237709dd80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid
 stderr: 2018-06-07 16:07:33.822761 7f237709dd80 -1 OSD::mkfs: ObjectStore::mkfs failed with error (2) No such file or directory
 stderr: 2018-06-07 16:07:33.822934 7f237709dd80 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0/: (2) No such file or directory
--> ceph-volume lvm prepare successful for: cah_foo/ceph
Running command: ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/cah_foo/ceph --path /var/lib/ceph/osd/ceph-0
 stderr: failed to read label for /dev/cah_foo/ceph: (2) No such file or directory
--> Was unable to complete a new OSD, will rollback changes
--> OSD will be fully purged from the cluster, because the ID was generated
Running command: ceph osd purge osd.0 --yes-i-really-mean-it
 stderr: purged osd.0
-->  RuntimeError: command returned non-zero exit status: 1

# ceph --version
ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)
Actions
Copy link
 #1

Updated by Igor Fedotov almost 6 years ago

  • Status changed from New to In Progress
  • Assignee set to Igor Fedotov
Actions
Copy link
 #2

Updated by Igor Fedotov almost 6 years ago

Alfredo, could you please collect OSD logs with increased log level: 'debug bluestore = 20'
I'm trying to reproduce the issue but getting an assert instead...

Actions
Copy link
 #3

Updated by Alfredo Deza almost 6 years ago 

root@node9:/home/vagrant# ceph-volume lvm create --data ceph-osd/small-data
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new e388f1ee-b686-43a5-a410-db5b9dc9ddf3
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1
Running command: chown -R ceph:ceph /dev/dm-23
Running command: ln -s /dev/ceph-osd/small-data /var/lib/ceph/osd/ceph-1/block
Running command: ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-1/activate.monmap
 stderr: got monmap epoch 1
Running command: ceph-authtool /var/lib/ceph/osd/ceph-1/keyring --create-keyring --name osd.1 --add-key AQCZOSFbW/VvIxAAIKrPXr7ZtnPXrrKd46HhkA==
 stdout: creating /var/lib/ceph/osd/ceph-1/keyring
 stdout: added entity osd.1 auth auth(auid = 18446744073709551615 key=AQCZOSFbW/VvIxAAIKrPXr7ZtnPXrrKd46HhkA== with 0 caps)
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/keyring
Running command: chown -R ceph:ceph /var/lib/ceph/osd/ceph-1/
Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 1 --monmap /var/lib/ceph/osd/ceph-1/activate.monmap --keyfile - --osd-data /var/lib/ceph/osd/ceph-1/ --osd-uuid e388f1ee-b686-43a5-a410-db5b9dc9ddf3 --setuser ceph --setgroup ceph
 stderr: 2018-06-13 18:35:05.750849 7f1f61c01e00 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid
 stderr: 2018-06-13 18:35:06.544599 7f1f61c01e00 -1 OSD::mkfs: ObjectStore::mkfs failed with error (2) No such file or directory
 stderr: 2018-06-13 18:35:06.544897 7f1f61c01e00 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-1/: (2) No such file or directory
--> ceph-volume lvm prepare successful for: ceph-osd/small-data
Running command: ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev /dev/ceph-osd/small-data --path /var/lib/ceph/osd/ceph-1
 stderr: failed to read label for
 stderr: /dev/ceph-osd/small-data
 stderr: :
 stderr: (2) No such file or directory
 stderr:
--> Was unable to complete a new OSD, will rollback changes
--> OSD will be fully purged from the cluster, because the ID was generated
Running command: ceph osd purge osd.1 --yes-i-really-mean-it
 stderr: purged osd.1
-->  RuntimeError: command returned non-zero exit status: 1

root@node9:/home/vagrant# cat /etc/ceph/ceph.conf | grep bluestore
debug bluestore = 20
Actions
Copy link
 #4

Updated by Igor Fedotov almost 6 years ago

I meant OSD log from /var/log/ceph/ceph-osd.*
The one you shared contains OSD stderr output only which is very limited.

Actions
Copy link
 #5

Updated by Alfredo Deza almost 6 years ago 

2018-06-13 21:02:08.359 7f1a93bdc240  0 set uid:gid to 64045:64045 (ceph:ceph)
2018-06-13 21:02:08.359 7f1a93bdc240  0 ceph version 14.0.0-376-g05d1e60 (05d1e60d2b6d4c94210c6b836be91dc235b24561) nautilus (dev), process ceph-osd, pid 4187
2018-06-13 21:02:08.359 7f1a93bdc240  0 pidfile_write: ignore empty --pid-file
2018-06-13 21:02:08.363 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) set_cache_shards 1
2018-06-13 21:02:08.363 7f1a93bdc240  1 bluestore(/var/lib/ceph/osd/ceph-1/) mkfs path /var/lib/ceph/osd/ceph-1/
2018-06-13 21:02:08.363 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label
2018-06-13 21:02:08.367 7f1a93bdc240  2 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::const_iterator&) decode past end of struct encoding
2018-06-13 21:02:08.367 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label
2018-06-13 21:02:08.367 7f1a93bdc240  2 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::const_iterator&) decode past end of struct encoding
2018-06-13 21:02:08.367 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label
2018-06-13 21:02:08.367 7f1a93bdc240  2 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::const_iterator&) decode past end of struct encoding
2018-06-13 21:02:08.367 7f1a93bdc240 -1 bluestore(/var/lib/ceph/osd/ceph-1/) _read_fsid unparsable uuid
2018-06-13 21:02:08.367 7f1a93bdc240  1 bluestore(/var/lib/ceph/osd/ceph-1/) mkfs using provided fsid 482e7b0c-39dd-4045-9759-661234670360
2018-06-13 21:02:08.367 7f1a93bdc240 20 bluestore(/var/lib/ceph/osd/ceph-1/) _setup_block_symlink_or_file name block path  size 10737418240 create=1
2018-06-13 21:02:08.367 7f1a93bdc240 20 bluestore(/var/lib/ceph/osd/ceph-1/) _setup_block_symlink_or_file name block.wal path  size 100663296 create=0
2018-06-13 21:02:08.367 7f1a93bdc240 20 bluestore(/var/lib/ceph/osd/ceph-1/) _setup_block_symlink_or_file name block.db path  size 0 create=0
2018-06-13 21:02:08.371 7f1a93bdc240  1 bdev create path /var/lib/ceph/osd/ceph-1//block type kernel
2018-06-13 21:02:08.371 7f1a93bdc240  1 bdev(0x55d872e90000 /var/lib/ceph/osd/ceph-1//block) open path /var/lib/ceph/osd/ceph-1//block
2018-06-13 21:02:08.371 7f1a93bdc240  1 bdev(0x55d872e90000 /var/lib/ceph/osd/ceph-1//block) open size 1073741824 (0x40000000, 1 GiB) block_size 4096 (4 KiB) rotational
2018-06-13 21:02:08.371 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _write_bdev_label path /var/lib/ceph/osd/ceph-1//block label bdev(osd_uuid 482e7b0c-39dd-4045-9759-661234670360, size 0x40000000, btime 2018-06-13 21:02:08.377775, desc main, 0 meta)
2018-06-13 21:02:08.371 7f1a93bdc240  1 bluestore(/var/lib/ceph/osd/ceph-1/) _set_cache_sizes cache_size 1073741824 meta 0.5 kv 0.5 data 0
2018-06-13 21:02:08.375 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) _open_db kv_backend = rocksdb
2018-06-13 21:02:08.375 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) _open_db do_bluefs = 1
2018-06-13 21:02:08.375 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) _open_db initializing bluefs
2018-06-13 21:02:08.375 7f1a93bdc240  1 bdev create path /var/lib/ceph/osd/ceph-1//block type kernel
2018-06-13 21:02:08.375 7f1a93bdc240  1 bdev(0x55d872e90380 /var/lib/ceph/osd/ceph-1//block) open path /var/lib/ceph/osd/ceph-1//block
2018-06-13 21:02:08.375 7f1a93bdc240  1 bdev(0x55d872e90380 /var/lib/ceph/osd/ceph-1//block) open size 1073741824 (0x40000000, 1 GiB) block_size 4096 (4 KiB) rotational
2018-06-13 21:02:08.375 7f1a93bdc240  1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-1//block size 1 GiB
2018-06-13 21:02:08.375 7f1a93bdc240  1 bluefs add_block_extent bdev 1 0x0~40000000
2018-06-13 21:02:08.375 7f1a93bdc240  1 bluefs mkfs osd_uuid 482e7b0c-39dd-4045-9759-661234670360
2018-06-13 21:02:08.375 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 init_add_free 0x0~40000000
2018-06-13 21:02:08.375 7f1a93bdc240  1 bluefs mkfs uuid 6c0e97d5-5b59-42d2-9ab7-767df6392461
2018-06-13 21:02:08.375 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 allocate_int want_size 0x400000 alloc_unit 0x100000 hint 0x0
2018-06-13 21:02:08.375 7f1a93bdc240  1 stupidalloc 0x0x55d872ab6b20 shutdown
2018-06-13 21:02:08.375 7f1a93bdc240  1 bluefs mount
2018-06-13 21:02:08.387 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 init_add_free 0x0~40000000
2018-06-13 21:02:08.387 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 init_rm_free 0x0~400000
2018-06-13 21:02:08.387 7f1a93bdc240 20 stupidalloc 0x0x55d872ab6b20 init_rm_free bin 9 rm 0x[0~400000]
2018-06-13 21:02:08.387 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) column family L:
2018-06-13 21:02:08.387 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) column family M:
2018-06-13 21:02:08.387 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) column family P:
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option compaction_readahead_size = 2097152
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option compression = kNoCompression
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option max_write_buffer_number = 4
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option min_write_buffer_number_to_merge = 1
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option recycle_log_file_num = 4
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option writable_file_max_buffer_size = 0
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option write_buffer_size = 268435456
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option compaction_readahead_size = 2097152
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option compression = kNoCompression
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option max_write_buffer_number = 4
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option min_write_buffer_number_to_merge = 1
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option recycle_log_file_num = 4
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option writable_file_max_buffer_size = 0
2018-06-13 21:02:08.387 7f1a93bdc240  0  set rocksdb option write_buffer_size = 268435456
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: RocksDB version: 5.14.0

2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: Git sha rocksdb_build_git_sha:@0@
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: Compile date Jun  9 2018
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: DB SUMMARY

2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: SST files in db dir, Total Num: 0, files:

2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: Write Ahead Log file in db:

2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                         Options.error_if_exists: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                       Options.create_if_missing: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                         Options.paranoid_checks: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                                     Options.env: 0x55d872e32d80
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                                Options.info_log: 0x55d872e35300
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                Options.max_file_opening_threads: 16
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                              Options.statistics: (nil)
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                               Options.use_fsync: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                       Options.max_log_file_size: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                  Options.max_manifest_file_size: 1073741824
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                   Options.log_file_time_to_roll: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                       Options.keep_log_file_num: 1000
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                    Options.recycle_log_file_num: 4
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                         Options.allow_fallocate: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                        Options.allow_mmap_reads: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                       Options.allow_mmap_writes: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                        Options.use_direct_reads: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                        Options.use_direct_io_for_flush_and_compaction: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:          Options.create_missing_column_families: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                              Options.db_log_dir:
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                                 Options.wal_dir: db
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                Options.table_cache_numshardbits: 6
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                      Options.max_subcompactions: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                  Options.max_background_flushes: -1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                         Options.WAL_ttl_seconds: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                       Options.WAL_size_limit_MB: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.manifest_preallocation_size: 4194304
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                     Options.is_fd_close_on_exec: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                   Options.advise_random_on_open: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                    Options.db_write_buffer_size: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                    Options.write_buffer_manager: 0x55d872e39450
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:         Options.access_hint_on_compaction_start: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:  Options.new_table_reader_for_compaction_inputs: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:           Options.random_access_max_buffer_size: 1048576
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                      Options.use_adaptive_mutex: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                            Options.rate_limiter: (nil)
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     Options.sst_file_manager.rate_bytes_per_sec: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                       Options.wal_recovery_mode: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                  Options.enable_thread_tracking: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                  Options.enable_pipelined_write: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:         Options.allow_concurrent_memtable_write: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:      Options.enable_write_thread_adaptive_yield: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.write_thread_max_yield_usec: 100
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:            Options.write_thread_slow_yield_usec: 3
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                               Options.row_cache: None
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                              Options.wal_filter: None
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.avoid_flush_during_recovery: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.allow_ingest_behind: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.preserve_deletes: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.two_write_queues: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.manual_wal_flush: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.max_background_jobs: 2
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.max_background_compactions: -1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.avoid_flush_during_shutdown: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:           Options.writable_file_max_buffer_size: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.delayed_write_rate : 16777216
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.max_total_wal_size: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:             Options.delete_obsolete_files_period_micros: 21600000000
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                   Options.stats_dump_period_sec: 600
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                          Options.max_open_files: -1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                          Options.bytes_per_sync: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:                      Options.wal_bytes_per_sync: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:       Options.compaction_readahead_size: 2097152
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: Compression algorithms supported:
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kZSTDNotFinalCompression supported: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kZSTD supported: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kXpressCompression supported: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kLZ4HCCompression supported: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kLZ4Compression supported: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kBZip2Compression supported: 0
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kZlibCompression supported: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb:     kSnappyCompression supported: 1
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: Fast CRC32 supported: Supported on x86
2018-06-13 21:02:08.387 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/db_impl_open.cc:222] Creating manifest 1

2018-06-13 21:02:08.387 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 allocate_int want_size 0x100000 alloc_unit 0x100000 hint 0x0
2018-06-13 21:02:08.387 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 allocate_int want_size 0x100000 alloc_unit 0x100000 hint 0x0
2018-06-13 21:02:08.391 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 allocate_int want_size 0x100000 alloc_unit 0x100000 hint 0x0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/version_set.cc:3096] Recovering from manifest file: MANIFEST-000001

2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/column_family.cc:475] --------------- Options for column family [default]:

2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:               Options.comparator: leveldb.BytewiseComparator
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:           Options.merge_operator: .T:int64_array.b:bitwise_xor
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:        Options.compaction_filter: None
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:        Options.compaction_filter_factory: None
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:         Options.memtable_factory: SkipListFactory
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:            Options.table_factory: BlockBasedTable
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:            table_factory options:   flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d872a89450)
  cache_index_and_filter_blocks: 1
  cache_index_and_filter_blocks_with_high_priority: 1
  pin_l0_filter_and_index_blocks_in_cache: 1
  index_type: 0
  hash_index_allow_collision: 1
  checksum: 1
  no_block_cache: 0
  block_cache: 0x55d872ac6910
  block_cache_name: LRUCache
  block_cache_options:
    capacity : 536870912
    num_shard_bits : 4
    strict_capacity_limit : 0
    high_pri_pool_ratio: 0.000
  block_cache_compressed: (nil)
  persistent_cache: (nil)
  block_size: 4096
  block_size_deviation: 10
  block_restart_interval: 16
  index_block_restart_interval: 1
  metadata_block_size: 4096
  partition_filters: 0
  use_delta_encoding: 1
  filter_policy: rocksdb.BuiltinBloomFilter
  whole_key_filtering: 1
  verify_compression: 0
  read_amp_bytes_per_bit: 0
  format_version: 2
  enable_index_compression: 1
  block_align: 0

2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:        Options.write_buffer_size: 268435456
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:  Options.max_write_buffer_number: 4
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:          Options.compression: NoCompression
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                  Options.bottommost_compression: Disabled
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:       Options.prefix_extractor: nullptr
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:   Options.memtable_insert_with_hint_prefix_extractor: nullptr
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:             Options.num_levels: 7
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:        Options.min_write_buffer_number_to_merge: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:     Options.max_write_buffer_number_to_maintain: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:            Options.compression_opts.window_bits: -14
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                  Options.compression_opts.level: 32767
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:               Options.compression_opts.strategy: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:         Options.compression_opts.max_dict_bytes: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:         Options.compression_opts.zstd_max_train_bytes: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:      Options.level0_file_num_compaction_trigger: 4
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:          Options.level0_slowdown_writes_trigger: 20
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:              Options.level0_stop_writes_trigger: 36
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                   Options.target_file_size_base: 67108864
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:             Options.target_file_size_multiplier: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                Options.max_bytes_for_level_base: 268435456
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:          Options.max_bytes_for_level_multiplier: 10.000000
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:       Options.max_sequential_skip_in_iterations: 8
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                    Options.max_compaction_bytes: 1677721600
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                        Options.arena_block_size: 33554432
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:   Options.soft_pending_compaction_bytes_limit: 68719476736
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:   Options.hard_pending_compaction_bytes_limit: 274877906944
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:       Options.rate_limit_delay_max_milliseconds: 100
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                Options.disable_auto_compactions: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                        Options.compaction_style: kCompactionStyleLevel
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                          Options.compaction_pri: kByCompensatedSize
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_universal.size_ratio: 1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: Options.compaction_options_fifo.ttl: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                   Options.table_properties_collectors:
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                   Options.inplace_update_support: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                 Options.inplace_update_num_locks: 10000
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:               Options.memtable_prefix_bloom_size_ratio: 0.000000
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:   Options.memtable_huge_page_size: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                           Options.bloom_locality: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                    Options.max_successive_merges: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                Options.optimize_filters_for_hits: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                Options.paranoid_file_checks: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                Options.force_consistency_checks: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                Options.report_bg_io_stats: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb:                               Options.ttl: 0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/version_set.cc:3374] Recovered from manifest file:db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0

2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/version_set.cc:3382] Column family [default] (ID 0), log number is 0

2018-06-13 21:02:08.391 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 allocate_int want_size 0x100000 alloc_unit 0x100000 hint 0x0
2018-06-13 21:02:08.391 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/db_impl_open.cc:1219] DB pointer 0x55d872cce000
2018-06-13 21:02:08.391 7f1a93bdc240  1 bluestore(/var/lib/ceph/osd/ceph-1/) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152
2018-06-13 21:02:08.391 7f1a93bdc240 20 bluestore(/var/lib/ceph/osd/ceph-1/) _open_fm initializing freespace
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist _init_misc bytes_per_key 0x800000, key_mask 0xffffffffff800000
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist create size 0x40000000 bytes_per_block 0x10000 blocks 0x4000 blocks_per_key 0x80
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist allocate 0x0~10000
2018-06-13 21:02:08.391 7f1a93bdc240 20 freelist _xor first_key 0x0 last_key 0x0
2018-06-13 21:02:08.391 7f1a93bdc240 20 bluestore(/var/lib/ceph/osd/ceph-1/) _open_fm reserved 0x40000000 for bluefs
2018-06-13 21:02:08.391 7f1a93bdc240 20 bluestore(/var/lib/ceph/osd/ceph-1/) _open_fm bluefs_extents 0x[0~40000000]
2018-06-13 21:02:08.391 7f1a93bdc240 10 stupidalloc 0x0x55d872ab6b20 allocate_int want_size 0x100000 alloc_unit 0x100000 hint 0x0
2018-06-13 21:02:08.391 7f1a93bdc240  1 freelist init
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist init blocks 0x4000
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist init blocks_per_key 0x80
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist init bytes_per_block 0x10000
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist init size 0x40000000
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist init size 0x40000000 bytes_per_block 0x10000 blocks 0x4000 blocks_per_key 0x80
2018-06-13 21:02:08.391 7f1a93bdc240 10 freelist _init_misc bytes_per_key 0x800000, key_mask 0xffffffffff800000
2018-06-13 21:02:08.391 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) _prepare_ondisk_format_super ondisk_format 2 min_compat_ondisk_format 2
2018-06-13 21:02:08.395 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label
2018-06-13 21:02:08.395 7f1a93bdc240  2 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::const_iterator&) decode past end of struct encoding
2018-06-13 21:02:08.395 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label
2018-06-13 21:02:08.395 7f1a93bdc240  2 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::const_iterator&) decode past end of struct encoding
2018-06-13 21:02:08.395 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1/) _close_fm
2018-06-13 21:02:08.395 7f1a93bdc240  1 freelist shutdown
2018-06-13 21:02:08.395 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/db_impl.cc:252] Shutdown: canceling all background work
2018-06-13 21:02:08.395 7f1a93bdc240  4 rocksdb: [/build/ceph-14.0.0-376-g05d1e60/src/rocksdb/db/db_impl.cc:398] Shutdown complete
2018-06-13 21:02:08.395 7f1a93bdc240  1 bluefs umount
2018-06-13 21:02:08.395 7f1a93bdc240  1 stupidalloc 0x0x55d872ab6b20 shutdown
2018-06-13 21:02:08.395 7f1a93bdc240  1 bdev(0x55d872e90380 /var/lib/ceph/osd/ceph-1//block) close
2018-06-13 21:02:08.663 7f1a93bdc240  1 bdev(0x55d872e90000 /var/lib/ceph/osd/ceph-1//block) close
2018-06-13 21:02:08.895 7f1a93bdc240  1 bluestore(/var/lib/ceph/osd/ceph-1/) _fsck <<<START>>> check (shallow) start
2018-06-13 21:02:08.895 7f1a93bdc240  1 bdev create path /var/lib/ceph/osd/ceph-1//block type kernel
2018-06-13 21:02:08.895 7f1a93bdc240  1 bdev(0x55d872e90000 /var/lib/ceph/osd/ceph-1//block) open path /var/lib/ceph/osd/ceph-1//block
2018-06-13 21:02:08.895 7f1a93bdc240  1 bdev(0x55d872e90000 /var/lib/ceph/osd/ceph-1//block) open size 1073741824 (0x40000000, 1 GiB) block_size 4096 (4 KiB) rotational
2018-06-13 21:02:08.895 7f1a93bdc240 10 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label
2018-06-13 21:02:08.895 7f1a93bdc240  2 bluestore(/var/lib/ceph/osd/ceph-1//block) _read_bdev_label unable to decode label at offset 102: buffer::malformed_input: void bluestore_bdev_label_t::decode(ceph::buffer::list::const_iterator&) decode past end of struct encoding
2018-06-13 21:02:08.895 7f1a93bdc240  1 bdev(0x55d872e90000 /var/lib/ceph/osd/ceph-1//block) close
2018-06-13 21:02:09.163 7f1a93bdc240 -1 OSD::mkfs: ObjectStore::mkfs failed with error (2) No such file or directory
2018-06-13 21:02:09.163 7f1a93bdc240 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-1/: (2) No such file or directory
Actions
Copy link
 #6

Updated by Igor Fedotov almost 6 years ago

  • Status changed from In Progress to Fix Under Review
Actions
Copy link
 #7

Updated by Kefu Chai over 5 years ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link

Also available in: Atom PDF