-- Logs begin at Mon 2021-07-19 11:14:39 UTC, end at Tue 2021-07-27 11:20:41 UTC. -- Jul 27 09:28:01 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:01.658+0000 7f0b5d362700 1 mon.newbrunswick1@1(peon).osd e87647 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 343932928 kv_alloc: 327155712 Jul 27 09:28:08 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:08.465+0000 7f0b5d362700 1 mon.newbrunswick1@1(peon).osd e87647 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 343932928 kv_alloc: 327155712 Jul 27 09:28:14 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:14.022+0000 7f0b5d362700 1 mon.newbrunswick1@1(peon).osd e87647 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 343932928 kv_alloc: 327155712 Jul 27 09:28:19 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:19.024+0000 7f0b5d362700 1 mon.newbrunswick1@1(peon).osd e87647 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 343932928 kv_alloc: 327155712 Jul 27 09:28:23 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:23.324+0000 7f0b5ab5d700 0 mon.newbrunswick1@1(peon) e2 handle_command mon_command({"prefix": "mon metadata", "id": "newbrunswick0"} v 0) v1 Jul 27 09:28:23 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:23.326+0000 7f0b5ab5d700 0 log_channel(audit) log [DBG] : from='mgr.3344287 10.245.0.40:0/56' entity='mgr.newbrunswick0' cmd=[{"prefix": "mon metadata", "id": "newbrunswick0"}]: dispatch Jul 27 09:28:23 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:23.329+0000 7f0b5ab5d700 0 mon.newbrunswick1@1(peon) e2 handle_command mon_command({"prefix": "mon metadata", "id": "newbrunswick1"} v 0) v1 Jul 27 09:28:23 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:23.330+0000 7f0b5ab5d700 0 log_channel(audit) log [DBG] : from='mgr.3344287 10.245.0.40:0/56' entity='mgr.newbrunswick0' cmd=[{"prefix": "mon metadata", "id": "newbrunswick1"}]: dispatch Jul 27 09:28:23 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:23.332+0000 7f0b5ab5d700 0 mon.newbrunswick1@1(peon) e2 handle_command mon_command({"prefix": "mon metadata", "id": "newbrunswick2"} v 0) v1 Jul 27 09:28:23 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:23.334+0000 7f0b5ab5d700 0 log_channel(audit) log [DBG] : from='mgr.3344287 10.245.0.40:0/56' entity='mgr.newbrunswick0' cmd=[{"prefix": "mon metadata", "id": "newbrunswick2"}]: dispatch Jul 27 09:28:24 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:24.761+0000 7f0b5d362700 1 mon.newbrunswick1@1(peon).osd e87647 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 343932928 kv_alloc: 327155712 Jul 27 09:28:29 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:29.762+0000 7f0b5d362700 1 mon.newbrunswick1@1(peon).osd e87647 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 343932928 full_alloc: 343932928 kv_alloc: 327155712 Jul 27 09:28:32 newbrunswick1 systemd[1]: Stopping Ceph Monitor... Jul 27 09:28:33 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:33.440+0000 7f0b6136a700 -1 received signal: Terminated from (PID: 444) UID: 0 Jul 27 09:28:33 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:33.440+0000 7f0b6136a700 -1 mon.newbrunswick1@1(peon) e2 *** Got Signal Terminated *** Jul 27 09:28:33 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:33.440+0000 7f0b6136a700 1 mon.newbrunswick1@1(peon) e2 shutdown Jul 27 09:28:33 newbrunswick1 docker[1153]: teardown: managing teardown after SIGTERM Jul 27 09:28:33 newbrunswick1 docker[1153]: teardown: Sending SIGTERM to PID 71 Jul 27 09:28:33 newbrunswick1 docker[1153]: teardown: Waiting PID 71 to terminate .debug 2021-07-27T09:28:33.460+0000 7f0b714f46c0 1 rocksdb: close waiting for compaction thread to stop Jul 27 09:28:33 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:33.461+0000 7f0b714f46c0 1 rocksdb: close compaction thread to stopped Jul 27 09:28:33 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:33.461+0000 7f0b714f46c0 4 rocksdb: [db/db_impl.cc:390] Shutdown: canceling all background work Jul 27 09:28:33 newbrunswick1 docker[1153]: debug 2021-07-27T09:28:33.463+0000 7f0b714f46c0 4 rocksdb: [db/db_impl.cc:563] Shutdown complete Jul 27 09:28:33 newbrunswick1 docker[1153]: . Jul 27 09:28:33 newbrunswick1 docker[1153]: teardown: Process 71 is terminated Jul 27 09:28:33 newbrunswick1 docker[4867]: ceph-mon-newbrunswick1 Jul 27 09:28:33 newbrunswick1 systemd[1]: ceph-mon@newbrunswick1.service: main process exited, code=exited, status=143/n/a Jul 27 09:28:33 newbrunswick1 systemd[1]: Stopped Ceph Monitor. Jul 27 09:28:33 newbrunswick1 systemd[1]: Unit ceph-mon@newbrunswick1.service entered failed state. Jul 27 09:28:33 newbrunswick1 systemd[1]: ceph-mon@newbrunswick1.service failed. -- Reboot -- Jul 27 09:29:07 newbrunswick1 systemd[1]: Starting Ceph Monitor... Jul 27 09:29:07 newbrunswick1 docker[1042]: Error response from daemon: No such container: ceph-mon-newbrunswick1 Jul 27 09:29:07 newbrunswick1 docker[1111]: Error: No such container: ceph-mon-newbrunswick1 Jul 27 09:29:07 newbrunswick1 systemd[1]: Started Ceph Monitor. Jul 27 09:29:08 newbrunswick1 docker[1150]: 2021-07-27 09:29:08 /opt/ceph-container/bin/entrypoint.sh: Existing mon, trying to rejoin cluster... Jul 27 09:29:16 newbrunswick1 docker[1150]: 2021-07-27 09:29:16 /opt/ceph-container/bin/entrypoint.sh: SUCCESS Jul 27 09:29:16 newbrunswick1 docker[1150]: exec: PID 72: spawning /usr/bin/ceph-mon --cluster ceph --setuser ceph --setgroup ceph --default-log-to-stderr=true --err-to-stderr=true --default-log-to-file=false --foreground --default-mon-cluster-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false -i newbrunswick1 --mon-data /var/lib/ceph/mon/ceph-newbrunswick1 --public-addr 10.245.0.41 Jul 27 09:29:16 newbrunswick1 docker[1150]: exec: Waiting 72 to quit Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.822+0000 7f2c352006c0 0 set uid:gid to 167:167 (ceph:ceph) Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.822+0000 7f2c352006c0 0 ceph version 15.2.13 (c44bc49e7a57a87d84dfff2a077a2058aa2172e2) octopus (stable), process ceph-mon, pid 72 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.823+0000 7f2c352006c0 0 pidfile_write: ignore empty --pid-file Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.851+0000 7f2c352006c0 0 load: jerasure load: lrc load: isa Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.873+0000 7f2c352006c0 0 set rocksdb option compression = kNoCompression Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.873+0000 7f2c352006c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.873+0000 7f2c352006c0 0 set rocksdb option write_buffer_size = 33554432 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.873+0000 7f2c352006c0 0 set rocksdb option compression = kNoCompression Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.873+0000 7f2c352006c0 0 set rocksdb option level_compaction_dynamic_level_bytes = true Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.873+0000 7f2c352006c0 0 set rocksdb option write_buffer_size = 33554432 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.914+0000 7f2c352006c0 1 rocksdb: do_open column families: [default] Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.915+0000 7f2c352006c0 4 rocksdb: RocksDB version: 6.1.2 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.915+0000 7f2c352006c0 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.915+0000 7f2c352006c0 4 rocksdb: Compile date May 26 2021 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.915+0000 7f2c352006c0 4 rocksdb: DB SUMMARY Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: CURRENT file: CURRENT Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: IDENTITY file: IDENTITY Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: MANIFEST file: MANIFEST-170579 size: 4475 Bytes Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-newbrunswick1/store.db dir, Total Num: 2, files: 170656.sst 170657.sst Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-newbrunswick1/store.db: 170654.log size: 4358605 ; Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.error_if_exists: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.create_if_missing: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.paranoid_checks: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.env: 0x55d73f2631c0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.info_log: 0x55d740f8edc0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.max_file_opening_threads: 16 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.statistics: (nil) Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.use_fsync: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.max_log_file_size: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.max_manifest_file_size: 1073741824 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.log_file_time_to_roll: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.keep_log_file_num: 1000 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.recycle_log_file_num: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.allow_fallocate: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.allow_mmap_reads: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.allow_mmap_writes: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.use_direct_reads: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.create_missing_column_families: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.db_log_dir: Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-newbrunswick1/store.db Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.table_cache_numshardbits: 6 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.max_subcompactions: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.max_background_flushes: -1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.WAL_ttl_seconds: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.WAL_size_limit_MB: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.manifest_preallocation_size: 4194304 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.917+0000 7f2c352006c0 4 rocksdb: Options.is_fd_close_on_exec: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.advise_random_on_open: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.db_write_buffer_size: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.write_buffer_manager: 0x55d740f4d5c0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.access_hint_on_compaction_start: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.random_access_max_buffer_size: 1048576 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.use_adaptive_mutex: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.rate_limiter: (nil) Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.wal_recovery_mode: 2 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.enable_thread_tracking: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.enable_pipelined_write: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.allow_concurrent_memtable_write: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.write_thread_max_yield_usec: 100 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.write_thread_slow_yield_usec: 3 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.row_cache: None Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.wal_filter: None Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.avoid_flush_during_recovery: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.allow_ingest_behind: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.preserve_deletes: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.two_write_queues: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.manual_wal_flush: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.atomic_flush: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.max_background_jobs: 2 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.max_background_compactions: -1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.avoid_flush_during_shutdown: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.delayed_write_rate : 16777216 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.max_total_wal_size: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.918+0000 7f2c352006c0 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.stats_dump_period_sec: 600 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.stats_persist_period_sec: 600 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.stats_history_buffer_size: 1048576 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.max_open_files: -1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.bytes_per_sync: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.wal_bytes_per_sync: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Options.compaction_readahead_size: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Compression algorithms supported: Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kZSTDNotFinalCompression supported: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kZSTD supported: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kXpressCompression supported: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kLZ4HCCompression supported: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kLZ4Compression supported: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kBZip2Compression supported: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kZlibCompression supported: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: kSnappyCompression supported: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.919+0000 7f2c352006c0 4 rocksdb: Fast CRC32 supported: Supported on x86 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: [db/version_set.cc:3543] Recovering from manifest file: /var/lib/ceph/mon/ceph-newbrunswick1/store.db/MANIFEST-170579 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: [db/column_family.cc:477] --------------- Options for column family [default]: Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.comparator: leveldb.BytewiseComparator Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.merge_operator: Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compaction_filter: None Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compaction_filter_factory: None Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.memtable_factory: SkipListFactory Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.table_factory: BlockBasedTable Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d740ea92c0) Jul 27 09:29:16 newbrunswick1 docker[1150]: cache_index_and_filter_blocks: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: cache_index_and_filter_blocks_with_high_priority: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: pin_l0_filter_and_index_blocks_in_cache: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: pin_top_level_index_and_filter: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: index_type: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: data_block_index_type: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: data_block_hash_table_util_ratio: 0.750000 Jul 27 09:29:16 newbrunswick1 docker[1150]: hash_index_allow_collision: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: checksum: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: no_block_cache: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: block_cache: 0x55d740ee0610 Jul 27 09:29:16 newbrunswick1 docker[1150]: block_cache_name: BinnedLRUCache Jul 27 09:29:16 newbrunswick1 docker[1150]: block_cache_options: Jul 27 09:29:16 newbrunswick1 docker[1150]: capacity : 536870912 Jul 27 09:29:16 newbrunswick1 docker[1150]: num_shard_bits : 4 Jul 27 09:29:16 newbrunswick1 docker[1150]: strict_capacity_limit : 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: high_pri_pool_ratio: 0.000 Jul 27 09:29:16 newbrunswick1 docker[1150]: block_cache_compressed: (nil) Jul 27 09:29:16 newbrunswick1 docker[1150]: persistent_cache: (nil) Jul 27 09:29:16 newbrunswick1 docker[1150]: block_size: 4096 Jul 27 09:29:16 newbrunswick1 docker[1150]: block_size_deviation: 10 Jul 27 09:29:16 newbrunswick1 docker[1150]: block_restart_interval: 16 Jul 27 09:29:16 newbrunswick1 docker[1150]: index_block_restart_interval: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: metadata_block_size: 4096 Jul 27 09:29:16 newbrunswick1 docker[1150]: partition_filters: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: use_delta_encoding: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: filter_policy: rocksdb.BuiltinBloomFilter Jul 27 09:29:16 newbrunswick1 docker[1150]: whole_key_filtering: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: verify_compression: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: read_amp_bytes_per_bit: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: format_version: 2 Jul 27 09:29:16 newbrunswick1 docker[1150]: enable_index_compression: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: block_align: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.write_buffer_size: 33554432 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.max_write_buffer_number: 2 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression: NoCompression Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression: Disabled Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.prefix_extractor: nullptr Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.num_levels: 7 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression_opts.level: 32767 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.bottommost_compression_opts.enabled: false Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression_opts.window_bits: -14 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression_opts.level: 32767 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression_opts.strategy: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.compression_opts.enabled: false Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.930+0000 7f2c352006c0 4 rocksdb: Options.level0_stop_writes_trigger: 36 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.target_file_size_base: 67108864 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.target_file_size_multiplier: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_base: 268435456 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 Jul 27 09:29:16 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.max_compaction_bytes: 1677721600 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.arena_block_size: 4194304 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.931+0000 7f2c352006c0 4 rocksdb: Options.disable_auto_compactions: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_style: kCompactionStyleLevel Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.table_properties_collectors: Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.inplace_update_support: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.inplace_update_num_locks: 10000 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.memtable_whole_key_filtering: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.memtable_huge_page_size: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.bloom_locality: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.max_successive_merges: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.optimize_filters_for_hits: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.paranoid_file_checks: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.force_consistency_checks: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.report_bg_io_stats: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.932+0000 7f2c352006c0 4 rocksdb: Options.ttl: 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.963+0000 7f2c352006c0 4 rocksdb: [db/version_set.cc:3757] Recovered from manifest file:/var/lib/ceph/mon/ceph-newbrunswick1/store.db/MANIFEST-170579 succeeded,manifest_file_number is 170579, next_file_number is 170659, last_sequence is 84342975, log_number is 170654,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.963+0000 7f2c352006c0 4 rocksdb: [db/version_set.cc:3766] Column family [default] (ID 0), log number is 170654 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.963+0000 7f2c352006c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1627378156963929, "job": 1, "event": "recovery_started", "log_files": [170654]} Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:16.963+0000 7f2c352006c0 4 rocksdb: [db/db_impl_open.cc:583] Recovering log #170654 mode 2 Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:17.762+0000 7f2c352006c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1627378157763394, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 170659, "file_size": 2910244, "table_properties": {"data_size": 2904817, "index_size": 4156, "filter_size": 453, "raw_key_size": 3390, "raw_average_key_size": 22, "raw_value_size": 2899505, "raw_average_value_size": 18951, "num_data_blocks": 118, "num_entries": 153, "filter_policy_name": "rocksdb.BuiltinBloomFilter"}} Jul 27 09:29:17 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:17.763+0000 7f2c352006c0 4 rocksdb: [db/version_set.cc:3036] Creating manifest 170660 Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.209+0000 7f2c352006c0 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1627378158209880, "job": 1, "event": "recovery_finished"} Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.273+0000 7f2c352006c0 4 rocksdb: DB pointer 0x55d741b0e400 Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.274+0000 7f2c1c064700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS ------- Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.274+0000 7f2c1c064700 4 rocksdb: [db/db_impl.cc:778] Jul 27 09:29:18 newbrunswick1 docker[1150]: ** DB Stats ** Jul 27 09:29:18 newbrunswick1 docker[1150]: Uptime(secs): 1.3 total, 1.3 interval Jul 27 09:29:18 newbrunswick1 docker[1150]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s Jul 27 09:29:18 newbrunswick1 docker[1150]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s Jul 27 09:29:18 newbrunswick1 docker[1150]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Jul 27 09:29:18 newbrunswick1 docker[1150]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s Jul 27 09:29:18 newbrunswick1 docker[1150]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s Jul 27 09:29:18 newbrunswick1 docker[1150]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent Jul 27 09:29:18 newbrunswick1 docker[1150]: ** Compaction Stats [default] ** Jul 27 09:29:18 newbrunswick1 docker[1150]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Jul 27 09:29:18 newbrunswick1 docker[1150]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Jul 27 09:29:18 newbrunswick1 docker[1150]: L0 1/0 2.78 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: L6 2/0 64.53 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Sum 3/0 67.31 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: ** Compaction Stats [default] ** Jul 27 09:29:18 newbrunswick1 docker[1150]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Jul 27 09:29:18 newbrunswick1 docker[1150]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Jul 27 09:29:18 newbrunswick1 docker[1150]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Uptime(secs): 1.3 total, 1.3 interval Jul 27 09:29:18 newbrunswick1 docker[1150]: Flush(GB): cumulative 0.003, interval 0.003 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(GB): cumulative 0.000, interval 0.000 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(Total Files): cumulative 0, interval 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(L0 Files): cumulative 0, interval 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(Keys): cumulative 0, interval 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Cumulative compaction: 0.00 GB write, 2.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.2 seconds Jul 27 09:29:18 newbrunswick1 docker[1150]: Interval compaction: 0.00 GB write, 2.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.2 seconds Jul 27 09:29:18 newbrunswick1 docker[1150]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Jul 27 09:29:18 newbrunswick1 docker[1150]: ** File Read Latency Histogram By Level [default] ** Jul 27 09:29:18 newbrunswick1 docker[1150]: ** Compaction Stats [default] ** Jul 27 09:29:18 newbrunswick1 docker[1150]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Jul 27 09:29:18 newbrunswick1 docker[1150]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Jul 27 09:29:18 newbrunswick1 docker[1150]: L0 1/0 2.78 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: L6 2/0 64.53 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Sum 3/0 67.31 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: ** Compaction Stats [default] ** Jul 27 09:29:18 newbrunswick1 docker[1150]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Jul 27 09:29:18 newbrunswick1 docker[1150]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Jul 27 09:29:18 newbrunswick1 docker[1150]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 13.2 0.21 0.00 1 0.210 0 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Uptime(secs): 1.3 total, 0.0 interval Jul 27 09:29:18 newbrunswick1 docker[1150]: Flush(GB): cumulative 0.003, interval 0.000 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(GB): cumulative 0.000, interval 0.000 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(Total Files): cumulative 0, interval 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(L0 Files): cumulative 0, interval 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: AddFile(Keys): cumulative 0, interval 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: Cumulative compaction: 0.00 GB write, 2.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.2 seconds Jul 27 09:29:18 newbrunswick1 docker[1150]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Jul 27 09:29:18 newbrunswick1 docker[1150]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count Jul 27 09:29:18 newbrunswick1 docker[1150]: ** File Read Latency Histogram By Level [default] ** Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.314+0000 7f2c352006c0 0 starting mon.newbrunswick1 rank 1 at public addrs [v2:10.245.0.41:3300/0,v1:10.245.0.41:6789/0] at bind addrs [v2:10.245.0.41:3300/0,v1:10.245.0.41:6789/0] mon_data /var/lib/ceph/mon/ceph-newbrunswick1 fsid 7a4265b6-605a-4dbc-9eaa-ec5d9ff62c2a Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.316+0000 7f2c352006c0 1 mon.newbrunswick1@-1(???) e2 preinit fsid 7a4265b6-605a-4dbc-9eaa-ec5d9ff62c2a Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.401+0000 7f2c352006c0 0 mon.newbrunswick1@-1(???).mds e287 new map Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.401+0000 7f2c352006c0 0 mon.newbrunswick1@-1(???).mds e287 print_map Jul 27 09:29:18 newbrunswick1 docker[1150]: e287 Jul 27 09:29:18 newbrunswick1 docker[1150]: enable_multiple, ever_enabled_multiple: 0,0 Jul 27 09:29:18 newbrunswick1 docker[1150]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:29:18 newbrunswick1 docker[1150]: legacy client fscid: 1 Jul 27 09:29:18 newbrunswick1 docker[1150]: Filesystem 'cephfs' (1) Jul 27 09:29:18 newbrunswick1 docker[1150]: fs_name cephfs Jul 27 09:29:18 newbrunswick1 docker[1150]: epoch 287 Jul 27 09:29:18 newbrunswick1 docker[1150]: flags 12 Jul 27 09:29:18 newbrunswick1 docker[1150]: created 2021-04-21T13:23:09.271232+0000 Jul 27 09:29:18 newbrunswick1 docker[1150]: modified 2021-07-27T08:47:41.653179+0000 Jul 27 09:29:18 newbrunswick1 docker[1150]: tableserver 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: root 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: session_timeout 60 Jul 27 09:29:18 newbrunswick1 docker[1150]: session_autoclose 300 Jul 27 09:29:18 newbrunswick1 docker[1150]: max_file_size 1099511627776 Jul 27 09:29:18 newbrunswick1 docker[1150]: min_compat_client 0 (unknown) Jul 27 09:29:18 newbrunswick1 docker[1150]: last_failure 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: last_failure_osd_epoch 87639 Jul 27 09:29:18 newbrunswick1 docker[1150]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:29:18 newbrunswick1 docker[1150]: max_mds 1 Jul 27 09:29:18 newbrunswick1 docker[1150]: in 0 Jul 27 09:29:18 newbrunswick1 docker[1150]: up {0=3344260} Jul 27 09:29:18 newbrunswick1 docker[1150]: failed Jul 27 09:29:18 newbrunswick1 docker[1150]: damaged Jul 27 09:29:18 newbrunswick1 docker[1150]: stopped Jul 27 09:29:18 newbrunswick1 docker[1150]: data_pools [1] Jul 27 09:29:18 newbrunswick1 docker[1150]: metadata_pool 2 Jul 27 09:29:18 newbrunswick1 docker[1150]: inline_data disabled Jul 27 09:29:18 newbrunswick1 docker[1150]: balancer Jul 27 09:29:18 newbrunswick1 docker[1150]: standby_count_wanted 1 Jul 27 09:29:18 newbrunswick1 docker[1150]: [mds.newbrunswick2{0:3344260} state up:active seq 10 addr [v2:10.245.0.42:6800/3840387237,v1:10.245.0.42:6801/3840387237]] Jul 27 09:29:18 newbrunswick1 docker[1150]: Standby daemons: Jul 27 09:29:18 newbrunswick1 docker[1150]: [mds.newbrunswick3{-1:3310161} state up:standby seq 3 addr [v2:10.245.0.43:6800/3165326145,v1:10.245.0.43:6801/3165326145]] Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.499+0000 7f2c352006c0 0 mon.newbrunswick1@-1(???).osd e87647 crush map has features 283675107524608, adjusting msgr requires Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.499+0000 7f2c352006c0 0 mon.newbrunswick1@-1(???).osd e87647 crush map has features 283675107524608, adjusting msgr requires Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.500+0000 7f2c352006c0 0 mon.newbrunswick1@-1(???).osd e87647 crush map has features 3026702624700514304, adjusting msgr requires Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.500+0000 7f2c352006c0 0 mon.newbrunswick1@-1(???).osd e87647 crush map has features 283675107524608, adjusting msgr requires Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.523+0000 7f2c352006c0 1 mon.newbrunswick1@-1(???).paxosservice(auth 2751..2789) refresh upgraded, format 0 -> 3 Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.634+0000 7f2c1d867700 1 mon.newbrunswick1@-1(???) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.635+0000 7f2c352006c0 0 mon.newbrunswick1@-1(probing) e2 my rank is now 1 (was -1) Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.638+0000 7f2c1e869700 0 log_channel(cluster) log [INF] : mon.newbrunswick1 calling monitor election Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.638+0000 7f2c1e869700 1 paxos.1).electionLogic(84512) init, last seen epoch 84512 Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.644+0000 7f2c22871700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.645+0000 7f2c22871700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.649+0000 7f2c22871700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.672+0000 7f2c22871700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.694+0000 7f2c22871700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.734+0000 7f2c1e869700 1 mon.newbrunswick1@1(electing) e2 collect_metadata : no unique device id for : fallback method has no model nor serial' Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.839+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.845+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.846+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.873+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.875+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:18.895+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.080+0000 7f2c1e869700 1 mon.newbrunswick1@1(electing) e2 collect_metadata : no unique device id for : fallback method has no model nor serial' Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.127+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.246+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.248+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.276+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.296+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.360+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.376+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.397+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.402+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.417+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.436+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.450+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.561+0000 7f2c1d867700 1 mon.newbrunswick1@1(electing) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.577+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.598+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.618+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.641+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.675+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.962+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:19 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:19.978+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.000+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.019+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.047+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.049+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.077+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.098+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.725+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.763+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.780+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.802+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.820+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:20 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:20.927+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.002+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.035+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.050+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.203+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.236+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.238+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.252+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.272+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.328+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.439+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.474+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.604+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.637+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.644+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.646+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.653+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.673+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.695+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.840+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.845+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.847+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.875+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.875+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:21 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:21.896+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.129+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.247+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.248+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.277+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.297+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.360+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.377+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.398+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.406+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.418+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.439+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.455+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.561+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.578+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.599+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.619+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.641+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.677+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.962+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:22 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:22.979+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.000+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.020+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.048+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.049+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.077+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.099+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.725+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.763+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.780+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.801+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.822+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:23.926+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.003+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.035+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.051+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.204+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.236+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.238+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.252+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.273+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.327+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.440+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.474+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.606+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.637+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.644+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.646+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.653+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.674+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.696+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.841+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.845+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.847+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.875+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.876+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:24 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:24.897+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:25 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:25.129+0000 7f2c1d867700 1 mon.newbrunswick1@1(peon) e2 handle_auth_request failed to assign global_id Jul 27 09:29:25 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:25.369+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).mds e287 mds mds.? [v2:10.245.0.42:6800/3891958835,v1:10.245.0.42:6801/3891958835] can't write to fsmap compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:29:26 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:26.791+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87647 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 Jul 27 09:29:26 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:26.792+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87647 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 Jul 27 09:29:26 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:26.793+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87648 e87648: 4 total, 1 up, 4 in Jul 27 09:29:28 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:28.922+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87648 _set_new_cache_sizes cache_size:1019939989 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:31 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:31.767+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon) e2 handle_command mon_command({"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["3"]} v 0) v1 Jul 27 09:29:31 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:31.767+0000 7f2c1e869700 0 log_channel(audit) log [INF] : from='osd.3 [v2:10.245.0.41:6800/1744,v1:10.245.0.41:6801/1744]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "hdd", "ids": ["3"]}]: dispatch Jul 27 09:29:33 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:33.632+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87649 e87649: 4 total, 1 up, 4 in Jul 27 09:29:33 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:33.968+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87649 _set_new_cache_sizes cache_size:1020053397 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:36 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:36.839+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87650 e87650: 4 total, 1 up, 4 in Jul 27 09:29:38 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:38.503+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon).mds e288 new map Jul 27 09:29:38 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:38.503+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon).mds e288 print_map Jul 27 09:29:38 newbrunswick1 docker[1150]: e288 Jul 27 09:29:38 newbrunswick1 docker[1150]: enable_multiple, ever_enabled_multiple: 0,0 Jul 27 09:29:38 newbrunswick1 docker[1150]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:29:38 newbrunswick1 docker[1150]: legacy client fscid: 1 Jul 27 09:29:38 newbrunswick1 docker[1150]: Filesystem 'cephfs' (1) Jul 27 09:29:38 newbrunswick1 docker[1150]: fs_name cephfs Jul 27 09:29:38 newbrunswick1 docker[1150]: epoch 288 Jul 27 09:29:38 newbrunswick1 docker[1150]: flags 12 Jul 27 09:29:38 newbrunswick1 docker[1150]: created 2021-04-21T13:23:09.271232+0000 Jul 27 09:29:38 newbrunswick1 docker[1150]: modified 2021-07-27T09:29:34.037104+0000 Jul 27 09:29:38 newbrunswick1 docker[1150]: tableserver 0 Jul 27 09:29:38 newbrunswick1 docker[1150]: root 0 Jul 27 09:29:38 newbrunswick1 docker[1150]: session_timeout 60 Jul 27 09:29:38 newbrunswick1 docker[1150]: session_autoclose 300 Jul 27 09:29:38 newbrunswick1 docker[1150]: max_file_size 1099511627776 Jul 27 09:29:38 newbrunswick1 docker[1150]: min_compat_client 0 (unknown) Jul 27 09:29:38 newbrunswick1 docker[1150]: last_failure 0 Jul 27 09:29:38 newbrunswick1 docker[1150]: last_failure_osd_epoch 87650 Jul 27 09:29:38 newbrunswick1 docker[1150]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:29:38 newbrunswick1 docker[1150]: max_mds 1 Jul 27 09:29:38 newbrunswick1 docker[1150]: in 0 Jul 27 09:29:38 newbrunswick1 docker[1150]: up {0=3364108} Jul 27 09:29:38 newbrunswick1 docker[1150]: failed Jul 27 09:29:38 newbrunswick1 docker[1150]: damaged Jul 27 09:29:38 newbrunswick1 docker[1150]: stopped Jul 27 09:29:38 newbrunswick1 docker[1150]: data_pools [1] Jul 27 09:29:38 newbrunswick1 docker[1150]: metadata_pool 2 Jul 27 09:29:38 newbrunswick1 docker[1150]: inline_data disabled Jul 27 09:29:38 newbrunswick1 docker[1150]: balancer Jul 27 09:29:38 newbrunswick1 docker[1150]: standby_count_wanted 1 Jul 27 09:29:38 newbrunswick1 docker[1150]: [mds.newbrunswick2{0:3364108} state up:replay seq 2 addr [v2:10.245.0.42:6800/3891958835,v1:10.245.0.42:6801/3891958835]] Jul 27 09:29:38 newbrunswick1 docker[1150]: Standby daemons: Jul 27 09:29:38 newbrunswick1 docker[1150]: [mds.newbrunswick3{-1:3310161} state up:standby seq 3 addr [v2:10.245.0.43:6800/3165326145,v1:10.245.0.43:6801/3165326145]] Jul 27 09:29:38 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:38.969+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87650 _set_new_cache_sizes cache_size:1020054716 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:41 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:41.292+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87651 e87651: 4 total, 1 up, 4 in Jul 27 09:29:42 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:42.162+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon) e2 handle_command mon_command({"prefix": "osd crush create-or-move", "id": 3, "weight":0.0977, "args": ["host=newbrunswick1", "root=default"]} v 0) v1 Jul 27 09:29:42 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:42.163+0000 7f2c1e869700 0 log_channel(audit) log [INF] : from='osd.3 [v2:10.245.0.41:6800/1744,v1:10.245.0.41:6801/1744]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0977, "args": ["host=newbrunswick1", "root=default"]}]: dispatch Jul 27 09:29:42 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:42.649+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87652 e87652: 4 total, 1 up, 4 in Jul 27 09:29:43 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:43.969+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87652 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:46 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:46.841+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87653 e87653: 4 total, 2 up, 4 in Jul 27 09:29:48 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:48.970+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87653 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:49 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:49.805+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87654 e87654: 4 total, 3 up, 4 in Jul 27 09:29:53 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:53.971+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87654 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:54 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:54.476+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87655 e87655: 4 total, 3 up, 4 in Jul 27 09:29:58 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:58.971+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87655 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:29:59 newbrunswick1 docker[1150]: debug 2021-07-27T09:29:59.745+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87656 e87656: 4 total, 4 up, 4 in Jul 27 09:30:02 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:02.660+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87657 e87657: 4 total, 4 up, 4 in Jul 27 09:30:03 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:03.971+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87657 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:08 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:08.972+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87657 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:10 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:10.225+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87658 e87658: 4 total, 4 up, 4 in Jul 27 09:30:13 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:13.971+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87658 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:15 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:15.828+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon).mds e289 new map Jul 27 09:30:15 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:15.828+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon).mds e289 print_map Jul 27 09:30:15 newbrunswick1 docker[1150]: e289 Jul 27 09:30:15 newbrunswick1 docker[1150]: enable_multiple, ever_enabled_multiple: 0,0 Jul 27 09:30:15 newbrunswick1 docker[1150]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:30:15 newbrunswick1 docker[1150]: legacy client fscid: 1 Jul 27 09:30:15 newbrunswick1 docker[1150]: Filesystem 'cephfs' (1) Jul 27 09:30:15 newbrunswick1 docker[1150]: fs_name cephfs Jul 27 09:30:15 newbrunswick1 docker[1150]: epoch 289 Jul 27 09:30:15 newbrunswick1 docker[1150]: flags 12 Jul 27 09:30:15 newbrunswick1 docker[1150]: created 2021-04-21T13:23:09.271232+0000 Jul 27 09:30:15 newbrunswick1 docker[1150]: modified 2021-07-27T09:30:11.081490+0000 Jul 27 09:30:15 newbrunswick1 docker[1150]: tableserver 0 Jul 27 09:30:15 newbrunswick1 docker[1150]: root 0 Jul 27 09:30:15 newbrunswick1 docker[1150]: session_timeout 60 Jul 27 09:30:15 newbrunswick1 docker[1150]: session_autoclose 300 Jul 27 09:30:15 newbrunswick1 docker[1150]: max_file_size 1099511627776 Jul 27 09:30:15 newbrunswick1 docker[1150]: min_compat_client 0 (unknown) Jul 27 09:30:15 newbrunswick1 docker[1150]: last_failure 0 Jul 27 09:30:15 newbrunswick1 docker[1150]: last_failure_osd_epoch 87659 Jul 27 09:30:15 newbrunswick1 docker[1150]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:30:15 newbrunswick1 docker[1150]: max_mds 1 Jul 27 09:30:15 newbrunswick1 docker[1150]: in 0 Jul 27 09:30:15 newbrunswick1 docker[1150]: up {} Jul 27 09:30:15 newbrunswick1 docker[1150]: failed Jul 27 09:30:15 newbrunswick1 docker[1150]: damaged 0 Jul 27 09:30:15 newbrunswick1 docker[1150]: stopped Jul 27 09:30:15 newbrunswick1 docker[1150]: data_pools [1] Jul 27 09:30:15 newbrunswick1 docker[1150]: metadata_pool 2 Jul 27 09:30:15 newbrunswick1 docker[1150]: inline_data disabled Jul 27 09:30:15 newbrunswick1 docker[1150]: balancer Jul 27 09:30:15 newbrunswick1 docker[1150]: standby_count_wanted 1 Jul 27 09:30:15 newbrunswick1 docker[1150]: Standby daemons: Jul 27 09:30:15 newbrunswick1 docker[1150]: [mds.newbrunswick3{-1:3310161} state up:standby seq 3 addr [v2:10.245.0.43:6800/3165326145,v1:10.245.0.43:6801/3165326145]] Jul 27 09:30:15 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:15.828+0000 7f2c1e869700 1 mon.newbrunswick1@1(peon).osd e87659 e87659: 4 total, 4 up, 4 in Jul 27 09:30:18 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:18.973+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:23.057+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon).mds e290 new map Jul 27 09:30:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:23.057+0000 7f2c1e869700 0 mon.newbrunswick1@1(peon).mds e290 print_map Jul 27 09:30:23 newbrunswick1 docker[1150]: e290 Jul 27 09:30:23 newbrunswick1 docker[1150]: enable_multiple, ever_enabled_multiple: 0,0 Jul 27 09:30:23 newbrunswick1 docker[1150]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:30:23 newbrunswick1 docker[1150]: legacy client fscid: 1 Jul 27 09:30:23 newbrunswick1 docker[1150]: Filesystem 'cephfs' (1) Jul 27 09:30:23 newbrunswick1 docker[1150]: fs_name cephfs Jul 27 09:30:23 newbrunswick1 docker[1150]: epoch 289 Jul 27 09:30:23 newbrunswick1 docker[1150]: flags 12 Jul 27 09:30:23 newbrunswick1 docker[1150]: created 2021-04-21T13:23:09.271232+0000 Jul 27 09:30:23 newbrunswick1 docker[1150]: modified 2021-07-27T09:30:11.081490+0000 Jul 27 09:30:23 newbrunswick1 docker[1150]: tableserver 0 Jul 27 09:30:23 newbrunswick1 docker[1150]: root 0 Jul 27 09:30:23 newbrunswick1 docker[1150]: session_timeout 60 Jul 27 09:30:23 newbrunswick1 docker[1150]: session_autoclose 300 Jul 27 09:30:23 newbrunswick1 docker[1150]: max_file_size 1099511627776 Jul 27 09:30:23 newbrunswick1 docker[1150]: min_compat_client 0 (unknown) Jul 27 09:30:23 newbrunswick1 docker[1150]: last_failure 0 Jul 27 09:30:23 newbrunswick1 docker[1150]: last_failure_osd_epoch 87659 Jul 27 09:30:23 newbrunswick1 docker[1150]: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} Jul 27 09:30:23 newbrunswick1 docker[1150]: max_mds 1 Jul 27 09:30:23 newbrunswick1 docker[1150]: in 0 Jul 27 09:30:23 newbrunswick1 docker[1150]: up {} Jul 27 09:30:23 newbrunswick1 docker[1150]: failed Jul 27 09:30:23 newbrunswick1 docker[1150]: damaged 0 Jul 27 09:30:23 newbrunswick1 docker[1150]: stopped Jul 27 09:30:23 newbrunswick1 docker[1150]: data_pools [1] Jul 27 09:30:23 newbrunswick1 docker[1150]: metadata_pool 2 Jul 27 09:30:23 newbrunswick1 docker[1150]: inline_data disabled Jul 27 09:30:23 newbrunswick1 docker[1150]: balancer Jul 27 09:30:23 newbrunswick1 docker[1150]: standby_count_wanted 1 Jul 27 09:30:23 newbrunswick1 docker[1150]: Standby daemons: Jul 27 09:30:23 newbrunswick1 docker[1150]: [mds.newbrunswick3{-1:3310161} state up:standby seq 3 addr [v2:10.245.0.43:6800/3165326145,v1:10.245.0.43:6801/3165326145]] Jul 27 09:30:23 newbrunswick1 docker[1150]: [mds.newbrunswick2{-1:3364295} state up:standby seq 2 addr [v2:10.245.0.42:6800/1642389801,v1:10.245.0.42:6801/1642389801]] Jul 27 09:30:23 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:23.973+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:28 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:28.973+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:33 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:33.974+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:38 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:38.975+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:43 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:43.975+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:48 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:48.975+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:54 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:54.135+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760 Jul 27 09:30:59 newbrunswick1 docker[1150]: debug 2021-07-27T09:30:59.135+0000 7f2c2106e700 1 mon.newbrunswick1@1(peon).osd e87659 _set_new_cache_sizes cache_size:1020054731 inc_alloc: 373293056 full_alloc: 373293056 kv_alloc: 272629760