-1021> 2020-05-07 00:10:31.027 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -1020> 2020-05-07 00:10:31.027 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -1019> 2020-05-07 00:10:31.027 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -1018> 2020-05-07 00:10:31.107 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -1017> 2020-05-07 00:25:31.107 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -1016> 2020-05-07 00:25:31.107 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -1015> 2020-05-07 00:25:31.107 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -1014> 2020-05-07 00:25:31.610 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -1013> 2020-05-07 00:55:31.188 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -1012> 2020-05-07 00:55:31.188 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -1011> 2020-05-07 00:55:31.188 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -1010> 2020-05-07 00:55:31.619 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -1009> 2020-05-07 01:25:31.275 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -1008> 2020-05-07 01:25:31.275 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -1007> 2020-05-07 01:25:31.275 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -1006> 2020-05-07 01:25:31.355 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -1005> 2020-05-07 01:40:31.356 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -1004> 2020-05-07 01:40:31.356 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -1003> 2020-05-07 01:40:31.356 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -1002> 2020-05-07 01:40:31.807 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -1001> 2020-05-07 02:10:31.436 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -1000> 2020-05-07 02:10:31.436 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -999> 2020-05-07 02:10:31.436 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -998> 2020-05-07 02:10:31.900 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -997> 2020-05-07 02:40:31.517 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -996> 2020-05-07 02:40:31.517 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -995> 2020-05-07 02:40:31.517 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -994> 2020-05-07 02:40:31.994 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -993> 2020-05-07 03:10:31.598 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -992> 2020-05-07 03:10:31.598 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -991> 2020-05-07 03:10:31.598 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -990> 2020-05-07 03:10:32.088 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -989> 2020-05-07 03:31:02.015 7f51b77e1700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw (PID: 1098218) UID: 0 -988> 2020-05-07 03:31:02.050 7f51b77e1700 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw (PID: 1098219) UID: 0 -987> 2020-05-07 03:40:31.679 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -986> 2020-05-07 03:40:31.679 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -985> 2020-05-07 03:40:31.679 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -984> 2020-05-07 03:40:32.183 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -983> 2020-05-07 04:10:31.759 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -982> 2020-05-07 04:10:31.759 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -981> 2020-05-07 04:10:31.759 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -980> 2020-05-07 04:10:31.840 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -979> 2020-05-07 04:13:22.462 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 621 from recycle list -978> 2020-05-07 04:13:22.462 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #625. Immutable memtables: 0. -977> 2020-05-07 04:13:22.462 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-04:13:22.463482) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -976> 2020-05-07 04:13:22.462 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 184] Flushing memtable with next log file: 625 -975> 2020-05-07 04:13:22.462 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588824802463616, "job": 184, "event": "flush_started", "num_memtables": 1, "num_entries": 362768, "num_deletes": 73127, "memory_usage": 260060992, "flush_reason": "Write Buffer Full"} -974> 2020-05-07 04:13:22.462 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 184] Level-0 flush table #626: started -973> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588824802632704, "cf_name": "default", "job": 184, "event": "table_file_creation", "file_number": 626, "file_size": 34026114, "table_properties": {"data_size": 33160820, "index_size": 488979, "filter_size": 375559, "raw_key_size": 11842278, "raw_average_key_size": 98, "raw_value_size": 27229181, "raw_average_value_size": 226, "num_data_blocks": 7795, "num_entries": 120380, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "69742", "kMergeOperands": "1263"}} -972> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 184] Level-0 flush table #626: 34026114 bytes OK -971> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 623 to recycle list -970> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-04:13:22.632748) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #626 started -969> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-04:13:22.632930) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #626: memtable #1 done -968> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-04:13:22.632952) EVENT_LOG_v1 {"time_micros": 1588824802632943, "job": 184, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [3, 2, 14, 0, 0, 0, 0], "immutable_memtables": 0} -967> 2020-05-07 04:13:22.631 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-04:13:22.632978) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[3 2 14 0 0 0 0] max score 0.75 -966> 2020-05-07 04:25:31.840 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -965> 2020-05-07 04:25:31.840 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -964> 2020-05-07 04:25:31.840 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -963> 2020-05-07 04:25:32.238 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -962> 2020-05-07 04:55:31.922 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -961> 2020-05-07 04:55:31.922 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -960> 2020-05-07 04:55:31.922 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -959> 2020-05-07 04:55:32.331 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -958> 2020-05-07 05:25:32.004 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -957> 2020-05-07 05:25:32.004 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -956> 2020-05-07 05:25:32.004 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -955> 2020-05-07 05:25:32.479 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -954> 2020-05-07 05:55:32.085 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -953> 2020-05-07 05:55:32.085 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -952> 2020-05-07 05:55:32.085 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -951> 2020-05-07 05:55:32.572 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -950> 2020-05-07 06:25:32.165 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -949> 2020-05-07 06:25:32.165 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -948> 2020-05-07 06:25:32.165 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -947> 2020-05-07 06:25:32.245 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -946> 2020-05-07 06:40:32.246 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -945> 2020-05-07 06:40:32.246 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -944> 2020-05-07 06:40:32.246 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -943> 2020-05-07 06:40:32.715 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -942> 2020-05-07 07:10:32.326 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -941> 2020-05-07 07:10:32.326 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -940> 2020-05-07 07:10:32.326 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -939> 2020-05-07 07:10:32.807 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -938> 2020-05-07 07:40:32.408 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -937> 2020-05-07 07:40:32.408 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -936> 2020-05-07 07:40:32.408 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -935> 2020-05-07 07:40:32.818 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -934> 2020-05-07 08:10:32.489 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -933> 2020-05-07 08:10:32.489 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -932> 2020-05-07 08:10:32.489 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -931> 2020-05-07 08:10:32.906 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -930> 2020-05-07 08:40:32.569 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -929> 2020-05-07 08:40:32.569 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -928> 2020-05-07 08:40:32.569 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -927> 2020-05-07 08:40:33.049 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -926> 2020-05-07 09:04:28.563 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 623 from recycle list -925> 2020-05-07 09:04:28.563 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #627. Immutable memtables: 0. -924> 2020-05-07 09:04:28.563 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-09:04:28.564218) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -923> 2020-05-07 09:04:28.563 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 185] Flushing memtable with next log file: 627 -922> 2020-05-07 09:04:28.563 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842268564356, "job": 185, "event": "flush_started", "num_memtables": 1, "num_entries": 368843, "num_deletes": 74943, "memory_usage": 260066424, "flush_reason": "Write Buffer Full"} -921> 2020-05-07 09:04:28.563 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 185] Level-0 flush table #628: started -920> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842268737387, "cf_name": "default", "job": 185, "event": "table_file_creation", "file_number": 628, "file_size": 34310677, "table_properties": {"data_size": 33432232, "index_size": 496075, "filter_size": 381614, "raw_key_size": 12062575, "raw_average_key_size": 98, "raw_value_size": 27392356, "raw_average_value_size": 223, "num_data_blocks": 7836, "num_entries": 122575, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "71447", "kMergeOperands": "1302"}} -919> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 185] Level-0 flush table #628: 34310677 bytes OK -918> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 625 to recycle list -917> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-09:04:28.737427) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #628 started -916> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-09:04:28.737603) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #628: memtable #1 done -915> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-09:04:28.737620) EVENT_LOG_v1 {"time_micros": 1588842268737614, "job": 185, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [4, 2, 14, 0, 0, 0, 0], "immutable_memtables": 0} -914> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-09:04:28.737648) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[4 2 14 0 0 0 0] max score 1.00 -913> 2020-05-07 09:04:28.736 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:347] [JOB 185] Try to delete WAL files size 251706227, prev total WAL file size 251707247, number of live WAL files 2. -912> 2020-05-07 09:04:28.736 7f51b174a700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl.cc:625] ------- DUMPING STATS ------- -911> 2020-05-07 09:04:28.736 7f51b174a700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl.cc:626] ** DB Stats ** Uptime(secs): 2632541.6 total, 68979.1 interval Cumulative writes: 13M writes, 38M keys, 13M commit groups, 1.0 writes per commit group, ingest: 26.80 GB, 0.01 MB/s Cumulative WAL: 13M writes, 6612K syncs, 2.00 writes per sync, written: 26.80 GB, 0.01 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 488K writes, 1454K keys, 488K commit groups, 1.0 writes per commit group, ingest: 960.52 MB, 0.01 MB/s Interval WAL: 488K writes, 244K syncs, 2.00 writes per sync, written: 0.94 MB, 0.01 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 4/0 129.69 MB 1.0 0.0 0.0 0.0 3.6 3.6 0.0 1.0 0.0 202.6 18 115 0.159 0 0 L1 2/0 80.26 MB 0.5 7.7 3.5 4.2 6.9 2.8 0.0 2.0 275.9 249.1 28 28 1.015 24M 3033K L2 14/0 830.07 MB 0.3 14.6 2.7 11.9 12.5 0.6 0.2 4.7 418.7 359.3 36 38 0.938 22M 7955K Sum 20/0 1.02 GB 0.0 22.2 6.2 16.0 23.0 7.0 0.2 6.4 276.5 286.5 82 181 0.455 47M 10M Int 0/0 0.00 KB 0.0 1.1 0.3 0.9 1.1 0.2 0.0 8.7 308.9 305.8 4 7 0.524 1911K 360K Uptime(secs): 2632541.6 total, 68979.1 interval Flush(GB): cumulative 3.614, interval 0.127 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.03 GB write, 0.01 MB/s write, 22.23 GB read, 0.01 MB/s read, 82.3 seconds Interval compaction: 1.10 GB write, 0.02 MB/s write, 1.11 GB read, 0.02 MB/s read, 3.7 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count ** File Read Latency Histogram By Level [default] ** ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 4/0 129.69 MB 1.0 0.0 0.0 0.0 3.6 3.6 0.0 1.0 0.0 202.6 18 115 0.159 0 0 L1 2/0 80.26 MB 0.5 7.7 3.5 4.2 6.9 2.8 0.0 2.0 275.9 249.1 28 28 1.015 24M 3033K L2 14/0 830.07 MB 0.3 14.6 2.7 11.9 12.5 0.6 0.2 4.7 418.7 359.3 36 38 0.938 22M 7955K Sum 20/0 1.02 GB 0.0 22.2 6.2 16.0 23.0 7.0 0.2 6.4 276.5 286.5 82 181 0.455 47M 10M Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0 Uptime(secs): 2632541.6 total, 0.0 interval Flush(GB): cumulative 3.614, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.03 GB write, 0.01 MB/s write, 22.23 GB read, 0.01 MB/s read, 82.3 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count ** File Read Latency Histogram By Level [default] ** -910> 2020-05-07 09:04:28.736 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 186] Compacting 4@0 + 2@1 files to L1, score 1.00 -909> 2020-05-07 09:04:28.736 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 185 Base level 0, inputs: [628(32MB) 626(32MB) 624(32MB) 622(32MB)], [605(66MB) 606(14MB)] -908> 2020-05-07 09:04:28.736 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842268737951, "job": 186, "event": "compaction_started", "compaction_reason": "LevelL0FilesNum", "files_L0": [628, 626, 624, 622], "files_L1": [605, 606], "score": 1, "input_data_size": 220144321} -907> 2020-05-07 09:04:28.982 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 186] Generated table #629: 165303 keys, 68654723 bytes -906> 2020-05-07 09:04:28.982 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842268983667, "cf_name": "default", "job": 186, "event": "table_file_creation", "file_number": 629, "file_size": 68654723, "table_properties": {"data_size": 67110372, "index_size": 978535, "filter_size": 565065, "raw_key_size": 24776244, "raw_average_key_size": 149, "raw_value_size": 53272342, "raw_average_value_size": 322, "num_data_blocks": 15349, "num_entries": 165303, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "113991", "kMergeOperands": "0"}} -905> 2020-05-07 09:04:29.276 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 186] Generated table #630: 277219 keys, 68954839 bytes -904> 2020-05-07 09:04:29.276 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269277375, "cf_name": "default", "job": 186, "event": "table_file_creation", "file_number": 630, "file_size": 68954839, "table_properties": {"data_size": 67112650, "index_size": 997510, "filter_size": 843928, "raw_key_size": 25025206, "raw_average_key_size": 90, "raw_value_size": 54983108, "raw_average_value_size": 198, "num_data_blocks": 15857, "num_entries": 277219, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "173009", "kMergeOperands": "0"}} -903> 2020-05-07 09:04:29.655 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 186] Generated table #631: 509757 keys, 67663615 bytes -902> 2020-05-07 09:04:29.655 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269656254, "cf_name": "default", "job": 186, "event": "table_file_creation", "file_number": 631, "file_size": 67663615, "table_properties": {"data_size": 65218452, "index_size": 1025668, "filter_size": 1418741, "raw_key_size": 24530582, "raw_average_key_size": 48, "raw_value_size": 57089024, "raw_average_value_size": 111, "num_data_blocks": 16469, "num_entries": 509757, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "266712", "kMergeOperands": "21698"}} -901> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 186] Compacted 4@0 + 2@1 files to L1 => 205273177 bytes -900> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/07-09:04:29.660483) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 3 14 0 0 0 0] max score 1.47, MB/sec: 239.7 rd, 223.5 wr, level 1, files in(4, 2) out(3) MB in(129.7, 80.3) out(195.8), read-write-amplify(3.1) write-amplify(1.5) OK, records in: 1011522, records dropped: 5408 -899> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/07-09:04:29.660508) EVENT_LOG_v1 {"time_micros": 1588842269660498, "job": 186, "event": "compaction_finished", "compaction_time_micros": 918520, "output_level": 1, "num_output_files": 3, "total_output_size": 205273177, "num_input_records": 1006366, "num_output_records": 952279, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 3, 14, 0, 0, 0, 0]} -898> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660623, "job": 186, "event": "table_file_deletion", "file_number": 628} -897> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660656, "job": 186, "event": "table_file_deletion", "file_number": 626} -896> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660685, "job": 186, "event": "table_file_deletion", "file_number": 624} -895> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660720, "job": 186, "event": "table_file_deletion", "file_number": 622} -894> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660752, "job": 186, "event": "table_file_deletion", "file_number": 606} -893> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660777, "job": 186, "event": "table_file_deletion", "file_number": 605} -892> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 187] Compacting 1@1 + 6@2 files to L2, score 1.47 -891> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 186 Base level 1, inputs: [630(65MB)], [616(65MB) 617(65MB) 618(65MB) 619(65MB) 620(61KB) 593(65MB)] -890> 2020-05-07 09:04:29.659 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269660831, "job": 187, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L1": [630], "files_L2": [616, 617, 618, 619, 620, 593], "score": 1.47097, "input_data_size": 410701863} -889> 2020-05-07 09:04:29.822 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 187] Generated table #632: 57824 keys, 68218410 bytes -888> 2020-05-07 09:04:29.822 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269823941, "cf_name": "default", "job": 187, "event": "table_file_creation", "file_number": 632, "file_size": 68218410, "table_properties": {"data_size": 67114142, "index_size": 806068, "filter_size": 297451, "raw_key_size": 8937532, "raw_average_key_size": 154, "raw_value_size": 59880666, "raw_average_value_size": 1035, "num_data_blocks": 15358, "num_entries": 57824, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -887> 2020-05-07 09:04:29.994 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 187] Generated table #633: 58157 keys, 68225872 bytes -886> 2020-05-07 09:04:29.994 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842269995905, "cf_name": "default", "job": 187, "event": "table_file_creation", "file_number": 633, "file_size": 68225872, "table_properties": {"data_size": 67111591, "index_size": 815261, "filter_size": 298271, "raw_key_size": 8990503, "raw_average_key_size": 154, "raw_value_size": 59878886, "raw_average_value_size": 1029, "num_data_blocks": 15369, "num_entries": 58157, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -885> 2020-05-07 09:04:30.166 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 187] Generated table #634: 58109 keys, 68224998 bytes -884> 2020-05-07 09:04:30.166 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270167371, "cf_name": "default", "job": 187, "event": "table_file_creation", "file_number": 634, "file_size": 68224998, "table_properties": {"data_size": 67113046, "index_size": 813018, "filter_size": 298185, "raw_key_size": 8982554, "raw_average_key_size": 154, "raw_value_size": 59873851, "raw_average_value_size": 1030, "num_data_blocks": 15363, "num_entries": 58109, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -883> 2020-05-07 09:04:30.346 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 187] Generated table #635: 57903 keys, 68220872 bytes -882> 2020-05-07 09:04:30.346 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270347325, "cf_name": "default", "job": 187, "event": "table_file_creation", "file_number": 635, "file_size": 68220872, "table_properties": {"data_size": 67113074, "index_size": 809475, "filter_size": 297574, "raw_key_size": 8949808, "raw_average_key_size": 154, "raw_value_size": 59887481, "raw_average_value_size": 1034, "num_data_blocks": 15332, "num_entries": 57903, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -881> 2020-05-07 09:04:30.599 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 187] Generated table #636: 199940 keys, 68776150 bytes -880> 2020-05-07 09:04:30.599 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270601092, "cf_name": "default", "job": 187, "event": "table_file_creation", "file_number": 636, "file_size": 68776150, "table_properties": {"data_size": 67112033, "index_size": 1014365, "filter_size": 649002, "raw_key_size": 11303358, "raw_average_key_size": 56, "raw_value_size": 62363316, "raw_average_value_size": 311, "num_data_blocks": 16632, "num_entries": 199940, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -879> 2020-05-07 09:04:30.601 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 187] Generated table #637: 832 keys, 241965 bytes -878> 2020-05-07 09:04:30.601 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270602711, "cf_name": "default", "job": 187, "event": "table_file_creation", "file_number": 637, "file_size": 241965, "table_properties": {"data_size": 234943, "index_size": 3682, "filter_size": 2601, "raw_key_size": 41600, "raw_average_key_size": 50, "raw_value_size": 220794, "raw_average_value_size": 265, "num_data_blocks": 60, "num_entries": 832, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -877> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 187] Compacted 1@1 + 6@2 files to L2 => 341908267 bytes -876> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/07-09:04:30.606533) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 2 14 0 0 0 0] max score 0.90, MB/sec: 436.0 rd, 363.0 wr, level 2, files in(1, 6) out(6) MB in(65.8, 325.9) out(326.1), read-write-amplify(10.9) write-amplify(5.0) OK, records in: 710784, records dropped: 2780 -875> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/07-09:04:30.606558) EVENT_LOG_v1 {"time_micros": 1588842270606548, "job": 187, "event": "compaction_finished", "compaction_time_micros": 941905, "output_level": 2, "num_output_files": 6, "total_output_size": 341908267, "num_input_records": 710784, "num_output_records": 432765, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 2, 14, 0, 0, 0, 0]} -874> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606678, "job": 187, "event": "table_file_deletion", "file_number": 630} -873> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606712, "job": 187, "event": "table_file_deletion", "file_number": 620} -872> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606734, "job": 187, "event": "table_file_deletion", "file_number": 619} -871> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606759, "job": 187, "event": "table_file_deletion", "file_number": 618} -870> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606796, "job": 187, "event": "table_file_deletion", "file_number": 617} -869> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606817, "job": 187, "event": "table_file_deletion", "file_number": 616} -868> 2020-05-07 09:04:30.605 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588842270606841, "job": 187, "event": "table_file_deletion", "file_number": 593} -867> 2020-05-07 09:10:32.650 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -866> 2020-05-07 09:10:32.650 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -865> 2020-05-07 09:10:32.650 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -864> 2020-05-07 09:10:32.730 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -863> 2020-05-07 09:25:32.730 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -862> 2020-05-07 09:25:32.730 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -861> 2020-05-07 09:25:32.730 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -860> 2020-05-07 09:25:33.192 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -859> 2020-05-07 09:55:32.811 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -858> 2020-05-07 09:55:32.811 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -857> 2020-05-07 09:55:32.811 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -856> 2020-05-07 09:55:33.291 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -855> 2020-05-07 10:25:32.891 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -854> 2020-05-07 10:25:32.891 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -853> 2020-05-07 10:25:32.891 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -852> 2020-05-07 10:25:33.382 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -851> 2020-05-07 10:55:32.972 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -850> 2020-05-07 10:55:32.972 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -849> 2020-05-07 10:55:32.972 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -848> 2020-05-07 10:55:33.393 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -847> 2020-05-07 11:25:33.052 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -846> 2020-05-07 11:25:33.052 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -845> 2020-05-07 11:25:33.052 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -844> 2020-05-07 11:25:33.132 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -843> 2020-05-07 11:40:33.133 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -842> 2020-05-07 11:40:33.133 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -841> 2020-05-07 11:40:33.133 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -840> 2020-05-07 11:40:33.585 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -839> 2020-05-07 12:10:33.213 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -838> 2020-05-07 12:10:33.213 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -837> 2020-05-07 12:10:33.213 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -836> 2020-05-07 12:10:33.685 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -835> 2020-05-07 12:37:01.183 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -834> 2020-05-07 12:38:53.642 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -833> 2020-05-07 12:40:33.294 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -832> 2020-05-07 12:40:33.294 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -831> 2020-05-07 12:40:33.294 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -830> 2020-05-07 12:40:33.777 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -829> 2020-05-07 13:10:33.375 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -828> 2020-05-07 13:10:33.375 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -827> 2020-05-07 13:10:33.375 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -826> 2020-05-07 13:10:33.415 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -825> 2020-05-07 13:25:33.415 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -824> 2020-05-07 13:25:33.415 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -823> 2020-05-07 13:25:33.415 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -822> 2020-05-07 13:25:33.415 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -821> 2020-05-07 13:40:33.415 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -820> 2020-05-07 13:40:33.415 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -819> 2020-05-07 13:40:33.415 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -818> 2020-05-07 13:40:33.496 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -817> 2020-05-07 13:55:33.496 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -816> 2020-05-07 13:55:33.496 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -815> 2020-05-07 13:55:33.496 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -814> 2020-05-07 13:55:33.576 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -813> 2020-05-07 13:57:07.845 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 625 from recycle list -812> 2020-05-07 13:57:07.845 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #638. Immutable memtables: 0. -811> 2020-05-07 13:57:07.846 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-13:57:07.847120) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -810> 2020-05-07 13:57:07.846 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 188] Flushing memtable with next log file: 638 -809> 2020-05-07 13:57:07.846 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588859827847250, "job": 188, "event": "flush_started", "num_memtables": 1, "num_entries": 364537, "num_deletes": 74123, "memory_usage": 260054744, "flush_reason": "Write Buffer Full"} -808> 2020-05-07 13:57:07.846 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 188] Level-0 flush table #639: started -807> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588859828016365, "cf_name": "default", "job": 188, "event": "table_file_creation", "file_number": 639, "file_size": 33963911, "table_properties": {"data_size": 33099155, "index_size": 486284, "filter_size": 377716, "raw_key_size": 11955168, "raw_average_key_size": 98, "raw_value_size": 27113793, "raw_average_value_size": 223, "num_data_blocks": 7799, "num_entries": 121309, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "70670", "kMergeOperands": "1108"}} -806> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 188] Level-0 flush table #639: 33963911 bytes OK -805> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 627 to recycle list -804> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-13:57:08.016404) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #639 started -803> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-13:57:08.016570) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #639: memtable #1 done -802> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-13:57:08.016587) EVENT_LOG_v1 {"time_micros": 1588859828016581, "job": 188, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [1, 2, 14, 0, 0, 0, 0], "immutable_memtables": 0} -801> 2020-05-07 13:57:08.015 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-13:57:08.016610) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[1 2 14 0 0 0 0] max score 0.90 -800> 2020-05-07 14:10:33.576 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -799> 2020-05-07 14:10:33.576 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -798> 2020-05-07 14:10:33.576 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -797> 2020-05-07 14:10:33.975 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -796> 2020-05-07 14:40:33.657 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -795> 2020-05-07 14:40:33.657 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -794> 2020-05-07 14:40:33.657 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -793> 2020-05-07 14:40:33.697 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -792> 2020-05-07 14:55:33.697 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -791> 2020-05-07 14:55:33.697 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -790> 2020-05-07 14:55:33.697 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -789> 2020-05-07 14:55:33.697 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -788> 2020-05-07 15:10:33.698 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -787> 2020-05-07 15:10:33.698 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -786> 2020-05-07 15:10:33.698 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -785> 2020-05-07 15:10:33.778 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -784> 2020-05-07 15:25:33.778 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -783> 2020-05-07 15:25:33.778 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -782> 2020-05-07 15:25:33.778 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -781> 2020-05-07 15:25:34.251 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -780> 2020-05-07 15:55:33.859 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -779> 2020-05-07 15:55:33.859 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -778> 2020-05-07 15:55:33.859 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -777> 2020-05-07 15:55:34.346 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -776> 2020-05-07 16:25:33.940 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -775> 2020-05-07 16:25:33.940 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -774> 2020-05-07 16:25:33.940 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -773> 2020-05-07 16:25:33.980 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -772> 2020-05-07 16:40:33.980 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -771> 2020-05-07 16:40:33.980 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -770> 2020-05-07 16:40:33.980 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -769> 2020-05-07 16:40:34.060 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -768> 2020-05-07 16:55:34.060 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -767> 2020-05-07 16:55:34.060 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -766> 2020-05-07 16:55:34.060 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -765> 2020-05-07 16:55:34.537 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -764> 2020-05-07 17:19:40.503 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -763> 2020-05-07 17:21:32.204 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -762> 2020-05-07 17:21:36.158 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -761> 2020-05-07 17:23:27.897 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -760> 2020-05-07 17:23:29.635 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -759> 2020-05-07 17:25:21.381 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -758> 2020-05-07 17:25:21.828 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -757> 2020-05-07 17:25:34.141 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -756> 2020-05-07 17:25:34.141 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -755> 2020-05-07 17:25:34.141 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -754> 2020-05-07 17:25:34.548 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -753> 2020-05-07 17:27:13.505 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -752> 2020-05-07 17:55:34.222 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -751> 2020-05-07 17:55:34.222 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -750> 2020-05-07 17:55:34.222 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -749> 2020-05-07 17:55:34.636 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -748> 2020-05-07 18:25:34.303 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -747> 2020-05-07 18:25:34.303 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -746> 2020-05-07 18:25:34.303 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -745> 2020-05-07 18:25:34.783 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -744> 2020-05-07 18:42:14.037 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 627 from recycle list -743> 2020-05-07 18:42:14.037 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #640. Immutable memtables: 0. -742> 2020-05-07 18:42:14.037 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-18:42:14.037306) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -741> 2020-05-07 18:42:14.037 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 189] Flushing memtable with next log file: 640 -740> 2020-05-07 18:42:14.037 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588876934037554, "job": 189, "event": "flush_started", "num_memtables": 1, "num_entries": 357693, "num_deletes": 73137, "memory_usage": 260081032, "flush_reason": "Write Buffer Full"} -739> 2020-05-07 18:42:14.037 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 189] Level-0 flush table #641: started -738> 2020-05-07 18:42:14.205 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588876934205855, "cf_name": "default", "job": 189, "event": "table_file_creation", "file_number": 641, "file_size": 33500588, "table_properties": {"data_size": 32636154, "index_size": 490973, "filter_size": 372705, "raw_key_size": 11755696, "raw_average_key_size": 98, "raw_value_size": 26747751, "raw_average_value_size": 223, "num_data_blocks": 7679, "num_entries": 119717, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "69826", "kMergeOperands": "1049"}} -737> 2020-05-07 18:42:14.205 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 189] Level-0 flush table #641: 33500588 bytes OK -736> 2020-05-07 18:42:14.206 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 638 to recycle list -735> 2020-05-07 18:42:14.206 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-18:42:14.205898) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #641 started -734> 2020-05-07 18:42:14.206 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-18:42:14.206073) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #641: memtable #1 done -733> 2020-05-07 18:42:14.206 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-18:42:14.206090) EVENT_LOG_v1 {"time_micros": 1588876934206083, "job": 189, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 2, 14, 0, 0, 0, 0], "immutable_memtables": 0} -732> 2020-05-07 18:42:14.206 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-18:42:14.206113) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[2 2 14 0 0 0 0] max score 0.90 -731> 2020-05-07 18:42:14.206 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:347] [JOB 189] Try to delete WAL files size 251956181, prev total WAL file size 251957237, number of live WAL files 2. -730> 2020-05-07 18:55:34.384 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -729> 2020-05-07 18:55:34.384 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -728> 2020-05-07 18:55:34.384 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -727> 2020-05-07 18:55:34.873 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -726> 2020-05-07 19:25:34.465 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -725> 2020-05-07 19:25:34.465 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -724> 2020-05-07 19:25:34.465 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -723> 2020-05-07 19:25:34.505 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -722> 2020-05-07 19:40:34.506 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -721> 2020-05-07 19:40:34.506 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -720> 2020-05-07 19:40:34.506 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -719> 2020-05-07 19:40:34.586 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -718> 2020-05-07 19:55:34.586 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -717> 2020-05-07 19:55:34.586 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -716> 2020-05-07 19:55:34.586 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -715> 2020-05-07 19:55:35.066 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -714> 2020-05-07 20:25:34.667 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -713> 2020-05-07 20:25:34.667 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -712> 2020-05-07 20:25:34.667 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -711> 2020-05-07 20:25:34.667 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -710> 2020-05-07 20:40:34.668 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -709> 2020-05-07 20:40:34.668 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -708> 2020-05-07 20:40:34.668 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -707> 2020-05-07 20:40:34.748 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -706> 2020-05-07 20:55:34.749 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -705> 2020-05-07 20:55:34.749 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -704> 2020-05-07 20:55:34.749 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -703> 2020-05-07 20:55:35.174 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -702> 2020-05-07 21:25:34.830 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -701> 2020-05-07 21:25:34.830 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -700> 2020-05-07 21:25:34.830 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -699> 2020-05-07 21:25:34.910 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -698> 2020-05-07 21:40:34.910 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -697> 2020-05-07 21:40:34.910 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -696> 2020-05-07 21:40:34.910 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -695> 2020-05-07 21:40:35.365 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -694> 2020-05-07 21:50:38.783 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -693> 2020-05-07 21:52:30.531 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -692> 2020-05-07 21:52:32.889 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -691> 2020-05-07 21:54:24.842 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -690> 2020-05-07 21:54:27.976 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -689> 2020-05-07 21:56:19.727 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -688> 2020-05-07 21:56:20.845 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -687> 2020-05-07 21:58:12.623 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -686> 2020-05-07 21:58:15.297 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -685> 2020-05-07 22:00:07.041 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -684> 2020-05-07 22:00:07.949 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -683> 2020-05-07 22:01:59.517 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -682> 2020-05-07 22:01:59.999 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -681> 2020-05-07 22:03:51.590 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -680> 2020-05-07 22:03:54.633 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -679> 2020-05-07 22:05:46.328 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -678> 2020-05-07 22:10:34.991 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -677> 2020-05-07 22:10:34.991 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -676> 2020-05-07 22:10:34.991 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -675> 2020-05-07 22:10:35.460 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -674> 2020-05-07 22:40:35.072 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -673> 2020-05-07 22:40:35.072 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -672> 2020-05-07 22:40:35.072 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -671> 2020-05-07 22:40:35.112 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -670> 2020-05-07 22:55:35.113 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -669> 2020-05-07 22:55:35.113 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -668> 2020-05-07 22:55:35.113 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -667> 2020-05-07 22:55:35.600 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -666> 2020-05-07 23:15:19.014 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 638 from recycle list -665> 2020-05-07 23:15:19.014 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #642. Immutable memtables: 0. -664> 2020-05-07 23:15:19.014 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-23:15:19.015753) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -663> 2020-05-07 23:15:19.014 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 190] Flushing memtable with next log file: 642 -662> 2020-05-07 23:15:19.014 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588893319015988, "job": 190, "event": "flush_started", "num_memtables": 1, "num_entries": 345477, "num_deletes": 70000, "memory_usage": 260057616, "flush_reason": "Write Buffer Full"} -661> 2020-05-07 23:15:19.014 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 190] Level-0 flush table #643: started -660> 2020-05-07 23:15:19.172 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588893319174026, "cf_name": "default", "job": 190, "event": "table_file_creation", "file_number": 643, "file_size": 32138469, "table_properties": {"data_size": 31313600, "index_size": 466512, "filter_size": 357601, "raw_key_size": 11258123, "raw_average_key_size": 98, "raw_value_size": 25656671, "raw_average_value_size": 223, "num_data_blocks": 7367, "num_entries": 114863, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "66880", "kMergeOperands": "1055"}} -659> 2020-05-07 23:15:19.172 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 190] Level-0 flush table #643: 32138469 bytes OK -658> 2020-05-07 23:15:19.173 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 640 to recycle list -657> 2020-05-07 23:15:19.173 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-23:15:19.174067) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #643 started -656> 2020-05-07 23:15:19.173 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-23:15:19.174256) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #643: memtable #1 done -655> 2020-05-07 23:15:19.173 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-23:15:19.174282) EVENT_LOG_v1 {"time_micros": 1588893319174273, "job": 190, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [3, 2, 14, 0, 0, 0, 0], "immutable_memtables": 0} -654> 2020-05-07 23:15:19.173 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/07-23:15:19.174304) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[3 2 14 0 0 0 0] max score 0.90 -653> 2020-05-07 23:25:35.194 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -652> 2020-05-07 23:25:35.194 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -651> 2020-05-07 23:25:35.194 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -650> 2020-05-07 23:25:35.697 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -649> 2020-05-07 23:55:35.275 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -648> 2020-05-07 23:55:35.275 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -647> 2020-05-07 23:55:35.275 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -646> 2020-05-07 23:55:35.315 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -645> 2020-05-08 00:10:35.315 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -644> 2020-05-08 00:10:35.315 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -643> 2020-05-08 00:10:35.315 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -642> 2020-05-08 00:10:35.395 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -641> 2020-05-08 00:25:35.396 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -640> 2020-05-08 00:25:35.396 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -639> 2020-05-08 00:25:35.396 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -638> 2020-05-08 00:25:35.802 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -637> 2020-05-08 00:55:35.478 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -636> 2020-05-08 00:55:35.478 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -635> 2020-05-08 00:55:35.478 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -634> 2020-05-08 00:55:35.518 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -633> 2020-05-08 01:10:35.518 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -632> 2020-05-08 01:10:35.518 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -631> 2020-05-08 01:10:35.518 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -630> 2020-05-08 01:10:35.518 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -629> 2020-05-08 01:25:35.519 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -628> 2020-05-08 01:25:35.519 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -627> 2020-05-08 01:25:35.519 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -626> 2020-05-08 01:25:35.605 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -625> 2020-05-08 01:40:35.605 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -624> 2020-05-08 01:40:35.605 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -623> 2020-05-08 01:40:35.605 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -622> 2020-05-08 01:40:36.091 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -621> 2020-05-08 02:10:35.686 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -620> 2020-05-08 02:10:35.686 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -619> 2020-05-08 02:10:35.686 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -618> 2020-05-08 02:10:35.726 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -617> 2020-05-08 02:25:35.727 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -616> 2020-05-08 02:25:35.727 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -615> 2020-05-08 02:25:35.727 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -614> 2020-05-08 02:25:35.808 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -613> 2020-05-08 02:40:35.808 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -612> 2020-05-08 02:40:35.808 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -611> 2020-05-08 02:40:35.808 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -610> 2020-05-08 02:40:36.280 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -609> 2020-05-08 02:47:58.042 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -608> 2020-05-08 02:49:49.890 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -607> 2020-05-08 02:49:50.571 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -606> 2020-05-08 02:51:42.430 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -605> 2020-05-08 02:51:43.138 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -604> 2020-05-08 02:53:35.061 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -603> 2020-05-08 02:53:36.494 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -602> 2020-05-08 02:55:28.487 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -601> 2020-05-08 03:10:35.889 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -600> 2020-05-08 03:10:35.889 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -599> 2020-05-08 03:10:35.889 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -598> 2020-05-08 03:10:36.374 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -597> 2020-05-08 03:16:01.510 7f51b77e1700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw (PID: 1273024) UID: 0 -596> 2020-05-08 03:16:01.545 7f51b77e1700 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw (PID: 1273025) UID: 0 -595> 2020-05-08 03:40:35.971 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -594> 2020-05-08 03:40:35.971 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -593> 2020-05-08 03:40:35.971 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -592> 2020-05-08 03:40:36.378 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -591> 2020-05-08 03:58:51.819 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 640 from recycle list -590> 2020-05-08 03:58:51.819 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #644. Immutable memtables: 0. -589> 2020-05-08 03:58:51.819 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-03:58:51.820471) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -588> 2020-05-08 03:58:51.819 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 191] Flushing memtable with next log file: 644 -587> 2020-05-08 03:58:51.819 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910331820592, "job": 191, "event": "flush_started", "num_memtables": 1, "num_entries": 358609, "num_deletes": 72948, "memory_usage": 260050072, "flush_reason": "Write Buffer Full"} -586> 2020-05-08 03:58:51.819 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 191] Level-0 flush table #645: started -585> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910331986452, "cf_name": "default", "job": 191, "event": "table_file_creation", "file_number": 645, "file_size": 34094112, "table_properties": {"data_size": 33226631, "index_size": 492212, "filter_size": 374513, "raw_key_size": 11827220, "raw_average_key_size": 98, "raw_value_size": 27304366, "raw_average_value_size": 227, "num_data_blocks": 7793, "num_entries": 119904, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "69530", "kMergeOperands": "1069"}} -584> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 191] Level-0 flush table #645: 34094112 bytes OK -583> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 642 to recycle list -582> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-03:58:51.986492) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #645 started -581> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-03:58:51.986661) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #645: memtable #1 done -580> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-03:58:51.986677) EVENT_LOG_v1 {"time_micros": 1588910331986670, "job": 191, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [4, 2, 14, 0, 0, 0, 0], "immutable_memtables": 0} -579> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-03:58:51.986702) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[4 2 14 0 0 0 0] max score 1.00 -578> 2020-05-08 03:58:51.985 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:347] [JOB 191] Try to delete WAL files size 251944299, prev total WAL file size 251944311, number of live WAL files 2. -577> 2020-05-08 03:58:51.985 7f51b174a700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl.cc:625] ------- DUMPING STATS ------- -576> 2020-05-08 03:58:51.985 7f51b174a700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl.cc:626] ** DB Stats ** Uptime(secs): 2700604.8 total, 68063.2 interval Cumulative writes: 13M writes, 39M keys, 13M commit groups, 1.0 writes per commit group, ingest: 27.74 GB, 0.01 MB/s Cumulative WAL: 13M writes, 6853K syncs, 2.00 writes per sync, written: 27.74 GB, 0.01 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 481K writes, 1426K keys, 481K commit groups, 1.0 writes per commit group, ingest: 961.24 MB, 0.01 MB/s Interval WAL: 481K writes, 240K syncs, 2.00 writes per sync, written: 0.94 MB, 0.01 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 4/0 127.50 MB 1.0 0.0 0.0 0.0 3.7 3.7 0.0 1.0 0.0 202.3 19 119 0.159 0 0 L1 2/0 130.00 MB 0.9 7.9 3.6 4.2 7.1 2.9 0.0 2.0 274.4 248.0 29 29 1.012 25M 3087K L2 14/0 830.22 MB 0.3 15.0 2.8 12.2 12.8 0.6 0.2 4.7 418.7 358.9 37 39 0.938 23M 8233K Sum 20/0 1.06 GB 0.0 22.8 6.4 16.4 23.7 7.2 0.2 6.3 275.4 285.6 85 187 0.454 48M 11M Int 0/0 0.00 KB 0.0 0.6 0.2 0.4 0.6 0.2 0.0 5.1 238.6 257.5 3 6 0.420 1722K 332K Uptime(secs): 2700604.8 total, 68063.2 interval Flush(GB): cumulative 3.739, interval 0.125 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.67 GB write, 0.01 MB/s write, 22.82 GB read, 0.01 MB/s read, 84.9 seconds Interval compaction: 0.63 GB write, 0.01 MB/s write, 0.59 GB read, 0.01 MB/s read, 2.5 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count ** File Read Latency Histogram By Level [default] ** ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 4/0 127.50 MB 1.0 0.0 0.0 0.0 3.7 3.7 0.0 1.0 0.0 202.3 19 119 0.159 0 0 L1 2/0 130.00 MB 0.9 7.9 3.6 4.2 7.1 2.9 0.0 2.0 274.4 248.0 29 29 1.012 25M 3087K L2 14/0 830.22 MB 0.3 15.0 2.8 12.2 12.8 0.6 0.2 4.7 418.7 358.9 37 39 0.938 23M 8233K Sum 20/0 1.06 GB 0.0 22.8 6.4 16.4 23.7 7.2 0.2 6.3 275.4 285.6 85 187 0.454 48M 11M Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0 Uptime(secs): 2700604.8 total, 0.0 interval Flush(GB): cumulative 3.739, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 23.67 GB write, 0.01 MB/s write, 22.82 GB read, 0.01 MB/s read, 84.9 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count ** File Read Latency Histogram By Level [default] ** -575> 2020-05-08 03:58:51.985 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 192] Compacting 4@0 + 2@1 files to L1, score 1.00 -574> 2020-05-08 03:58:51.985 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 191 Base level 0, inputs: [645(32MB) 643(30MB) 641(31MB) 639(32MB)], [629(65MB) 631(64MB)] -573> 2020-05-08 03:58:51.985 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910331987007, "job": 192, "event": "compaction_started", "compaction_reason": "LevelL0FilesNum", "files_L0": [645, 643, 641, 639], "files_L1": [629, 631], "score": 1, "input_data_size": 270015418} -572> 2020-05-08 03:58:52.240 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 192] Generated table #646: 177296 keys, 68684506 bytes -571> 2020-05-08 03:58:52.240 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910332241789, "cf_name": "default", "job": 192, "event": "table_file_creation", "file_number": 646, "file_size": 68684506, "table_properties": {"data_size": 67114445, "index_size": 974225, "filter_size": 595085, "raw_key_size": 25865534, "raw_average_key_size": 145, "raw_value_size": 53078385, "raw_average_value_size": 299, "num_data_blocks": 15353, "num_entries": 177296, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "126155", "kMergeOperands": "0"}} -570> 2020-05-08 03:58:52.480 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 192] Generated table #647: 154776 keys, 68626510 bytes -569> 2020-05-08 03:58:52.480 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910332481940, "cf_name": "default", "job": 192, "event": "table_file_creation", "file_number": 647, "file_size": 68626510, "table_properties": {"data_size": 67109281, "index_size": 977811, "filter_size": 538667, "raw_key_size": 23925336, "raw_average_key_size": 154, "raw_value_size": 53418506, "raw_average_value_size": 345, "num_data_blocks": 15373, "num_entries": 154776, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "102974", "kMergeOperands": "0"}} -568> 2020-05-08 03:58:52.787 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 192] Generated table #648: 343341 keys, 69142763 bytes -567> 2020-05-08 03:58:52.787 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910332788945, "cf_name": "default", "job": 192, "event": "table_file_creation", "file_number": 648, "file_size": 69142763, "table_properties": {"data_size": 67112842, "index_size": 1020966, "filter_size": 1008204, "raw_key_size": 23665609, "raw_average_key_size": 68, "raw_value_size": 57135318, "raw_average_value_size": 166, "num_data_blocks": 16308, "num_entries": 343341, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "187314", "kMergeOperands": "0"}} -566> 2020-05-08 03:58:53.102 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 192] Generated table #649: 432102 keys, 52778391 bytes -565> 2020-05-08 03:58:53.102 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333103763, "cf_name": "default", "job": 192, "event": "table_file_creation", "file_number": 649, "file_size": 52778391, "table_properties": {"data_size": 50788315, "index_size": 796619, "filter_size": 1192704, "raw_key_size": 20650743, "raw_average_key_size": 47, "raw_value_size": 43925775, "raw_average_value_size": 101, "num_data_blocks": 12821, "num_entries": 432102, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "238644", "kMergeOperands": "21698"}} -564> 2020-05-08 03:58:53.106 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 192] Compacted 4@0 + 2@1 files to L1 => 259232170 bytes -563> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-03:58:53.108236) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 4 14 0 0 0 0] max score 1.88, MB/sec: 241.7 rd, 232.1 wr, level 1, files in(4, 2) out(4) MB in(127.5, 130.0) out(247.2), read-write-amplify(4.0) write-amplify(1.9) OK, records in: 1150853, records dropped: 390 -562> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-03:58:53.108264) EVENT_LOG_v1 {"time_micros": 1588910333108254, "job": 192, "event": "compaction_finished", "compaction_time_micros": 1117021, "output_level": 1, "num_output_files": 4, "total_output_size": 259232170, "num_input_records": 1146552, "num_output_records": 1107515, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 4, 14, 0, 0, 0, 0]} -561> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108387, "job": 192, "event": "table_file_deletion", "file_number": 645} -560> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108432, "job": 192, "event": "table_file_deletion", "file_number": 643} -559> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108459, "job": 192, "event": "table_file_deletion", "file_number": 641} -558> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108484, "job": 192, "event": "table_file_deletion", "file_number": 639} -557> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108517, "job": 192, "event": "table_file_deletion", "file_number": 631} -556> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108548, "job": 192, "event": "table_file_deletion", "file_number": 629} -555> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 193] Compacting 1@1 + 4@2 files to L2, score 1.88 -554> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 192 Base level 1, inputs: [646(65MB)], [607(65MB) 608(65MB) 609(65MB) 610(65MB)] -553> 2020-05-08 03:58:53.107 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333108604, "job": 193, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L1": [646], "files_L2": [607, 608, 609, 610], "score": 1.88224, "input_data_size": 341563393} -552> 2020-05-08 03:58:53.293 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 193] Generated table #650: 58884 keys, 68216889 bytes -551> 2020-05-08 03:58:53.293 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333294487, "cf_name": "default", "job": 193, "event": "table_file_creation", "file_number": 650, "file_size": 68216889, "table_properties": {"data_size": 67111455, "index_size": 804590, "filter_size": 300095, "raw_key_size": 8966874, "raw_average_key_size": 152, "raw_value_size": 59873731, "raw_average_value_size": 1016, "num_data_blocks": 15329, "num_entries": 58884, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -550> 2020-05-08 03:58:53.471 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 193] Generated table #651: 58025 keys, 68218420 bytes -549> 2020-05-08 03:58:53.471 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333472936, "cf_name": "default", "job": 193, "event": "table_file_creation", "file_number": 651, "file_size": 68218420, "table_properties": {"data_size": 67109791, "index_size": 809987, "filter_size": 297893, "raw_key_size": 8968796, "raw_average_key_size": 154, "raw_value_size": 59869148, "raw_average_value_size": 1031, "num_data_blocks": 15355, "num_entries": 58025, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -548> 2020-05-08 03:58:53.649 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 193] Generated table #652: 57603 keys, 68204118 bytes -547> 2020-05-08 03:58:53.649 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333650730, "cf_name": "default", "job": 193, "event": "table_file_creation", "file_number": 652, "file_size": 68204118, "table_properties": {"data_size": 67110356, "index_size": 796117, "filter_size": 296896, "raw_key_size": 8901244, "raw_average_key_size": 154, "raw_value_size": 59884828, "raw_average_value_size": 1039, "num_data_blocks": 15349, "num_entries": 57603, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -546> 2020-05-08 03:58:53.826 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 193] Generated table #653: 57745 keys, 67986165 bytes -545> 2020-05-08 03:58:53.826 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333827599, "cf_name": "default", "job": 193, "event": "table_file_creation", "file_number": 653, "file_size": 67986165, "table_properties": {"data_size": 66876692, "index_size": 812065, "filter_size": 296659, "raw_key_size": 8925651, "raw_average_key_size": 154, "raw_value_size": 59665903, "raw_average_value_size": 1033, "num_data_blocks": 15294, "num_entries": 57745, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -544> 2020-05-08 03:58:53.828 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 193] Compacted 1@1 + 4@2 files to L2 => 272625592 bytes -543> 2020-05-08 03:58:53.828 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-03:58:53.829946) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 3 14 0 0 0 0] max score 1.29, MB/sec: 475.0 rd, 379.2 wr, level 2, files in(1, 4) out(4) MB in(65.5, 260.2) out(260.0), read-write-amplify(8.9) write-amplify(4.0) OK, records in: 410088, records dropped: 17782 -542> 2020-05-08 03:58:53.828 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-03:58:53.829970) EVENT_LOG_v1 {"time_micros": 1588910333829961, "job": 193, "event": "compaction_finished", "compaction_time_micros": 719044, "output_level": 2, "num_output_files": 4, "total_output_size": 272625592, "num_input_records": 410080, "num_output_records": 232257, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 3, 14, 0, 0, 0, 0]} -541> 2020-05-08 03:58:53.828 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333830084, "job": 193, "event": "table_file_deletion", "file_number": 646} -540> 2020-05-08 03:58:53.828 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333830119, "job": 193, "event": "table_file_deletion", "file_number": 610} -539> 2020-05-08 03:58:53.829 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333830159, "job": 193, "event": "table_file_deletion", "file_number": 609} -538> 2020-05-08 03:58:53.829 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333830208, "job": 193, "event": "table_file_deletion", "file_number": 608} -537> 2020-05-08 03:58:53.829 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333830229, "job": 193, "event": "table_file_deletion", "file_number": 607} -536> 2020-05-08 03:58:53.829 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 194] Compacting 1@1 + 6@2 files to L2, score 1.29 -535> 2020-05-08 03:58:53.829 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 193 Base level 1, inputs: [647(65MB)], [653(64MB) 611(65MB) 612(65MB) 613(65MB) 632(65MB) 633(65MB)] -534> 2020-05-08 03:58:53.829 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333830282, "job": 194, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L1": [647], "files_L2": [653, 611, 612, 613, 632, 633], "score": 1.28712, "input_data_size": 477721020} -533> 2020-05-08 03:58:53.986 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #654: 57941 keys, 68226494 bytes -532> 2020-05-08 03:58:53.986 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910333987724, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 654, "file_size": 68226494, "table_properties": {"data_size": 67111850, "index_size": 816203, "filter_size": 297692, "raw_key_size": 8955951, "raw_average_key_size": 154, "raw_value_size": 59874869, "raw_average_value_size": 1033, "num_data_blocks": 15348, "num_entries": 57941, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -531> 2020-05-08 03:58:54.165 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #655: 57992 keys, 68227760 bytes -530> 2020-05-08 03:58:54.166 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334167111, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 655, "file_size": 68227760, "table_properties": {"data_size": 67114738, "index_size": 814432, "filter_size": 297841, "raw_key_size": 8963634, "raw_average_key_size": 154, "raw_value_size": 59878207, "raw_average_value_size": 1032, "num_data_blocks": 15365, "num_entries": 57992, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -529> 2020-05-08 03:58:54.344 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #656: 58184 keys, 68227715 bytes -528> 2020-05-08 03:58:54.344 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334345688, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 656, "file_size": 68227715, "table_properties": {"data_size": 67110923, "index_size": 817704, "filter_size": 298339, "raw_key_size": 8994132, "raw_average_key_size": 154, "raw_value_size": 59867426, "raw_average_value_size": 1028, "num_data_blocks": 15371, "num_entries": 58184, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -527> 2020-05-08 03:58:54.524 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #657: 57858 keys, 68209658 bytes -526> 2020-05-08 03:58:54.524 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334526103, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 657, "file_size": 68209658, "table_properties": {"data_size": 67110527, "index_size": 800817, "filter_size": 297565, "raw_key_size": 8942144, "raw_average_key_size": 154, "raw_value_size": 59882869, "raw_average_value_size": 1034, "num_data_blocks": 15366, "num_entries": 57858, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -525> 2020-05-08 03:58:54.700 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #658: 57802 keys, 68217974 bytes -524> 2020-05-08 03:58:54.700 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334701356, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 658, "file_size": 68217974, "table_properties": {"data_size": 67112778, "index_size": 807042, "filter_size": 297405, "raw_key_size": 8934108, "raw_average_key_size": 154, "raw_value_size": 59883964, "raw_average_value_size": 1036, "num_data_blocks": 15345, "num_entries": 57802, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -523> 2020-05-08 03:58:54.857 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #659: 58165 keys, 68229339 bytes -522> 2020-05-08 03:58:54.857 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334858452, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 659, "file_size": 68229339, "table_properties": {"data_size": 67114589, "index_size": 815710, "filter_size": 298291, "raw_key_size": 8991773, "raw_average_key_size": 154, "raw_value_size": 59881189, "raw_average_value_size": 1029, "num_data_blocks": 15370, "num_entries": 58165, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -521> 2020-05-08 03:58:54.857 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 194] Generated table #660: 31 keys, 34574 bytes -520> 2020-05-08 03:58:54.857 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334858852, "cf_name": "default", "job": 194, "event": "table_file_creation", "file_number": 660, "file_size": 34574, "table_properties": {"data_size": 33238, "index_size": 434, "filter_size": 165, "raw_key_size": 4795, "raw_average_key_size": 154, "raw_value_size": 29529, "raw_average_value_size": 952, "num_data_blocks": 9, "num_entries": 31, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -519> 2020-05-08 03:58:54.860 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 194] Compacted 1@1 + 6@2 files to L2 => 409373514 bytes -518> 2020-05-08 03:58:54.860 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-03:58:54.861938) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 2 15 0 0 0 0] max score 0.80, MB/sec: 464.4 rd, 398.0 wr, level 2, files in(1, 6) out(7) MB in(65.4, 390.1) out(390.4), read-write-amplify(12.9) write-amplify(6.0) OK, records in: 502421, records dropped: 1544 -517> 2020-05-08 03:58:54.860 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-03:58:54.861961) EVENT_LOG_v1 {"time_micros": 1588910334861952, "job": 194, "event": "compaction_finished", "compaction_time_micros": 1028594, "output_level": 2, "num_output_files": 7, "total_output_size": 409373514, "num_input_records": 502421, "num_output_records": 347973, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 2, 15, 0, 0, 0, 0]} -516> 2020-05-08 03:58:54.860 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862092, "job": 194, "event": "table_file_deletion", "file_number": 653} -515> 2020-05-08 03:58:54.860 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862119, "job": 194, "event": "table_file_deletion", "file_number": 647} -514> 2020-05-08 03:58:54.861 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862181, "job": 194, "event": "table_file_deletion", "file_number": 633} -513> 2020-05-08 03:58:54.861 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862215, "job": 194, "event": "table_file_deletion", "file_number": 632} -512> 2020-05-08 03:58:54.861 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862236, "job": 194, "event": "table_file_deletion", "file_number": 613} -511> 2020-05-08 03:58:54.861 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862261, "job": 194, "event": "table_file_deletion", "file_number": 612} -510> 2020-05-08 03:58:54.861 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588910334862282, "job": 194, "event": "table_file_deletion", "file_number": 611} -509> 2020-05-08 04:10:36.052 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -508> 2020-05-08 04:10:36.052 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -507> 2020-05-08 04:10:36.052 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -506> 2020-05-08 04:10:36.467 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -505> 2020-05-08 04:40:36.133 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -504> 2020-05-08 04:40:36.133 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -503> 2020-05-08 04:40:36.133 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -502> 2020-05-08 04:40:36.630 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -501> 2020-05-08 05:10:36.214 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -500> 2020-05-08 05:10:36.214 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -499> 2020-05-08 05:10:36.214 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -498> 2020-05-08 05:10:36.708 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -497> 2020-05-08 05:40:36.294 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -496> 2020-05-08 05:40:36.294 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -495> 2020-05-08 05:40:36.294 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -494> 2020-05-08 05:40:36.374 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -493> 2020-05-08 05:55:36.375 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -492> 2020-05-08 05:55:36.375 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -491> 2020-05-08 05:55:36.375 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -490> 2020-05-08 05:55:36.849 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -489> 2020-05-08 06:25:36.455 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -488> 2020-05-08 06:25:36.455 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -487> 2020-05-08 06:25:36.455 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -486> 2020-05-08 06:25:36.495 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -485> 2020-05-08 06:40:36.496 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -484> 2020-05-08 06:40:36.496 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -483> 2020-05-08 06:40:36.496 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -482> 2020-05-08 06:40:36.903 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -481> 2020-05-08 07:10:36.579 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -480> 2020-05-08 07:10:36.579 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -479> 2020-05-08 07:10:36.579 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -478> 2020-05-08 07:10:36.994 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -477> 2020-05-08 07:40:36.659 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -476> 2020-05-08 07:40:36.659 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -475> 2020-05-08 07:40:36.659 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -474> 2020-05-08 07:40:37.135 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -473> 2020-05-08 08:10:36.740 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -472> 2020-05-08 08:10:36.740 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -471> 2020-05-08 08:10:36.740 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -470> 2020-05-08 08:10:37.230 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -469> 2020-05-08 08:40:36.820 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -468> 2020-05-08 08:40:36.820 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -467> 2020-05-08 08:40:36.820 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -466> 2020-05-08 08:40:36.900 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -465> 2020-05-08 08:45:18.796 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 642 from recycle list -464> 2020-05-08 08:45:18.796 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #661. Immutable memtables: 0. -463> 2020-05-08 08:45:18.797 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-08:45:18.797147) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -462> 2020-05-08 08:45:18.797 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 195] Flushing memtable with next log file: 661 -461> 2020-05-08 08:45:18.797 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588927518797247, "job": 195, "event": "flush_started", "num_memtables": 1, "num_entries": 364157, "num_deletes": 73317, "memory_usage": 260049488, "flush_reason": "Write Buffer Full"} -460> 2020-05-08 08:45:18.797 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 195] Level-0 flush table #662: started -459> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588927518968191, "cf_name": "default", "job": 195, "event": "table_file_creation", "file_number": 662, "file_size": 34270243, "table_properties": {"data_size": 33393877, "index_size": 498650, "filter_size": 376960, "raw_key_size": 11863668, "raw_average_key_size": 98, "raw_value_size": 27427855, "raw_average_value_size": 227, "num_data_blocks": 7839, "num_entries": 120732, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "69900", "kMergeOperands": "1051"}} -458> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 195] Level-0 flush table #662: 34270243 bytes OK -457> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 644 to recycle list -456> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-08:45:18.968230) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #662 started -455> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-08:45:18.968402) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #662: memtable #1 done -454> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-08:45:18.968418) EVENT_LOG_v1 {"time_micros": 1588927518968412, "job": 195, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [1, 2, 15, 0, 0, 0, 0], "immutable_memtables": 0} -453> 2020-05-08 08:45:18.968 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-08:45:18.968441) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[1 2 15 0 0 0 0] max score 0.80 -452> 2020-05-08 08:55:36.900 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -451> 2020-05-08 08:55:36.900 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -450> 2020-05-08 08:55:36.900 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -449> 2020-05-08 08:55:37.369 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -448> 2020-05-08 09:25:36.981 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -447> 2020-05-08 09:25:36.981 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -446> 2020-05-08 09:25:36.981 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -445> 2020-05-08 09:25:36.981 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -444> 2020-05-08 09:40:36.981 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -443> 2020-05-08 09:40:36.981 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -442> 2020-05-08 09:40:36.981 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -441> 2020-05-08 09:40:37.061 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -440> 2020-05-08 09:55:37.062 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -439> 2020-05-08 09:55:37.062 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -438> 2020-05-08 09:55:37.062 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -437> 2020-05-08 09:55:37.555 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -436> 2020-05-08 10:25:37.142 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -435> 2020-05-08 10:25:37.142 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -434> 2020-05-08 10:25:37.142 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -433> 2020-05-08 10:25:37.562 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -432> 2020-05-08 10:55:37.222 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -431> 2020-05-08 10:55:37.222 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -430> 2020-05-08 10:55:37.222 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -429> 2020-05-08 10:55:37.703 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -428> 2020-05-08 11:25:37.303 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -427> 2020-05-08 11:25:37.303 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -426> 2020-05-08 11:25:37.303 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -425> 2020-05-08 11:25:37.303 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -424> 2020-05-08 11:40:37.303 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -423> 2020-05-08 11:40:37.303 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -422> 2020-05-08 11:40:37.303 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -421> 2020-05-08 11:40:37.383 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -420> 2020-05-08 11:55:37.383 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -419> 2020-05-08 11:55:37.383 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -418> 2020-05-08 11:55:37.383 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -417> 2020-05-08 11:55:37.463 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -416> 2020-05-08 12:10:37.463 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -415> 2020-05-08 12:10:37.463 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -414> 2020-05-08 12:10:37.463 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -413> 2020-05-08 12:10:37.463 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -412> 2020-05-08 12:25:37.464 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -411> 2020-05-08 12:25:37.464 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -410> 2020-05-08 12:25:37.464 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -409> 2020-05-08 12:25:37.544 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -408> 2020-05-08 12:40:37.544 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -407> 2020-05-08 12:40:37.544 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -406> 2020-05-08 12:40:37.544 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -405> 2020-05-08 12:40:38.031 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -404> 2020-05-08 13:10:37.624 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -403> 2020-05-08 13:10:37.624 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -402> 2020-05-08 13:10:37.624 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -401> 2020-05-08 13:10:38.123 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -400> 2020-05-08 13:27:51.151 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 644 from recycle list -399> 2020-05-08 13:27:51.151 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #663. Immutable memtables: 0. -398> 2020-05-08 13:27:51.152 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-13:27:51.152031) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -397> 2020-05-08 13:27:51.152 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 196] Flushing memtable with next log file: 663 -396> 2020-05-08 13:27:51.152 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588944471152166, "job": 196, "event": "flush_started", "num_memtables": 1, "num_entries": 358803, "num_deletes": 72804, "memory_usage": 260049776, "flush_reason": "Write Buffer Full"} -395> 2020-05-08 13:27:51.152 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 196] Level-0 flush table #664: started -394> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588944471321119, "cf_name": "default", "job": 196, "event": "table_file_creation", "file_number": 664, "file_size": 34137472, "table_properties": {"data_size": 33276375, "index_size": 486256, "filter_size": 374085, "raw_key_size": 11769082, "raw_average_key_size": 98, "raw_value_size": 27365143, "raw_average_value_size": 228, "num_data_blocks": 7767, "num_entries": 119677, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "69455", "kMergeOperands": "995"}} -393> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 196] Level-0 flush table #664: 34137472 bytes OK -392> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 661 to recycle list -391> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-13:27:51.321165) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #664 started -390> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-13:27:51.321333) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #664: memtable #1 done -389> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-13:27:51.321350) EVENT_LOG_v1 {"time_micros": 1588944471321344, "job": 196, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 2, 15, 0, 0, 0, 0], "immutable_memtables": 0} -388> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-13:27:51.321372) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[2 2 15 0 0 0 0] max score 0.80 -387> 2020-05-08 13:27:51.321 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:347] [JOB 196] Try to delete WAL files size 251925222, prev total WAL file size 251929650, number of live WAL files 2. -386> 2020-05-08 13:40:37.705 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -385> 2020-05-08 13:40:37.705 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -384> 2020-05-08 13:40:37.705 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -383> 2020-05-08 13:40:38.127 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -382> 2020-05-08 14:10:37.785 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -381> 2020-05-08 14:10:37.785 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -380> 2020-05-08 14:10:37.785 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -379> 2020-05-08 14:10:37.865 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -378> 2020-05-08 14:25:37.865 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -377> 2020-05-08 14:25:37.865 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -376> 2020-05-08 14:25:37.865 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -375> 2020-05-08 14:25:38.318 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -374> 2020-05-08 14:55:37.946 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -373> 2020-05-08 14:55:37.946 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -372> 2020-05-08 14:55:37.946 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -371> 2020-05-08 14:55:37.986 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -370> 2020-05-08 15:10:37.986 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -369> 2020-05-08 15:10:37.986 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -368> 2020-05-08 15:10:37.986 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -367> 2020-05-08 15:10:38.461 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -366> 2020-05-08 15:40:38.066 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -365> 2020-05-08 15:40:38.066 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -364> 2020-05-08 15:40:38.066 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -363> 2020-05-08 15:40:38.557 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -362> 2020-05-08 16:10:38.147 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -361> 2020-05-08 16:10:38.147 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -360> 2020-05-08 16:10:38.147 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -359> 2020-05-08 16:10:38.227 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -358> 2020-05-08 16:25:38.227 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -357> 2020-05-08 16:25:38.227 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -356> 2020-05-08 16:25:38.227 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -355> 2020-05-08 16:25:38.701 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -354> 2020-05-08 16:55:38.307 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -353> 2020-05-08 16:55:38.307 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -352> 2020-05-08 16:55:38.307 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -351> 2020-05-08 16:55:38.347 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -350> 2020-05-08 17:10:38.348 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -349> 2020-05-08 17:10:38.348 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -348> 2020-05-08 17:10:38.348 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -347> 2020-05-08 17:10:38.348 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -346> 2020-05-08 17:25:38.348 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -345> 2020-05-08 17:25:38.348 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -344> 2020-05-08 17:25:38.348 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -343> 2020-05-08 17:25:38.388 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -342> 2020-05-08 17:40:38.388 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -341> 2020-05-08 17:40:38.388 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -340> 2020-05-08 17:40:38.388 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -339> 2020-05-08 17:40:38.428 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -338> 2020-05-08 17:55:38.428 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -337> 2020-05-08 17:55:38.428 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -336> 2020-05-08 17:55:38.428 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -335> 2020-05-08 17:55:38.508 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -334> 2020-05-08 18:10:38.509 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -333> 2020-05-08 18:10:38.509 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -332> 2020-05-08 18:10:38.509 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -331> 2020-05-08 18:10:38.509 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -330> 2020-05-08 18:12:58.867 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 661 from recycle list -329> 2020-05-08 18:12:58.867 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #665. Immutable memtables: 0. -328> 2020-05-08 18:12:58.867 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-18:12:58.868366) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -327> 2020-05-08 18:12:58.867 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 197] Flushing memtable with next log file: 665 -326> 2020-05-08 18:12:58.867 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588961578868567, "job": 197, "event": "flush_started", "num_memtables": 1, "num_entries": 354067, "num_deletes": 71724, "memory_usage": 260049600, "flush_reason": "Write Buffer Full"} -325> 2020-05-08 18:12:58.867 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 197] Level-0 flush table #666: started -324> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588961579037158, "cf_name": "default", "job": 197, "event": "table_file_creation", "file_number": 666, "file_size": 33705522, "table_properties": {"data_size": 32848423, "index_size": 486611, "filter_size": 369732, "raw_key_size": 11610494, "raw_average_key_size": 98, "raw_value_size": 27010588, "raw_average_value_size": 228, "num_data_blocks": 7714, "num_entries": 118320, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "68441", "kMergeOperands": "955"}} -323> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 197] Level-0 flush table #666: 33705522 bytes OK -322> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 663 to recycle list -321> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-18:12:59.037199) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #666 started -320> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-18:12:59.037372) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #666: memtable #1 done -319> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-18:12:59.037388) EVENT_LOG_v1 {"time_micros": 1588961579037382, "job": 197, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [3, 2, 15, 0, 0, 0, 0], "immutable_memtables": 0} -318> 2020-05-08 18:12:59.036 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-18:12:59.037412) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[3 2 15 0 0 0 0] max score 0.80 -317> 2020-05-08 18:25:38.509 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -316> 2020-05-08 18:25:38.509 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -315> 2020-05-08 18:25:38.509 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -314> 2020-05-08 18:25:38.589 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -313> 2020-05-08 18:40:38.589 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -312> 2020-05-08 18:40:38.589 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -311> 2020-05-08 18:40:38.589 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -310> 2020-05-08 18:40:38.589 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -309> 2020-05-08 18:55:38.589 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -308> 2020-05-08 18:55:38.589 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -307> 2020-05-08 18:55:38.589 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -306> 2020-05-08 18:55:38.669 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -305> 2020-05-08 19:10:38.670 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -304> 2020-05-08 19:10:38.670 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -303> 2020-05-08 19:10:38.670 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -302> 2020-05-08 19:10:38.750 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -301> 2020-05-08 19:25:38.750 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -300> 2020-05-08 19:25:38.750 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -299> 2020-05-08 19:25:38.750 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -298> 2020-05-08 19:25:38.750 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -297> 2020-05-08 19:40:38.750 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -296> 2020-05-08 19:40:38.750 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -295> 2020-05-08 19:40:38.750 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -294> 2020-05-08 19:40:38.830 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -293> 2020-05-08 19:55:38.830 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -292> 2020-05-08 19:55:38.830 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -291> 2020-05-08 19:55:38.830 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -290> 2020-05-08 19:55:39.342 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -289> 2020-05-08 20:25:38.911 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -288> 2020-05-08 20:25:38.911 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -287> 2020-05-08 20:25:38.911 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -286> 2020-05-08 20:25:39.323 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -285> 2020-05-08 20:55:38.991 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -284> 2020-05-08 20:55:38.991 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -283> 2020-05-08 20:55:38.991 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -282> 2020-05-08 20:55:39.464 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -281> 2020-05-08 21:25:39.072 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -280> 2020-05-08 21:25:39.072 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -279> 2020-05-08 21:25:39.072 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -278> 2020-05-08 21:25:39.559 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -277> 2020-05-08 21:55:39.152 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -276> 2020-05-08 21:55:39.152 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -275> 2020-05-08 21:55:39.152 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -274> 2020-05-08 21:55:39.152 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -273> 2020-05-08 22:10:39.152 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -272> 2020-05-08 22:10:39.152 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -271> 2020-05-08 22:10:39.152 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -270> 2020-05-08 22:10:39.192 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -269> 2020-05-08 22:25:39.192 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -268> 2020-05-08 22:25:39.192 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -267> 2020-05-08 22:25:39.192 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -266> 2020-05-08 22:25:39.272 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -265> 2020-05-08 22:40:39.273 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -264> 2020-05-08 22:40:39.273 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -263> 2020-05-08 22:40:39.273 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -262> 2020-05-08 22:40:39.353 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -261> 2020-05-08 22:54:30.090 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 663 from recycle list -260> 2020-05-08 22:54:30.090 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #667. Immutable memtables: 0. -259> 2020-05-08 22:54:30.091 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-22:54:30.092060) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -258> 2020-05-08 22:54:30.091 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 198] Flushing memtable with next log file: 667 -257> 2020-05-08 22:54:30.091 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978470092320, "job": 198, "event": "flush_started", "num_memtables": 1, "num_entries": 356661, "num_deletes": 72099, "memory_usage": 260083656, "flush_reason": "Write Buffer Full"} -256> 2020-05-08 22:54:30.091 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 198] Level-0 flush table #668: started -255> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978470257493, "cf_name": "default", "job": 198, "event": "table_file_creation", "file_number": 668, "file_size": 33747520, "table_properties": {"data_size": 32892994, "index_size": 483255, "filter_size": 370515, "raw_key_size": 11646844, "raw_average_key_size": 98, "raw_value_size": 27020198, "raw_average_value_size": 227, "num_data_blocks": 7714, "num_entries": 118602, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "68859", "kMergeOperands": "953"}} -254> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 198] Level-0 flush table #668: 33747520 bytes OK -253> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 665 to recycle list -252> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-22:54:30.257532) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #668 started -251> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-22:54:30.257697) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #668: memtable #1 done -250> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-22:54:30.257713) EVENT_LOG_v1 {"time_micros": 1588978470257707, "job": 198, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [4, 2, 15, 0, 0, 0, 0], "immutable_memtables": 0} -249> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/08-22:54:30.257740) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[4 2 15 0 0 0 0] max score 1.00 -248> 2020-05-08 22:54:30.256 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:347] [JOB 198] Try to delete WAL files size 251997796, prev total WAL file size 251998816, number of live WAL files 2. -247> 2020-05-08 22:54:30.256 7f51b174a700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl.cc:625] ------- DUMPING STATS ------- -246> 2020-05-08 22:54:30.256 7f51b174a700 3 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl.cc:626] ** DB Stats ** Uptime(secs): 2768743.1 total, 68138.3 interval Cumulative writes: 14M writes, 41M keys, 14M commit groups, 1.0 writes per commit group, ingest: 28.68 GB, 0.01 MB/s Cumulative WAL: 14M writes, 7096K syncs, 2.00 writes per sync, written: 28.68 GB, 0.01 MB/s Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent Interval writes: 485K writes, 1433K keys, 485K commit groups, 1.0 writes per commit group, ingest: 961.07 MB, 0.01 MB/s Interval WAL: 485K writes, 242K syncs, 2.00 writes per sync, written: 0.94 MB, 0.01 MB/s Interval stall: 00:00:0.000 H:M:S, 0.0 percent ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 4/0 129.57 MB 1.0 0.0 0.0 0.0 3.9 3.9 0.0 1.0 0.0 201.9 20 123 0.159 0 0 L1 2/0 116.27 MB 0.8 8.1 3.8 4.4 7.3 3.0 0.0 2.0 272.8 247.0 30 30 1.015 26M 3126K L2 15/0 830.25 MB 0.3 15.7 2.9 12.8 13.5 0.6 0.2 4.7 420.0 359.5 38 41 0.935 24M 8565K Sum 21/0 1.05 GB 0.0 23.8 6.6 17.2 24.7 7.5 0.2 6.4 276.1 285.8 88 194 0.456 51M 11M Int 0/0 0.00 KB 0.0 1.0 0.3 0.8 1.0 0.2 0.0 7.9 293.6 290.3 4 7 0.506 2063K 371K Uptime(secs): 2768743.1 total, 68138.3 interval Flush(GB): cumulative 3.865, interval 0.127 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 24.67 GB write, 0.01 MB/s write, 23.83 GB read, 0.01 MB/s read, 88.4 seconds Interval compaction: 1.00 GB write, 0.02 MB/s write, 1.01 GB read, 0.02 MB/s read, 3.5 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count ** File Read Latency Histogram By Level [default] ** ** Compaction Stats [default] ** Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop ---------------------------------------------------------------------------------------------------------------------------------------------------------- L0 4/0 129.57 MB 1.0 0.0 0.0 0.0 3.9 3.9 0.0 1.0 0.0 201.9 20 123 0.159 0 0 L1 2/0 116.27 MB 0.8 8.1 3.8 4.4 7.3 3.0 0.0 2.0 272.8 247.0 30 30 1.015 26M 3126K L2 15/0 830.25 MB 0.3 15.7 2.9 12.8 13.5 0.6 0.2 4.7 420.0 359.5 38 41 0.935 24M 8565K Sum 21/0 1.05 GB 0.0 23.8 6.6 17.2 24.7 7.5 0.2 6.4 276.1 285.8 88 194 0.456 51M 11M Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 0.000 0 0 Uptime(secs): 2768743.1 total, 0.0 interval Flush(GB): cumulative 3.865, interval 0.000 AddFile(GB): cumulative 0.000, interval 0.000 AddFile(Total Files): cumulative 0, interval 0 AddFile(L0 Files): cumulative 0, interval 0 AddFile(Keys): cumulative 0, interval 0 Cumulative compaction: 24.67 GB write, 0.01 MB/s write, 23.83 GB read, 0.01 MB/s read, 88.4 seconds Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count ** File Read Latency Histogram By Level [default] ** -245> 2020-05-08 22:54:30.256 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 199] Compacting 4@0 + 2@1 files to L1, score 1.00 -244> 2020-05-08 22:54:30.256 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 198 Base level 0, inputs: [668(32MB) 666(32MB) 664(32MB) 662(32MB)], [648(65MB) 649(50MB)] -243> 2020-05-08 22:54:30.256 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978470258052, "job": 199, "event": "compaction_started", "compaction_reason": "LevelL0FilesNum", "files_L0": [668, 666, 664, 662], "files_L1": [648, 649], "score": 1, "input_data_size": 257781911} -242> 2020-05-08 22:54:30.504 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 199] Generated table #669: 163733 keys, 68650300 bytes -241> 2020-05-08 22:54:30.504 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978470506007, "cf_name": "default", "job": 199, "event": "table_file_creation", "file_number": 669, "file_size": 68650300, "table_properties": {"data_size": 67111105, "index_size": 977307, "filter_size": 561137, "raw_key_size": 24541489, "raw_average_key_size": 149, "raw_value_size": 53364458, "raw_average_value_size": 325, "num_data_blocks": 15366, "num_entries": 163733, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "112029", "kMergeOperands": "0"}} -240> 2020-05-08 22:54:30.775 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 199] Generated table #670: 226653 keys, 68826103 bytes -239> 2020-05-08 22:54:30.775 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978470776589, "cf_name": "default", "job": 199, "event": "table_file_creation", "file_number": 670, "file_size": 68826103, "table_properties": {"data_size": 67112201, "index_size": 995225, "filter_size": 717926, "raw_key_size": 24666331, "raw_average_key_size": 108, "raw_value_size": 54254115, "raw_average_value_size": 239, "num_data_blocks": 15650, "num_entries": 226653, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "145335", "kMergeOperands": "0"}} -238> 2020-05-08 22:54:31.153 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 199] Generated table #671: 496634 keys, 69567653 bytes -237> 2020-05-08 22:54:31.154 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471155117, "cf_name": "default", "job": 199, "event": "table_file_creation", "file_number": 671, "file_size": 69567653, "table_properties": {"data_size": 67111659, "index_size": 1065099, "filter_size": 1390143, "raw_key_size": 24826375, "raw_average_key_size": 49, "raw_value_size": 59222163, "raw_average_value_size": 119, "num_data_blocks": 16958, "num_entries": 496634, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "273280", "kMergeOperands": "0"}} -236> 2020-05-08 22:54:31.350 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 199] Generated table #672: 280744 keys, 27783542 bytes -235> 2020-05-08 22:54:31.350 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471352010, "cf_name": "default", "job": 199, "event": "table_file_creation", "file_number": 672, "file_size": 27783542, "table_properties": {"data_size": 26609204, "index_size": 412732, "filter_size": 760853, "raw_key_size": 13084973, "raw_average_key_size": 46, "raw_value_size": 22182904, "raw_average_value_size": 79, "num_data_blocks": 6711, "num_entries": 280744, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "169232", "kMergeOperands": "21698"}} -234> 2020-05-08 22:54:31.354 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 199] Compacted 4@0 + 2@1 files to L1 => 234827598 bytes -233> 2020-05-08 22:54:31.354 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-22:54:31.355953) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 4 15 0 0 0 0] max score 1.93, MB/sec: 235.6 rd, 214.6 wr, level 1, files in(4, 2) out(4) MB in(129.6, 116.3) out(223.9), read-write-amplify(3.6) write-amplify(1.7) OK, records in: 1252774, records dropped: 810 -232> 2020-05-08 22:54:31.354 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-22:54:31.355978) EVENT_LOG_v1 {"time_micros": 1588978471355968, "job": 199, "event": "compaction_finished", "compaction_time_micros": 1094075, "output_level": 1, "num_output_files": 4, "total_output_size": 234827598, "num_input_records": 1248785, "num_output_records": 1167764, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 4, 15, 0, 0, 0, 0]} -231> 2020-05-08 22:54:31.354 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356087, "job": 199, "event": "table_file_deletion", "file_number": 668} -230> 2020-05-08 22:54:31.354 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356113, "job": 199, "event": "table_file_deletion", "file_number": 666} -229> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356146, "job": 199, "event": "table_file_deletion", "file_number": 664} -228> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356191, "job": 199, "event": "table_file_deletion", "file_number": 662} -227> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356216, "job": 199, "event": "table_file_deletion", "file_number": 649} -226> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356240, "job": 199, "event": "table_file_deletion", "file_number": 648} -225> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 200] Compacting 1@1 + 6@2 files to L2, score 1.93 -224> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 199 Base level 1, inputs: [670(65MB)], [658(65MB) 659(65MB) 660(33KB) 634(65MB) 635(65MB) 636(65MB)] -223> 2020-05-08 22:54:31.355 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471356294, "job": 200, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L1": [670], "files_L2": [658, 659, 660, 634, 635, 636], "score": 1.92915, "input_data_size": 410530010} -222> 2020-05-08 22:54:31.515 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 200] Generated table #673: 57777 keys, 68217427 bytes -221> 2020-05-08 22:54:31.515 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471516605, "cf_name": "default", "job": 200, "event": "table_file_creation", "file_number": 673, "file_size": 68217427, "table_properties": {"data_size": 67113198, "index_size": 806169, "filter_size": 297311, "raw_key_size": 8930037, "raw_average_key_size": 154, "raw_value_size": 59884979, "raw_average_value_size": 1036, "num_data_blocks": 15340, "num_entries": 57777, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -220> 2020-05-08 22:54:31.695 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 200] Generated table #674: 58143 keys, 68221983 bytes -219> 2020-05-08 22:54:31.695 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471696649, "cf_name": "default", "job": 200, "event": "table_file_creation", "file_number": 674, "file_size": 68221983, "table_properties": {"data_size": 67110524, "index_size": 812467, "filter_size": 298243, "raw_key_size": 8988381, "raw_average_key_size": 154, "raw_value_size": 59885011, "raw_average_value_size": 1029, "num_data_blocks": 15346, "num_entries": 58143, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -218> 2020-05-08 22:54:31.876 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 200] Generated table #675: 58128 keys, 68231983 bytes -217> 2020-05-08 22:54:31.876 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978471877832, "cf_name": "default", "job": 200, "event": "table_file_creation", "file_number": 675, "file_size": 68231983, "table_properties": {"data_size": 67113647, "index_size": 819383, "filter_size": 298204, "raw_key_size": 8985064, "raw_average_key_size": 154, "raw_value_size": 59867250, "raw_average_value_size": 1029, "num_data_blocks": 15366, "num_entries": 58128, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -216> 2020-05-08 22:54:32.053 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 200] Generated table #676: 57970 keys, 68220481 bytes -215> 2020-05-08 22:54:32.053 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472054519, "cf_name": "default", "job": 200, "event": "table_file_creation", "file_number": 676, "file_size": 68220481, "table_properties": {"data_size": 67109154, "index_size": 812848, "filter_size": 297730, "raw_key_size": 8960351, "raw_average_key_size": 154, "raw_value_size": 59876218, "raw_average_value_size": 1032, "num_data_blocks": 15341, "num_entries": 57970, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -214> 2020-05-08 22:54:32.289 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 200] Generated table #677: 198658 keys, 68770390 bytes -213> 2020-05-08 22:54:32.289 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472290936, "cf_name": "default", "job": 200, "event": "table_file_creation", "file_number": 677, "file_size": 68770390, "table_properties": {"data_size": 67109742, "index_size": 1014089, "filter_size": 645809, "raw_key_size": 11280104, "raw_average_key_size": 56, "raw_value_size": 62342259, "raw_average_value_size": 313, "num_data_blocks": 16618, "num_entries": 198658, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -212> 2020-05-08 22:54:32.291 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 200] Generated table #678: 946 keys, 275071 bytes -211> 2020-05-08 22:54:32.291 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472292610, "cf_name": "default", "job": 200, "event": "table_file_creation", "file_number": 678, "file_size": 275071, "table_properties": {"data_size": 267196, "index_size": 4178, "filter_size": 2958, "raw_key_size": 47300, "raw_average_key_size": 50, "raw_value_size": 251121, "raw_average_value_size": 265, "num_data_blocks": 68, "num_entries": 946, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -210> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 200] Compacted 1@1 + 6@2 files to L2 => 341937335 bytes -209> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-22:54:32.295445) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 3 15 0 0 0 0] max score 1.38, MB/sec: 438.4 rd, 365.2 wr, level 2, files in(1, 6) out(6) MB in(65.6, 325.9) out(326.1), read-write-amplify(10.9) write-amplify(5.0) OK, records in: 658603, records dropped: 2269 -208> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-22:54:32.295470) EVENT_LOG_v1 {"time_micros": 1588978472295460, "job": 200, "event": "compaction_finished", "compaction_time_micros": 936337, "output_level": 2, "num_output_files": 6, "total_output_size": 341937335, "num_input_records": 658603, "num_output_records": 431622, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 3, 15, 0, 0, 0, 0]} -207> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295583, "job": 200, "event": "table_file_deletion", "file_number": 670} -206> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295608, "job": 200, "event": "table_file_deletion", "file_number": 660} -205> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295646, "job": 200, "event": "table_file_deletion", "file_number": 659} -204> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295669, "job": 200, "event": "table_file_deletion", "file_number": 658} -203> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295699, "job": 200, "event": "table_file_deletion", "file_number": 636} -202> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295728, "job": 200, "event": "table_file_deletion", "file_number": 635} -201> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295754, "job": 200, "event": "table_file_deletion", "file_number": 634} -200> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1636] [default] [JOB 201] Compacting 1@1 + 8@2 files to L2, score 1.38 -199> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1640] [default] Compaction start summary: Base version 200 Base level 1, inputs: [669(65MB)], [650(65MB) 651(65MB) 652(65MB) 654(65MB) 655(65MB) 656(65MB) 657(65MB) 673(65MB)] -198> 2020-05-08 22:54:32.294 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472295805, "job": 201, "event": "compaction_started", "compaction_reason": "LevelMaxLevelSize", "files_L1": [669], "files_L2": [650, 651, 652, 654, 655, 656, 657, 673], "score": 1.38181, "input_data_size": 614398781} -197> 2020-05-08 22:54:32.465 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #679: 58978 keys, 68223532 bytes -196> 2020-05-08 22:54:32.465 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472467089, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 679, "file_size": 68223532, "table_properties": {"data_size": 67114344, "index_size": 808137, "filter_size": 300302, "raw_key_size": 8981974, "raw_average_key_size": 152, "raw_value_size": 59859948, "raw_average_value_size": 1014, "num_data_blocks": 15346, "num_entries": 58978, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -195> 2020-05-08 22:54:32.637 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #680: 58061 keys, 68229064 bytes -194> 2020-05-08 22:54:32.637 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472638851, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 680, "file_size": 68229064, "table_properties": {"data_size": 67112384, "index_size": 817991, "filter_size": 297940, "raw_key_size": 8974533, "raw_average_key_size": 154, "raw_value_size": 59867045, "raw_average_value_size": 1031, "num_data_blocks": 15353, "num_entries": 58061, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -193> 2020-05-08 22:54:32.800 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #681: 57688 keys, 68210727 bytes -192> 2020-05-08 22:54:32.800 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472801424, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 681, "file_size": 68210727, "table_properties": {"data_size": 67112997, "index_size": 799913, "filter_size": 297068, "raw_key_size": 8914949, "raw_average_key_size": 154, "raw_value_size": 59884434, "raw_average_value_size": 1038, "num_data_blocks": 15338, "num_entries": 57688, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -191> 2020-05-08 22:54:32.969 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #682: 58033 keys, 68229853 bytes -190> 2020-05-08 22:54:32.969 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978472970494, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 682, "file_size": 68229853, "table_properties": {"data_size": 67112839, "index_size": 818308, "filter_size": 297957, "raw_key_size": 8970669, "raw_average_key_size": 154, "raw_value_size": 59874382, "raw_average_value_size": 1031, "num_data_blocks": 15365, "num_entries": 58033, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -189> 2020-05-08 22:54:33.138 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #683: 58012 keys, 68221531 bytes -188> 2020-05-08 22:54:33.138 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473139521, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 683, "file_size": 68221531, "table_properties": {"data_size": 67111409, "index_size": 811545, "filter_size": 297828, "raw_key_size": 8966888, "raw_average_key_size": 154, "raw_value_size": 59881037, "raw_average_value_size": 1032, "num_data_blocks": 15338, "num_entries": 58012, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -187> 2020-05-08 22:54:33.309 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #684: 58154 keys, 68220535 bytes -186> 2020-05-08 22:54:33.309 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473310187, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 684, "file_size": 68220535, "table_properties": {"data_size": 67113230, "index_size": 808341, "filter_size": 298215, "raw_key_size": 8988996, "raw_average_key_size": 154, "raw_value_size": 59873694, "raw_average_value_size": 1029, "num_data_blocks": 15356, "num_entries": 58154, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -185> 2020-05-08 22:54:33.479 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #685: 57916 keys, 68216815 bytes -184> 2020-05-08 22:54:33.479 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473480276, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 685, "file_size": 68216815, "table_properties": {"data_size": 67111728, "index_size": 806641, "filter_size": 297697, "raw_key_size": 8951670, "raw_average_key_size": 154, "raw_value_size": 59885477, "raw_average_value_size": 1034, "num_data_blocks": 15366, "num_entries": 57916, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -183> 2020-05-08 22:54:33.645 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #686: 57807 keys, 68219683 bytes -182> 2020-05-08 22:54:33.645 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473646459, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 686, "file_size": 68219683, "table_properties": {"data_size": 67111343, "index_size": 810218, "filter_size": 297373, "raw_key_size": 8934829, "raw_average_key_size": 154, "raw_value_size": 59883433, "raw_average_value_size": 1035, "num_data_blocks": 15352, "num_entries": 57807, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -181> 2020-05-08 22:54:33.652 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1324] [default] [JOB 201] Generated table #687: 971 keys, 1132917 bytes -180> 2020-05-08 22:54:33.652 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473654089, "cf_name": "default", "job": 201, "event": "table_file_creation", "file_number": 687, "file_size": 1132917, "table_properties": {"data_size": 1113858, "index_size": 13345, "filter_size": 4974, "raw_key_size": 150071, "raw_average_key_size": 154, "raw_value_size": 992870, "raw_average_value_size": 1022, "num_data_blocks": 255, "num_entries": 971, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "0", "kMergeOperands": "0"}} -179> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:1390] [default] [JOB 201] Compacted 1@1 + 8@2 files to L2 => 546904657 bytes -178> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-22:54:33.658615) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/compaction_job.cc:752] [default] compacted to: base level 1 max bytes base 268435456 files[0 2 16 0 0 0 0] max score 0.85, MB/sec: 452.3 rd, 402.6 wr, level 2, files in(1, 8) out(9) MB in(65.5, 520.5) out(521.6), read-write-amplify(16.9) write-amplify(8.0) OK, records in: 627997, records dropped: 1623 -177> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: (Original Log Time 2020/05/08-22:54:33.658641) EVENT_LOG_v1 {"time_micros": 1588978473658630, "job": 201, "event": "compaction_finished", "compaction_time_micros": 1358311, "output_level": 2, "num_output_files": 9, "total_output_size": 546904657, "num_input_records": 627986, "num_output_records": 465620, "num_subcompactions": 1, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 2, 16, 0, 0, 0, 0]} -176> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658799, "job": 201, "event": "table_file_deletion", "file_number": 673} -175> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658827, "job": 201, "event": "table_file_deletion", "file_number": 669} -174> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658853, "job": 201, "event": "table_file_deletion", "file_number": 657} -173> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658881, "job": 201, "event": "table_file_deletion", "file_number": 656} -172> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658916, "job": 201, "event": "table_file_deletion", "file_number": 655} -171> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658942, "job": 201, "event": "table_file_deletion", "file_number": 654} -170> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658967, "job": 201, "event": "table_file_deletion", "file_number": 652} -169> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473658995, "job": 201, "event": "table_file_deletion", "file_number": 651} -168> 2020-05-08 22:54:33.657 7f51b174a700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588978473659018, "job": 201, "event": "table_file_deletion", "file_number": 650} -167> 2020-05-08 22:55:39.353 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -166> 2020-05-08 22:55:39.353 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -165> 2020-05-08 22:55:39.353 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -164> 2020-05-08 22:55:39.841 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -163> 2020-05-08 23:25:39.434 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -162> 2020-05-08 23:25:39.434 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -161> 2020-05-08 23:25:39.434 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -160> 2020-05-08 23:25:39.856 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -159> 2020-05-08 23:55:39.514 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -158> 2020-05-08 23:55:39.514 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -157> 2020-05-08 23:55:39.514 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -156> 2020-05-08 23:55:39.594 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -155> 2020-05-09 00:10:39.595 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -154> 2020-05-09 00:10:39.595 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -153> 2020-05-09 00:10:39.595 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -152> 2020-05-09 00:10:39.595 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -151> 2020-05-09 00:25:39.595 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -150> 2020-05-09 00:25:39.595 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -149> 2020-05-09 00:25:39.595 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -148> 2020-05-09 00:25:39.675 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -147> 2020-05-09 00:40:39.675 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -146> 2020-05-09 00:40:39.675 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -145> 2020-05-09 00:40:39.675 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -144> 2020-05-09 00:40:40.140 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -143> 2020-05-09 01:10:39.757 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -142> 2020-05-09 01:10:39.757 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -141> 2020-05-09 01:10:39.757 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -140> 2020-05-09 01:10:40.235 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -139> 2020-05-09 01:40:39.839 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -138> 2020-05-09 01:40:39.839 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -137> 2020-05-09 01:40:39.839 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -136> 2020-05-09 01:40:40.328 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -135> 2020-05-09 02:10:39.920 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -134> 2020-05-09 02:10:39.920 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -133> 2020-05-09 02:10:39.920 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -132> 2020-05-09 02:10:40.419 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -131> 2020-05-09 02:40:40.002 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -130> 2020-05-09 02:40:40.002 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -129> 2020-05-09 02:40:40.002 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -128> 2020-05-09 02:40:40.428 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -127> 2020-05-09 03:10:40.083 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -126> 2020-05-09 03:10:40.083 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -125> 2020-05-09 03:10:40.083 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -124> 2020-05-09 03:10:40.163 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -123> 2020-05-09 03:25:40.164 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -122> 2020-05-09 03:25:40.164 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -121> 2020-05-09 03:25:40.164 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -120> 2020-05-09 03:25:40.566 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -119> 2020-05-09 03:36:11.274 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 665 from recycle list -118> 2020-05-09 03:36:11.274 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #688. Immutable memtables: 0. -117> 2020-05-09 03:36:11.275 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-03:36:11.276063) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -116> 2020-05-09 03:36:11.275 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 202] Flushing memtable with next log file: 688 -115> 2020-05-09 03:36:11.275 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588995371276207, "job": 202, "event": "flush_started", "num_memtables": 1, "num_entries": 358562, "num_deletes": 73733, "memory_usage": 260056456, "flush_reason": "Write Buffer Full"} -114> 2020-05-09 03:36:11.275 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 202] Level-0 flush table #689: started -113> 2020-05-09 03:36:11.442 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1588995371443902, "cf_name": "default", "job": 202, "event": "table_file_creation", "file_number": 689, "file_size": 33854608, "table_properties": {"data_size": 32987426, "index_size": 490701, "filter_size": 375725, "raw_key_size": 11946235, "raw_average_key_size": 99, "raw_value_size": 27042198, "raw_average_value_size": 224, "num_data_blocks": 7744, "num_entries": 120612, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "70267", "kMergeOperands": "961"}} -112> 2020-05-09 03:36:11.442 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 202] Level-0 flush table #689: 33854608 bytes OK -111> 2020-05-09 03:36:11.443 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 667 to recycle list -110> 2020-05-09 03:36:11.443 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-03:36:11.443940) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #689 started -109> 2020-05-09 03:36:11.443 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-03:36:11.444103) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #689: memtable #1 done -108> 2020-05-09 03:36:11.443 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-03:36:11.444120) EVENT_LOG_v1 {"time_micros": 1588995371444114, "job": 202, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [1, 2, 16, 0, 0, 0, 0], "immutable_memtables": 0} -107> 2020-05-09 03:36:11.443 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-03:36:11.444151) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[1 2 16 0 0 0 0] max score 0.85 -106> 2020-05-09 03:44:42.155 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -105> 2020-05-09 03:46:33.736 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -104> 2020-05-09 03:46:36.194 7f519bf1f700 0 log_channel(cluster) log [DBG] : 1.7b scrub starts -103> 2020-05-09 03:48:27.634 7f519e724700 0 log_channel(cluster) log [DBG] : 1.7b scrub ok -102> 2020-05-09 03:49:02.000 7f51b77e1700 -1 received signal: Hangup from killall -q -1 ceph-mon ceph-mgr ceph-mds ceph-osd ceph-fuse radosgw (PID: 1452643) UID: 0 -101> 2020-05-09 03:49:02.034 7f51b77e1700 -1 received signal: Hangup from pkill -1 -x ceph-mon|ceph-mgr|ceph-mds|ceph-osd|ceph-fuse|radosgw (PID: 1452644) UID: 0 -100> 2020-05-09 03:55:40.244 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -99> 2020-05-09 03:55:40.244 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -98> 2020-05-09 03:55:40.244 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -97> 2020-05-09 03:55:40.707 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -96> 2020-05-09 04:25:40.325 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -95> 2020-05-09 04:25:40.325 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -94> 2020-05-09 04:25:40.325 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -93> 2020-05-09 04:25:40.803 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -92> 2020-05-09 04:55:40.406 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -91> 2020-05-09 04:55:40.406 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -90> 2020-05-09 04:55:40.406 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -89> 2020-05-09 04:55:40.896 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -88> 2020-05-09 05:25:40.487 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -87> 2020-05-09 05:25:40.487 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -86> 2020-05-09 05:25:40.487 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -85> 2020-05-09 05:25:40.990 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -84> 2020-05-09 05:55:40.567 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -83> 2020-05-09 05:55:40.567 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -82> 2020-05-09 05:55:40.567 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -81> 2020-05-09 05:55:40.647 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -80> 2020-05-09 06:10:40.648 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -79> 2020-05-09 06:10:40.648 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -78> 2020-05-09 06:10:40.648 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -77> 2020-05-09 06:10:40.688 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -76> 2020-05-09 06:25:40.688 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -75> 2020-05-09 06:25:40.688 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -74> 2020-05-09 06:25:40.688 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -73> 2020-05-09 06:25:41.090 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -72> 2020-05-09 06:55:40.769 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -71> 2020-05-09 06:55:40.769 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -70> 2020-05-09 06:55:40.769 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -69> 2020-05-09 06:55:41.231 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -68> 2020-05-09 07:25:40.849 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -67> 2020-05-09 07:25:40.849 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -66> 2020-05-09 07:25:40.849 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -65> 2020-05-09 07:25:41.324 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -64> 2020-05-09 07:55:40.931 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -63> 2020-05-09 07:55:40.931 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -62> 2020-05-09 07:55:40.931 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -61> 2020-05-09 07:55:41.434 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -60> 2020-05-09 08:22:50.566 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1383] reusing log 667 from recycle list -59> 2020-05-09 08:22:50.566 7f51aa73c700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_write.cc:1423] [default] New memtable created with log file: #690. Immutable memtables: 0. -58> 2020-05-09 08:22:50.566 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-08:22:50.567353) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:1560] Calling FlushMemTableToOutputFile with column family [default], flush slots available 1, compaction slots available 1, flush slots scheduled 1, compaction slots scheduled 0 -57> 2020-05-09 08:22:50.566 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:304] [default] [JOB 203] Flushing memtable with next log file: 690 -56> 2020-05-09 08:22:50.566 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1589012570567454, "job": 203, "event": "flush_started", "num_memtables": 1, "num_entries": 357299, "num_deletes": 72066, "memory_usage": 260049960, "flush_reason": "Write Buffer Full"} -55> 2020-05-09 08:22:50.566 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:334] [default] [JOB 203] Level-0 flush table #691: started -54> 2020-05-09 08:22:50.733 7f51b0f49700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1589012570735087, "cf_name": "default", "job": 203, "event": "table_file_creation", "file_number": 691, "file_size": 34133826, "table_properties": {"data_size": 33276114, "index_size": 484566, "filter_size": 372390, "raw_key_size": 11667216, "raw_average_key_size": 98, "raw_value_size": 27391100, "raw_average_value_size": 230, "num_data_blocks": 7811, "num_entries": 119002, "filter_policy_name": "rocksdb.BuiltinBloomFilter", "kDeletedKeys": "68624", "kMergeOperands": "1070"}} -53> 2020-05-09 08:22:50.733 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/flush_job.cc:374] [default] [JOB 203] Level-0 flush table #691: 34133826 bytes OK -52> 2020-05-09 08:22:50.734 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:186] adding log 688 to recycle list -51> 2020-05-09 08:22:50.734 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-08:22:50.735134) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:377] [default] Level-0 commit table #691 started -50> 2020-05-09 08:22:50.734 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-08:22:50.735304) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/memtable_list.cc:424] [default] Level-0 commit table #691: memtable #1 done -49> 2020-05-09 08:22:50.734 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-08:22:50.735320) EVENT_LOG_v1 {"time_micros": 1589012570735314, "job": 203, "event": "flush_finished", "output_compression": "NoCompression", "lsm_state": [2, 2, 16, 0, 0, 0, 0], "immutable_memtables": 0} -48> 2020-05-09 08:22:50.734 7f51b0f49700 4 rocksdb: (Original Log Time 2020/05/09-08:22:50.735344) [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_compaction_flush.cc:177] [default] Level summary: base level 1 max bytes base 268435456 files[2 2 16 0 0 0 0] max score 0.85 -47> 2020-05-09 08:22:50.734 7f51b0f49700 4 rocksdb: [/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.2.2/rpm/el7/BUILD/ceph-14.2.2/src/rocksdb/db/db_impl_files.cc:347] [JOB 203] Try to delete WAL files size 251937501, prev total WAL file size 251937513, number of live WAL files 2. -46> 2020-05-09 08:25:41.012 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -45> 2020-05-09 08:25:41.012 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -44> 2020-05-09 08:25:41.012 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -43> 2020-05-09 08:25:41.510 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -42> 2020-05-09 08:55:41.092 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -41> 2020-05-09 08:55:41.092 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -40> 2020-05-09 08:55:41.092 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -39> 2020-05-09 08:55:41.172 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -38> 2020-05-09 09:10:41.173 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -37> 2020-05-09 09:10:41.173 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -36> 2020-05-09 09:10:41.173 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -35> 2020-05-09 09:10:41.657 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -34> 2020-05-09 09:40:41.253 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -33> 2020-05-09 09:40:41.253 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -32> 2020-05-09 09:40:41.253 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -31> 2020-05-09 09:40:41.663 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -30> 2020-05-09 10:10:41.334 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -29> 2020-05-09 10:10:41.334 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -28> 2020-05-09 10:10:41.334 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -27> 2020-05-09 10:10:41.749 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -26> 2020-05-09 10:40:41.415 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -25> 2020-05-09 10:40:41.415 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -24> 2020-05-09 10:40:41.415 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -23> 2020-05-09 10:40:41.895 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -22> 2020-05-09 11:10:41.496 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -21> 2020-05-09 11:10:41.496 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -20> 2020-05-09 11:10:41.496 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -19> 2020-05-09 11:10:41.576 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -18> 2020-05-09 11:25:41.577 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -17> 2020-05-09 11:25:41.577 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -16> 2020-05-09 11:25:41.577 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -15> 2020-05-09 11:25:42.041 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -14> 2020-05-09 11:55:41.657 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -13> 2020-05-09 11:55:41.657 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -12> 2020-05-09 11:55:41.657 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -11> 2020-05-09 11:55:41.697 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -10> 2020-05-09 12:10:41.698 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e8a63fe000 -9> 2020-05-09 12:10:41.698 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -8> 2020-05-09 12:10:41.698 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -7> 2020-05-09 12:10:41.738 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -6> 2020-05-09 12:25:41.738 7f51ad4d7700 4 mgrc ms_handle_reset ms_handle_reset con 0x55e88ac17000 -5> 2020-05-09 12:25:41.738 7f51ad4d7700 4 mgrc reconnect Terminating session with v2:172.25.20.39:7040/44588 -4> 2020-05-09 12:25:41.738 7f51ad4d7700 4 mgrc reconnect Starting new session with [v2:172.25.20.39:7040/44588,v1:172.25.20.39:7041/44588] -3> 2020-05-09 12:25:42.227 7f51ad4d7700 4 mgrc handle_mgr_configure stats_period=5 -2> 2020-05-09 12:35:43.861 7f51a9f3b700 0 log_channel(cluster) log [WRN] : Monitor daemon marked osd.61 down, but it is still running -1> 2020-05-09 12:35:43.861 7f51a9f3b700 0 log_channel(cluster) log [DBG] : map e2959 wrongly marked me down at e2958 0> 2020-05-09 12:37:29.499 7f519b71e700 -1 *** Caught signal (Aborted) ** in thread 7f519b71e700 thread_name:tp_osd_tp ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stable) 1: (()+0xf5d0) [0x7f51be5965d0] 2: (pthread_cond_wait()+0xc5) [0x7f51be592965] 3: (std::condition_variable::wait(std::unique_lock&)+0xc) [0x7f51bdce682c] 4: (BlueStore::Collection::flush()+0x86) [0x55e84b47f606] 5: (BlueStore::collection_list(boost::intrusive_ptr&, ghobject_t const&, ghobject_t const&, int, std::vector >*, ghobject_t*)+0x51) [0x55e84b4c7961] 6: (PGBackend::objects_list_range(hobject_t const&, hobject_t const&, std::vector >*, std::vector >*)+0x147) [0x55e84b268187] 7: (PG::build_scrub_map_chunk(ScrubMap&, ScrubMapBuilder&, hobject_t, hobject_t, bool, ThreadPool::TPHandle&)+0x28a) [0x55e84b1144da] 8: (PG::chunky_scrub(ThreadPool::TPHandle&)+0x169c) [0x55e84b142d0c] 9: (PG::scrub(unsigned int, ThreadPool::TPHandle&)+0xaf) [0x55e84b143d4f] 10: (PGScrub::run(OSD*, OSDShard*, boost::intrusive_ptr&, ThreadPool::TPHandle&)+0x12) [0x55e84b2f32e2] 11: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x9f4) [0x55e84b072ef4] 12: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x433) [0x55e84b671ce3] 13: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0x55e84b674d80] 14: (()+0x7dd5) [0x7f51be58edd5] 15: (clone()+0x6d) [0x7f51bd44dead] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 0 lockdep 0/ 0 context 0/ 0 crush 0/ 0 mds 0/ 0 mds_balancer 0/ 0 mds_locker 0/ 0 mds_log 0/ 0 mds_log_expire 0/ 0 mds_migrator 0/ 0 buffer 0/ 0 timer 0/ 0 filer 0/ 1 striper 0/ 0 objecter 0/ 0 rados 0/ 0 rbd 0/ 5 rbd_mirror 0/ 5 rbd_replay 0/ 0 journaler 0/ 0 objectcacher 0/ 0 client 0/ 0 osd 0/ 0 optracker 0/ 0 objclass 0/ 0 filestore 0/ 0 journal 0/ 0 ms 1/ 5 mon 0/ 0 monc 0/ 0 paxos 0/ 0 tp 0/ 0 auth 1/ 5 crypto 0/ 0 finisher 1/ 1 reserver 0/ 0 heartbeatmap 0/ 0 perfcounter 0/ 0 rgw 1/ 5 rgw_sync 1/10 civetweb 1/ 5 javaclient 0/ 0 asok 0/ 0 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 0 bluestore 1/ 0 bluefs 0/ 0 bdev 1/ 5 kstore 4/ 5 rocksdb 4/ 5 leveldb 4/ 5 memdb 1/ 5 kinetic 1/ 5 fuse 1/ 5 mgr 1/ 5 mgrc 1/ 5 dpdk 1/ 5 eventtrace -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /var/log/ceph/ceph-osd.61.log --- end dump of recent events --- os4.te3edm1c2.cdn ~#