|
[truncated]
|
|
-87> 2019-12-13 04:12:32.280 7f9f07378700 10 monclient: tick
|
|
-86> 2019-12-13 04:12:32.280 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:12:02.281906)
|
|
-85> 2019-12-13 04:12:34.700 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-84> 2019-12-13 04:12:35.044 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-83> 2019-12-13 04:12:35.044 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-82> 2019-12-13 04:12:39.400 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-81> 2019-12-13 04:12:40.064 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-80> 2019-12-13 04:12:40.064 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-79> 2019-12-13 04:12:42.280 7f9f07378700 10 monclient: tick
|
|
-78> 2019-12-13 04:12:42.280 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:12:12.282130)
|
|
-77> 2019-12-13 04:12:42.300 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-76> 2019-12-13 04:12:45.080 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-75> 2019-12-13 04:12:45.080 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-74> 2019-12-13 04:12:45.800 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-73> 2019-12-13 04:12:46.300 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-72> 2019-12-13 04:12:48.004 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-71> 2019-12-13 04:12:48.908 7f9f0837a700 4 rocksdb: [db/db_impl.cc:777] ------- DUMPING STATS -------
|
|
-70> 2019-12-13 04:12:48.908 7f9f0837a700 4 rocksdb: [db/db_impl.cc:778]
|
|
** DB Stats **
|
|
Uptime(secs): 1340401.5 total, 600.0 interval
|
|
Cumulative writes: 410K writes, 804K keys, 410K commit groups, 1.0 writes per commit group, ingest: 0.20 GB, 0.00 MB/s
|
|
Cumulative WAL: 410K writes, 202K syncs, 2.02 writes per sync, written: 0.20 GB, 0.00 MB/s
|
|
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
|
|
Interval writes: 361 writes, 646 keys, 361 commit groups, 1.0 writes per commit group, ingest: 0.17 MB, 0.00 MB/s
|
|
Interval WAL: 361 writes, 173 syncs, 2.07 writes per sync, written: 0.00 MB, 0.00 MB/s
|
|
Interval stall: 00:00:0.000 H:M:S, 0.0 percent
|
|
|
|
** Compaction Stats [default] **
|
|
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
|
|
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
L0 2/0 2.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.002 0 0
|
|
Sum 2/0 2.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.002 0 0
|
|
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
|
|
|
|
** Compaction Stats [default] **
|
|
Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
|
|
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.00 0.00 1 0.002 0 0
|
|
Uptime(secs): 1340401.5 total, 600.0 interval
|
|
Flush(GB): cumulative 0.000, interval 0.000
|
|
AddFile(GB): cumulative 0.000, interval 0.000
|
|
AddFile(Total Files): cumulative 0, interval 0
|
|
AddFile(L0 Files): cumulative 0, interval 0
|
|
AddFile(Keys): cumulative 0, interval 0
|
|
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
|
|
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
|
|
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
|
|
|
|
** File Read Latency Histogram By Level [default] **
|
|
|
|
** Compaction Stats [default] **
|
|
Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
|
|
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
L0 2/0 2.60 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.002 0 0
|
|
Sum 2/0 2.60 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.6 0.00 0.00 1 0.002 0 0
|
|
Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0
|
|
|
|
** Compaction Stats [default] **
|
|
Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
|
|
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
|
User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.6 0.00 0.00 1 0.002 0 0
|
|
Uptime(secs): 1340401.5 total, 0.0 interval
|
|
Flush(GB): cumulative 0.000, interval 0.000
|
|
AddFile(GB): cumulative 0.000, interval 0.000
|
|
AddFile(Total Files): cumulative 0, interval 0
|
|
AddFile(L0 Files): cumulative 0, interval 0
|
|
AddFile(Keys): cumulative 0, interval 0
|
|
Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
|
|
Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
|
|
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count
|
|
|
|
** File Read Latency Histogram By Level [default] **
|
|
|
|
-69> 2019-12-13 04:12:50.096 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-68> 2019-12-13 04:12:50.096 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-67> 2019-12-13 04:12:50.304 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-66> 2019-12-13 04:12:52.280 7f9f07378700 10 monclient: tick
|
|
-65> 2019-12-13 04:12:52.280 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:12:22.282356)
|
|
-64> 2019-12-13 04:12:52.604 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-63> 2019-12-13 04:12:53.700 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-62> 2019-12-13 04:12:54.204 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-61> 2019-12-13 04:12:55.112 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-60> 2019-12-13 04:12:55.112 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-59> 2019-12-13 04:12:57.704 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-58> 2019-12-13 04:13:00.124 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-57> 2019-12-13 04:13:00.124 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-56> 2019-12-13 04:13:01.204 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-55> 2019-12-13 04:13:02.280 7f9f07378700 10 monclient: tick
|
|
-54> 2019-12-13 04:13:02.280 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:12:32.282578)
|
|
-53> 2019-12-13 04:13:02.304 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-52> 2019-12-13 04:13:05.144 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-51> 2019-12-13 04:13:05.144 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-50> 2019-12-13 04:13:07.604 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-49> 2019-12-13 04:13:10.157 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-48> 2019-12-13 04:13:10.157 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-47> 2019-12-13 04:13:11.709 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-46> 2019-12-13 04:13:12.281 7f9f07378700 10 monclient: tick
|
|
-45> 2019-12-13 04:13:12.281 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:12:42.282859)
|
|
-44> 2019-12-13 04:13:14.609 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-43> 2019-12-13 04:13:15.177 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-42> 2019-12-13 04:13:15.177 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-41> 2019-12-13 04:13:18.709 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-40> 2019-12-13 04:13:19.809 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-39> 2019-12-13 04:13:20.193 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-38> 2019-12-13 04:13:20.193 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-37> 2019-12-13 04:13:22.281 7f9f07378700 10 monclient: tick
|
|
-36> 2019-12-13 04:13:22.281 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:12:52.283143)
|
|
-35> 2019-12-13 04:13:23.309 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-34> 2019-12-13 04:13:24.409 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-33> 2019-12-13 04:13:25.209 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-32> 2019-12-13 04:13:25.209 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-31> 2019-12-13 04:13:30.225 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-30> 2019-12-13 04:13:30.225 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-29> 2019-12-13 04:13:30.309 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-28> 2019-12-13 04:13:32.281 7f9f07378700 10 monclient: tick
|
|
-27> 2019-12-13 04:13:32.281 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:13:02.283397)
|
|
-26> 2019-12-13 04:13:35.241 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-25> 2019-12-13 04:13:35.241 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-24> 2019-12-13 04:13:36.209 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-23> 2019-12-13 04:13:39.709 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-22> 2019-12-13 04:13:40.257 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-21> 2019-12-13 04:13:40.257 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-20> 2019-12-13 04:13:42.009 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-19> 2019-12-13 04:13:42.281 7f9f07378700 10 monclient: tick
|
|
-18> 2019-12-13 04:13:42.281 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:13:12.283616)
|
|
-17> 2019-12-13 04:13:42.513 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-16> 2019-12-13 04:13:45.273 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-15> 2019-12-13 04:13:45.273 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-14> 2019-12-13 04:13:45.413 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-13> 2019-12-13 04:13:50.114 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-12> 2019-12-13 04:13:50.290 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-11> 2019-12-13 04:13:50.290 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-10> 2019-12-13 04:13:52.282 7f9f07378700 10 monclient: tick
|
|
-9> 2019-12-13 04:13:52.282 7f9f07378700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2019-12-13 04:13:22.283848)
|
|
-8> 2019-12-13 04:13:54.214 7f9efa35e700 5 osd.3 111 heartbeat osd_stat(store_statfs(0x50000/0x3f100000/0x40000000, data 0x881e0/0xe10000, compress 0x0/0x0/0x0, omap 0x0, meta 0x3f100000), peers [0,1,2] op hist [])
|
|
-7> 2019-12-13 04:13:55.306 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _tune_cache_size target: 4294967296 heap: 330711040 unmapped: 13713408 mapped: 316997632 old cache_size: 2845415832 new cache size: 2845415832
|
|
-6> 2019-12-13 04:13:55.306 7f9f0e839700 5 bluestore.MempoolThread(0xc0ddb60) _trim_shards cache_size: 2845415832 kv_alloc: 1073741824 kv_used: 1888 meta_alloc: 1073741824 meta_used: 735298 data_alloc: 671088640 data_used: 0
|
|
-5> 2019-12-13 04:13:55.914 7f9f0f83b700 1 bluefs _allocate failed to allocate 0x100000 on bdev 1, free 0x0; fallback to bdev 2
|
|
-4> 2019-12-13 04:13:55.914 7f9f0f83b700 1 bluefs _allocate unable to allocate 0x100000 on bdev 2, free 0xffffffffffffffff; fallback to slow device expander
|
|
-3> 2019-12-13 04:13:55.914 7f9f0f83b700 -1 bluefs _allocate failed to allocate 0x100000 on bdev 1, free 0x0
|
|
-2> 2019-12-13 04:13:55.914 7f9f0f83b700 -1 bluefs _flush_range allocated: 0xce00000 offset: 0xcdffe68 length: 0x40d
|
|
-1> 2019-12-13 04:13:55.938 7f9f0f83b700 -1 /build/ceph-14.2.4/src/os/bluestore/BlueFS.cc: In function 'int BlueFS::_flush_range(BlueFS::FileWriter*, uint64_t, uint64_t)' thread 7f9f0f83b700 time 2019-12-13 04:13:55.919740
|
|
/build/ceph-14.2.4/src/os/bluestore/BlueFS.cc: 2132: ceph_abort_msg("bluefs enospc")
|
|
|
|
ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable)
|
|
1: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0xdf) [0x85ca72]
|
|
2: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1e26) [0xe84d16]
|
|
3: (BlueFS::_flush(BlueFS::FileWriter*, bool)+0x11c) [0xe8506c]
|
|
4: (BlueFS::_fsync(BlueFS::FileWriter*, std::unique_lock<std::mutex>&)+0x4d) [0xe881ad]
|
|
5: (BlueRocksWritableFile::Sync()+0x63) [0xea62c3]
|
|
6: (rocksdb::WritableFileWriter::SyncInternal(bool)+0x3e9) [0x15184b9]
|
|
7: (rocksdb::WritableFileWriter::Sync(bool)+0x376) [0x151b8a6]
|
|
8: (rocksdb::DBImpl::WriteToWAL(rocksdb::WriteThread::WriteGroup const&, rocksdb::log::Writer*, unsigned long*, bool, bool, unsigned long)+0x32c) [0x13866dc]
|
|
9: (rocksdb::DBImpl::WriteImpl(rocksdb::WriteOptions const&, rocksdb::WriteBatch*, rocksdb::WriteCallback*, unsigned long*, unsigned long, bool, unsigned long*, unsigned long, rocksdb::PreReleaseCallback*)+0x245d) [0x138f75d]
|
|
10: (rocksdb::DBImpl::Write(rocksdb::WriteOptions const&, rocksdb::WriteBatch*)+0x30) [0x138fdf0]
|
|
11: (RocksDBStore::submit_common(rocksdb::WriteOptions&, std::shared_ptr<KeyValueDB::TransactionImpl>)+0x81) [0xe0e0d1]
|
|
12: (RocksDBStore::submit_transaction_sync(std::shared_ptr<KeyValueDB::TransactionImpl>)+0x97) [0xe0eab7]
|
|
13: (BlueStore::_kv_sync_thread()+0x1b2a) [0xdc1b8a]
|
|
14: (BlueStore::KVSyncThread::entry()+0xd) [0xde352d]
|
|
15: (()+0x76ba) [0x7f9f1e88f6ba]
|
|
16: (clone()+0x6d) [0x7f9f1de9641d]
|
|
|
|
0> 2019-12-13 04:13:55.990 7f9f0f83b700 -1 *** Caught signal (Aborted) **
|
|
in thread 7f9f0f83b700 thread_name:bstore_kv_sync
|
|
|
|
ceph version 14.2.4 (75f4de193b3ea58512f204623e6c5a16e6c1e1ba) nautilus (stable)
|
|
1: (()+0x11390) [0x7f9f1e899390]
|
|
2: (gsignal()+0x38) [0x7f9f1ddc4428]
|
|
3: (abort()+0x16a) [0x7f9f1ddc602a]
|
|
4: (ceph::__ceph_abort(char const*, int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x1b0) [0x85cb43]
|
|
5: (BlueFS::_flush_range(BlueFS::FileWriter*, unsigned long, unsigned long)+0x1e26) [0xe84d16]
|
|
6: (BlueFS::_flush(BlueFS::FileWriter*, bool)+0x11c) [0xe8506c]
|
|
7: (BlueFS::_fsync(BlueFS::FileWriter*, std::unique_lock<std::mutex>&)+0x4d) [0xe881ad]
|
|
8: (BlueRocksWritableFile::Sync()+0x63) [0xea62c3]
|
|
9: (rocksdb::WritableFileWriter::SyncInternal(bool)+0x3e9) [0x15184b9]
|
|
10: (rocksdb::WritableFileWriter::Sync(bool)+0x376) [0x151b8a6]
|
|
11: (rocksdb::DBImpl::WriteToWAL(rocksdb::WriteThread::WriteGroup const&, rocksdb::log::Writer*, unsigned long*, bool, bool, unsigned long)+0x32c) [0x13866dc]
|
|
12: (rocksdb::DBImpl::WriteImpl(rocksdb::WriteOptions const&, rocksdb::WriteBatch*, rocksdb::WriteCallback*, unsigned long*, unsigned long, bool, unsigned long*, unsigned long, rocksdb::PreReleaseCallback*)+0x245d) [0x138f75d]
|
|
13: (rocksdb::DBImpl::Write(rocksdb::WriteOptions const&, rocksdb::WriteBatch*)+0x30) [0x138fdf0]
|
|
14: (RocksDBStore::submit_common(rocksdb::WriteOptions&, std::shared_ptr<KeyValueDB::TransactionImpl>)+0x81) [0xe0e0d1]
|
|
15: (RocksDBStore::submit_transaction_sync(std::shared_ptr<KeyValueDB::TransactionImpl>)+0x97) [0xe0eab7]
|
|
16: (BlueStore::_kv_sync_thread()+0x1b2a) [0xdc1b8a]
|
|
17: (BlueStore::KVSyncThread::entry()+0xd) [0xde352d]
|
|
18: (()+0x76ba) [0x7f9f1e88f6ba]
|
|
19: (clone()+0x6d) [0x7f9f1de9641d]
|
|
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
|
|
|
|
--- logging levels ---
|
|
0/ 5 none
|
|
0/ 1 lockdep
|
|
0/ 1 context
|
|
1/ 1 crush
|
|
1/ 5 mds
|
|
1/ 5 mds_balancer
|
|
1/ 5 mds_locker
|
|
1/ 5 mds_log
|
|
1/ 5 mds_log_expire
|
|
1/ 5 mds_migrator
|
|
0/ 1 buffer
|
|
0/ 1 timer
|
|
0/ 1 filer
|
|
0/ 1 striper
|
|
0/ 1 objecter
|
|
0/ 5 rados
|
|
0/ 5 rbd
|
|
0/ 5 rbd_mirror
|
|
0/ 5 rbd_replay
|
|
0/ 5 journaler
|
|
0/ 5 objectcacher
|
|
0/ 5 client
|
|
1/ 5 osd
|
|
0/ 5 optracker
|
|
0/ 5 objclass
|
|
1/ 3 filestore
|
|
1/ 3 journal
|
|
0/ 0 ms
|
|
1/ 5 mon
|
|
0/10 monc
|
|
1/ 5 paxos
|
|
0/ 5 tp
|
|
1/ 5 auth
|
|
1/ 5 crypto
|
|
1/ 1 finisher
|
|
1/ 1 reserver
|
|
1/ 5 heartbeatmap
|
|
1/ 5 perfcounter
|
|
1/ 5 rgw
|
|
1/ 5 rgw_sync
|
|
1/10 civetweb
|
|
1/ 5 javaclient
|
|
1/ 5 asok
|
|
1/ 1 throttle
|
|
0/ 0 refs
|
|
1/ 5 xio
|
|
1/ 5 compressor
|
|
1/ 5 bluestore
|
|
1/ 5 bluefs
|
|
1/ 3 bdev
|
|
1/ 5 kstore
|
|
4/ 5 rocksdb
|
|
4/ 5 leveldb
|
|
4/ 5 memdb
|
|
1/ 5 kinetic
|
|
1/ 5 fuse
|
|
1/ 5 mgr
|
|
1/ 5 mgrc
|
|
1/ 5 dpdk
|
|
1/ 5 eventtrace
|
|
-2/-2 (syslog threshold)
|
|
-1/-1 (stderr threshold)
|
|
max_recent 10000
|
|
max_new 1000
|
|
log_file /var/lib/ceph/crash/2019-12-13_03:13:55.981131Z_e3f0cf94-b3a9-4acc-8917-4d6dcee735a6/log
|
|
--- end dump of recent events ---
|