Project

General

Profile

Bug #10946 » ceph-osd.1.log

Artem Savinov, 02/25/2015 08:51 AM

 
2015-02-25 10:35:18.283711 7f89e2bc8800 0 ceph version 0.87 (c51c8f9d80fa4e0168aa52685b8de40e42758578), process ceph-osd, pid 28756
2015-02-25 10:35:18.291103 7f89e2bc8800 0 filestore(/var/lib/ceph/osd/ceph-1) backend generic (magic 0xef53)
2015-02-25 10:35:19.598310 7f89e2bc8800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-1) detect_features: FIEMAP ioctl is supported and appears to work
2015-02-25 10:35:19.598353 7f89e2bc8800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-02-25 10:35:20.403936 7f89e2bc8800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-1) detect_features: syscall(SYS_syncfs, fd) fully supported
2015-02-25 10:35:20.410387 7f89e2bc8800 0 filestore(/var/lib/ceph/osd/ceph-1) limited size xattrs
2015-02-25 10:35:20.506595 7f89e2bc8800 0 filestore(/var/lib/ceph/osd/ceph-1) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2015-02-25 10:35:20.514795 7f89e2bc8800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2015-02-25 10:35:20.514860 7f89e2bc8800 1 journal _open /var/lib/ceph/osd/ceph-1/journal fd 21: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-02-25 10:35:20.669552 7f89e2bc8800 1 journal _open /var/lib/ceph/osd/ceph-1/journal fd 21: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-02-25 10:35:20.726818 7f89e2bc8800 1 journal close /var/lib/ceph/osd/ceph-1/journal
2015-02-25 10:35:20.744981 7f89e2bc8800 0 filestore(/var/lib/ceph/osd/ceph-1) backend generic (magic 0xef53)
2015-02-25 10:35:22.131528 7f89e2bc8800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-1) detect_features: FIEMAP ioctl is supported and appears to work
2015-02-25 10:35:22.131552 7f89e2bc8800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-02-25 10:35:22.149922 7f89e2bc8800 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-1) detect_features: syscall(SYS_syncfs, fd) fully supported
2015-02-25 10:35:22.155553 7f89e2bc8800 0 filestore(/var/lib/ceph/osd/ceph-1) limited size xattrs
2015-02-25 10:35:22.206968 7f89e2bc8800 0 filestore(/var/lib/ceph/osd/ceph-1) mount: WRITEAHEAD journal mode explicitly enabled in conf
2015-02-25 10:35:22.212249 7f89e2bc8800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway
2015-02-25 10:35:22.212285 7f89e2bc8800 1 journal _open /var/lib/ceph/osd/ceph-1/journal fd 21: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-02-25 10:35:22.212560 7f89e2bc8800 1 journal _open /var/lib/ceph/osd/ceph-1/journal fd 21: 1048576000 bytes, block size 4096 bytes, directio = 1, aio = 0
2015-02-25 10:35:22.218159 7f89e2bc8800 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello
2015-02-25 10:35:22.218661 7f89e2bc8800 0 osd.1 37 crush map has features 1107558400, adjusting msgr requires for clients
2015-02-25 10:35:22.218696 7f89e2bc8800 0 osd.1 37 crush map has features 1107558400 was 8705, adjusting msgr requires for mons
2015-02-25 10:35:22.218720 7f89e2bc8800 0 osd.1 37 crush map has features 1107558400, adjusting msgr requires for osds
2015-02-25 10:35:22.218772 7f89e2bc8800 0 osd.1 37 load_pgs
2015-02-25 10:35:22.477396 7f89e2bc8800 0 osd.1 37 load_pgs opened 768 pgs
2015-02-25 10:35:22.478708 7f89e2bc8800 -1 osd.1 37 set_disk_tp_priority(22) Invalid argument: osd_disk_thread_ioprio_class is but only the following values are allowed: idle, be or rt
2015-02-25 10:35:22.484795 7f89cf0bd700 0 osd.1 37 ignoring osdmap until we have initialized
2015-02-25 10:35:22.485014 7f89cf0bd700 0 osd.1 37 ignoring osdmap until we have initialized
2015-02-25 10:35:23.021257 7f89e2bc8800 0 osd.1 37 done with init, starting boot process
2015-02-25 10:35:31.367623 7f89bc49f700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849731.367617000

2015-02-25 10:35:31.552267 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849731.552264000

2015-02-25 10:35:31.594677 7f89c14a7700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849731.594674000

2015-02-25 10:35:31.667746 7f89beca3700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849731.667742000

2015-02-25 10:35:31.731633 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849731.731629000

2015-02-25 10:35:31.885138 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849731.885134000

2015-02-25 10:35:32.051935 7f89c0aa6700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849732.051931000

2015-02-25 10:35:32.210565 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849732.210561000

2015-02-25 10:36:33.137118 7f89bf6a4700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849793.137115000

2015-02-25 10:36:33.317460 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849793.317457000

2015-02-25 10:36:33.359376 7f89c14a7700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849793.359373000

2015-02-25 10:36:38.342231 7f89c1ea8700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849798.342227000

2015-02-25 10:36:38.393592 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849798.393588000

2015-02-25 10:36:38.617144 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849798.617141000

2015-02-25 10:36:38.772340 7f89c0aa6700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849798.772337000

2015-02-25 10:36:38.926254 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849798.926252000

2015-02-25 10:37:32.673552 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849852.673549000

2015-02-25 10:37:32.822290 7f89bc49f700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849852.822287000

2015-02-25 10:37:32.961407 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849852.961404000

2015-02-25 10:37:33.004667 7f89c14a7700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849853.004664000

2015-02-25 10:37:33.081347 7f89beca3700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849853.081344000

2015-02-25 10:37:33.166768 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849853.166765000

2015-02-25 10:37:33.299641 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849853.299638000

2015-02-25 10:37:33.434840 7f89bd8a1700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849853.434834000

2015-02-25 10:38:32.542199 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849912.542196000

2015-02-25 10:38:32.591587 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849912.591584000

2015-02-25 10:38:32.657359 7f89c1ea8700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849912.657348000

2015-02-25 10:38:34.222251 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849914.222247000

2015-02-25 10:38:39.605801 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849919.605797000

2015-02-25 10:38:40.665110 7f89bd8a1700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849920.665107000

2015-02-25 10:38:40.817785 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849920.817781000

2015-02-25 10:38:40.999449 7f89bf6a4700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849920.999446000

2015-02-25 10:39:33.180919 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849973.180916000

2015-02-25 10:39:33.274523 7f89bd8a1700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849973.274521000

2015-02-25 10:39:33.425470 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849973.425467000

2015-02-25 10:39:34.324063 7f89bc49f700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849974.324060000

2015-02-25 10:39:36.674600 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849976.674596000

2015-02-25 10:39:36.848605 7f89c14a7700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849976.848603000

2015-02-25 10:39:36.925911 7f89c1ea8700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849976.925909000

2015-02-25 10:39:36.954071 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424849976.954069000

2015-02-25 10:40:42.350151 7f89bf6a4700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850042.350148000

2015-02-25 10:40:42.508977 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850042.508974000

2015-02-25 10:40:42.581284 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850042.581281000

2015-02-25 10:40:42.684143 7f89beca3700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850042.684141000

2015-02-25 10:40:42.725213 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850042.725211000

2015-02-25 10:40:42.843137 7f89c14a7700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850042.843134000

2015-02-25 10:40:43.084137 7f89bd8a1700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850043.084135000

2015-02-25 10:40:43.232216 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850043.232213000

2015-02-25 10:41:37.561505 7f89bcea0700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850097.561501000

2015-02-25 10:41:38.814011 7f89be2a2700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850098.814008000

2015-02-25 10:41:39.374856 7f89c1ea8700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850099.374853000

2015-02-25 10:41:39.407257 7f89c00a5700 0 <cls> cls/rgw/cls_rgw.cc:1458: gc_iterate_entries end_key=1_01424850099.407254000

2015-02-25 10:41:58.035412 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:38.035409)
2015-02-25 10:41:58.035444 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:38.035409)
2015-02-25 10:41:59.035577 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:39.035575)
2015-02-25 10:41:59.035597 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:39.035575)
2015-02-25 10:42:00.035745 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:40.035743)
2015-02-25 10:42:00.035766 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:40.035743)
2015-02-25 10:42:01.035894 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:41.035892)
2015-02-25 10:42:01.035916 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:41.035892)
2015-02-25 10:42:01.612679 7f89b9c9b700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:41.612678)
2015-02-25 10:42:01.612686 7f89b9c9b700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:41.612678)
2015-02-25 10:42:02.036061 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:42.036059)
2015-02-25 10:42:02.036081 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:42.036059)
2015-02-25 10:42:03.036261 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:43.036259)
2015-02-25 10:42:03.036293 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:43.036259)
2015-02-25 10:42:04.036422 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:44.036420)
2015-02-25 10:42:04.036441 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:44.036420)
2015-02-25 10:42:04.783643 7f89b5e93700 0 -- 172.20.32.17:6802/28756 >> 172.20.32.5:6804/29835 pipe(0x4819340 sd=27 :41552 s=2 pgs=4 cs=1 l=0 c=0x573b180).fault with nothing to send, going to standby
2015-02-25 10:42:05.036576 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:45.036575)
2015-02-25 10:42:05.036600 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:45.036575)
2015-02-25 10:42:05.112935 7f89b9c9b700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:45.112934)
2015-02-25 10:42:05.112941 7f89b9c9b700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:45.112934)
2015-02-25 10:42:05.613198 7f89b9c9b700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:45.613197)
2015-02-25 10:42:05.613206 7f89b9c9b700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:45.613197)
2015-02-25 10:42:06.036751 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:46.036749)
2015-02-25 10:42:06.036775 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:46.036749)
2015-02-25 10:42:07.036972 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.0 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:47.036970)
2015-02-25 10:42:07.036998 7f89d72ee700 -1 osd.1 48 heartbeat_check: no reply from osd.2 since back 2015-02-25 10:41:37.910366 front 2015-02-25 10:41:37.910366 (cutoff 2015-02-25 10:41:47.036970)
2015-02-25 10:42:07.038547 7f89cf0bd700 0 log_channel(default) log [WRN] : map e50 wrongly marked me down
2015-02-25 10:43:10.055910 7f89b4382700 0 -- 172.20.32.17:6800/1028756 >> 172.20.32.5:6804/29835 pipe(0x6b0bf40 sd=27 :0 s=1 pgs=0 cs=0 l=0 c=0x573bf40).fault with nothing to send, going to standby
(5-5/10)