Project

General

Profile

Bug #8947 » ceph-osd.2.log

log for crashed osd - Mallikarjun Biradar, 07/28/2014 08:15 AM

 
-44> 2014-07-25 18:28:15.643832 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.549632, event: header_read, op: pg_info(1 pgs e65:4.ba)
-43> 2014-07-25 18:28:15.643840 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.549632, event: throttled, op: pg_info(1 pgs e65:4.ba)
-42> 2014-07-25 18:28:15.643847 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.549648, event: all_read, op: pg_info(1 pgs e65:4.ba)
-41> 2014-07-25 18:28:15.643854 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.643829, event: dispatched, op: pg_info(1 pgs e65:4.ba)
-40> 2014-07-25 18:28:15.643861 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.643859, event: waiting_for_osdmap, op: pg_info(1 pgs e65:4.ba)
-39> 2014-07-25 18:28:15.643870 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.643867, event: started, op: pg_info(1 pgs e65:4.ba)
-38> 2014-07-25 18:28:15.643887 7f4f3c1d7700 5 -- op tracker -- , seq: 1259, time: 2014-07-25 18:28:15.643885, event: done, op: pg_info(1 pgs e65:4.ba)
-37> 2014-07-25 18:28:15.643897 7f4f3c1d7700 1 -- 10.242.24.154:6803/28779 <== osd.1 10.242.24.111:6801/31290 595 ==== pg_info(1 pgs e65:6.13) v4 ==== 714+0+0 (1148986833 0 0) 0x5230d80 con 0x487c420
-36> 2014-07-25 18:28:15.643908 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.550659, event: header_read, op: pg_info(1 pgs e65:6.13)
-35> 2014-07-25 18:28:15.643916 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.550660, event: throttled, op: pg_info(1 pgs e65:6.13)
-34> 2014-07-25 18:28:15.643923 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.550873, event: all_read, op: pg_info(1 pgs e65:6.13)
-33> 2014-07-25 18:28:15.643930 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.643905, event: dispatched, op: pg_info(1 pgs e65:6.13)
-32> 2014-07-25 18:28:15.643938 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.643936, event: waiting_for_osdmap, op: pg_info(1 pgs e65:6.13)
-31> 2014-07-25 18:28:15.643946 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.643944, event: started, op: pg_info(1 pgs e65:6.13)
-30> 2014-07-25 18:28:15.643962 7f4f3c1d7700 5 -- op tracker -- , seq: 1260, time: 2014-07-25 18:28:15.643961, event: done, op: pg_info(1 pgs e65:6.13)
-29> 2014-07-25 18:28:15.643972 7f4f3c1d7700 1 -- 10.242.24.154:6803/28779 <== osd.0 10.66.26.5:6801/12812 626 ==== pg_info(1 pgs e65:1.a) v4 ==== 714+0+0 (96229761 0 0) 0x4dd0bc0 con 0x487c160
-28> 2014-07-25 18:28:15.643983 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.549656, event: header_read, op: pg_info(1 pgs e65:1.a)
-27> 2014-07-25 18:28:15.644024 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.549657, event: throttled, op: pg_info(1 pgs e65:1.a)
-26> 2014-07-25 18:28:15.644032 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.549748, event: all_read, op: pg_info(1 pgs e65:1.a)
-25> 2014-07-25 18:28:15.644039 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.643981, event: dispatched, op: pg_info(1 pgs e65:1.a)
-24> 2014-07-25 18:28:15.644046 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.644045, event: waiting_for_osdmap, op: pg_info(1 pgs e65:1.a)
-23> 2014-07-25 18:28:15.644055 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.644053, event: started, op: pg_info(1 pgs e65:1.a)
-22> 2014-07-25 18:28:15.644071 7f4f3c1d7700 5 -- op tracker -- , seq: 1261, time: 2014-07-25 18:28:15.644069, event: done, op: pg_info(1 pgs e65:1.a)
-21> 2014-07-25 18:28:15.644082 7f4f3c1d7700 1 -- 10.242.24.154:6803/28779 <== osd.1 10.242.24.111:6801/31290 596 ==== pg_info(1 pgs e65:6.10) v4 ==== 714+0+0 (522635800 0 0) 0x5230680 con 0x487c420
-20> 2014-07-25 18:28:15.644093 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.550890, event: header_read, op: pg_info(1 pgs e65:6.10)
-19> 2014-07-25 18:28:15.644101 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.550891, event: throttled, op: pg_info(1 pgs e65:6.10)
-18> 2014-07-25 18:28:15.644108 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.550907, event: all_read, op: pg_info(1 pgs e65:6.10)
-17> 2014-07-25 18:28:15.644117 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.644091, event: dispatched, op: pg_info(1 pgs e65:6.10)
-16> 2014-07-25 18:28:15.644125 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.644123, event: waiting_for_osdmap, op: pg_info(1 pgs e65:6.10)
-15> 2014-07-25 18:28:15.644133 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.644132, event: started, op: pg_info(1 pgs e65:6.10)
-14> 2014-07-25 18:28:15.644150 7f4f3c1d7700 5 -- op tracker -- , seq: 1262, time: 2014-07-25 18:28:15.644148, event: done, op: pg_info(1 pgs e65:6.10)
-13> 2014-07-25 18:28:15.644164 7f4f3c1d7700 1 -- 10.242.24.154:6803/28779 <== osd.0 10.66.26.5:6801/12812 627 ==== pg_info(1 pgs e65:8.9) v4 ==== 714+0+0 (527915233 0 0) 0x4dcec40 con 0x487c160
-12> 2014-07-25 18:28:15.644175 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.549772, event: header_read, op: pg_info(1 pgs e65:8.9)
-11> 2014-07-25 18:28:15.644184 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.549772, event: throttled, op: pg_info(1 pgs e65:8.9)
-10> 2014-07-25 18:28:15.644191 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.549987, event: all_read, op: pg_info(1 pgs e65:8.9)
-9> 2014-07-25 18:28:15.644168 7f4f299b2700 -1 *** Caught signal (Aborted) **
in thread 7f4f299b2700

ceph version andisk-sprint-2-drop-3-390-g2dbd85c (2dbd85c94cf27a1ff0419c5ea9359af7fe30e9b6)
1: /usr/bin/ceph-osd() [0x974a1f]
2: (()+0x10340) [0x7f4f4c5ee340]
3: (gsignal()+0x39) [0x7f4f4a8c2f79]
4: (abort()+0x148) [0x7f4f4a8c6388]
5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7f4f4b1ce6b5]
6: (()+0x5e836) [0x7f4f4b1cc836]
7: (()+0x5e863) [0x7f4f4b1cc863]
8: (()+0x5eaa2) [0x7f4f4b1ccaa2]
9: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1f2) [0xa5a692]
10: (ReplicatedPG::agent_estimate_atime_temp(hobject_t const&, int*, int*)+0x12e) [0x7ad25e]
11: (ReplicatedPG::agent_maybe_evict(std::tr1::shared_ptr<ObjectContext>&)+0x7a) [0x7e1a2a]
12: (ReplicatedPG::agent_work(int)+0xf45) [0x7fb1d5]
13: (OSDService::agent_entry()+0x557) [0x6090c7]
14: (OSDService::AgentThread::entry()+0xd) [0x67201d]
15: (()+0x8182) [0x7f4f4c5e6182]
16: (clone()+0x6d) [0x7f4f4a98730d]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

-8> 2014-07-25 18:28:15.644199 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.644172, event: dispatched, op: pg_info(1 pgs e65:8.9)
-7> 2014-07-25 18:28:15.644207 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.644205, event: waiting_for_osdmap, op: pg_info(1 pgs e65:8.9)
-6> 2014-07-25 18:28:15.644233 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.644230, event: started, op: pg_info(1 pgs e65:8.9)
-5> 2014-07-25 18:28:15.644249 7f4f3c1d7700 5 -- op tracker -- , seq: 1263, time: 2014-07-25 18:28:15.644247, event: done, op: pg_info(1 pgs e65:8.9)
-4> 2014-07-25 18:28:15.644258 7f4f3c1d7700 1 -- 10.242.24.154:6803/28779 <== osd.1 10.242.24.111:6801/31290 597 ==== pg_info(1 pgs e65:8.26) v4 ==== 714+0+0 (367980315 0 0) 0x4dcf180 con 0x487c420
-3> 2014-07-25 18:28:15.644267 7f4f3c1d7700 5 -- op tracker -- , seq: 1264, time: 2014-07-25 18:28:15.550916, event: header_read, op: pg_info(1 pgs e65:8.26)
-2> 2014-07-25 18:28:15.644273 7f4f3c1d7700 5 -- op tracker -- , seq: 1264, time: 2014-07-25 18:28:15.550916, event: throttled, op: pg_info(1 pgs e65:8.26)
-1> 2014-07-25 18:28:15.644282 7f4f3c1d7700 5 -- op tracker -- , seq: 1264, time: 2014-07-25 18:28:15.551111, event: all_read, op: pg_info(1 pgs e65:8.26)
0> 2014-07-25 18:28:15.644291 7f4f3c1d7700 5 -- op tracker -- , seq: 1264, time: 2014-07-25 18:28:15.644265, event: dispatched, op: pg_info(1 pgs e65:8.26)
--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
0/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 keyvaluestore
1/ 3 journal
0/ 5 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 5 heartbeatmap
1/ 5 perfcounter
1/ 5 rgw
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
-2/-2 (syslog threshold)
-1/-1 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/log/ceph/ceph-osd.2.log
--- end dump of recent events ---

(2-2/5)