Actions
Bug #19956
closedradosgw Segmentation fault when service is restarted during lifecycle processing
Status:
Resolved
Priority:
High
Assignee:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
kraken
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
ceph version 11.2.0 (f223e27eeb35991352ebc1f67423d4ebc252adb7),
1. Trigger a lifecycle gc run
2. Restart the service during this run. (this also happens when the process is HUP'd for the nightly log rotation)
---> ceph will sometimes segfault on restart
/var/log/secure:
May 16 20:55:08 <> ; USER=root ; COMMAND=/bin/systemctl restart ceph.target
/var/log/ceph/client.<host>.log
2017-05-16 20:55:16.798034 7fb5e9b9b700 20 reading from .rgw:.bucket.meta.<>:default.42048218.4 2017-05-16 20:55:16.798049 7fb5e9b9b700 20 get_system_obj_state: rctx=0x7fb5e9b97c90 obj=.rgw:.bucket.meta.<>:default.42048218.4 state=0x7fb6a062b298 s->prefetch_data=0 2017-05-16 20:55:16.799199 7fb5e9b9b700 20 get_system_obj_state: s->obj_tag was set empty 2017-05-16 20:55:16.799248 7fb5e9b9b700 20 rados->read ofs=0 len=524288 2017-05-16 20:55:16.801033 7fb5e9b9b700 20 rados->read r=0 bl.length=257 2017-05-16 20:55:16.801101 7fb5e9b9b700 20 bucket index object: .dir.default.42048218.4.15 2017-05-16 20:55:16.906092 7fb6003c8700 0 gc::process: removing .rgw.buckets:default.42048218.37_%SHADOW_DIR%/generatedshaders/9ffbb662f47bc73c.hlsl/7c5b3169b15b90bf85d0a78b640b35d6 2017-05-16 20:55:16.918084 7fb6003c8700 2 garbage collection: stop 2017-05-16 20:55:16.950814 7fb5e9b9b700 -1 *** Caught signal (Segmentation fault) ** in thread 7fb5e9b9b700 thread_name:lifecycle_thr ceph version 11.2.0 (f223e27eeb35991352ebc1f67423d4ebc252adb7) 1: (()+0x50bdca) [0x7fb694bfadca] 2: (()+0xf370) [0x7fb689e58370] 3: (RGWGC::add_chain(librados::ObjectWriteOperation&, cls_rgw_obj_chain&, std::string const&)+0x62) [0x7fb694b42d62] 4: (RGWGC::send_chain(cls_rgw_obj_chain&, std::string const&, bool)+0x5f) [0x7fb694b42e3f] 5: (RGWRados::Object::complete_atomic_modification()+0xb9) [0x7fb694a01a29] 6: (RGWRados::Object::Delete::delete_obj()+0xd70) [0x7fb694a512a0] 7: (RGWRados::delete_obj(RGWObjectCtx&, RGWBucketInfo&, rgw_obj const&, int, unsigned short, std::chrono::time_point<ceph::time_detail::real_clock, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> > > const&)+0x1c8) [0x7fb694a51ce8] 8: (rgw_remove_object(RGWRados*, RGWBucketInfo&, rgw_bucket&, rgw_obj_key&)+0x25d) [0x7fb694948ced] 9: (RGWLC::bucket_lc_process(std::string&)+0x1281) [0x7fb694b4d971] 10: (RGWLC::process(int, int)+0x4f8) [0x7fb694b4e928] 11: (RGWLC::process()+0x71) [0x7fb694b4edc1] 12: (RGWLC::LCWorker::entry()+0x243) [0x7fb694b4f033] 13: (()+0x7dc5) [0x7fb689e50dc5] 14: (clone()+0x6d) [0x7fb6886b173d] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Actions