Actions
Bug #10572
closedrgw: crash on shutdown, when syncing user stats
% Done:
0%
Source:
other
Tags:
Backport:
firefly
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
A crash during shutdown when sync user stats operation is running:
ceph version 0.90-648-gdbaa142 (dbaa1420c40092e26b26da6b3175d026983e32a3) 1: radosgw() [0x6880fa] 2: (()+0x10340) [0x8111340] 3: (PerfCounters::inc(int, unsigned long)+0x7) [0x71ffb7] 4: (ObjectCache::get(std::string&, ObjectCacheInfo&, unsigned int, rgw_cache_entry_info*)+0x5d8) [0x62d988] 5: (RGWCache<RGWRados>::obj_stat(void*, rgw_obj&, unsigned long*, long*, unsigned long*, std::map<std::string, ceph::buffer::list, std::less<std::string>, std::allocator<std::pair<std::string const, ceph::buffer::list> > >*, ceph::buffer::list*, RGWObjVersionTracker*)+0x283) [0x6025f3] 6: (RGWRados::get_obj_state(RGWRadosCtx*, rgw_obj&, RGWObjState**, RGWObjVersionTracker*)+0xd2) [0x5da502] 7: (RGWRados::prepare_get_obj(void*, rgw_obj&, long*, long*, std::map<std::string, ceph::buffer::list, std::less<std::string>, std::allocator<std::pair<std::string const, ceph::buffer::list> > >*, long const*, long const*, long*, char const*, char const*, unsigned long*, unsigned long*, RGWObjVersionTracker*, void**, rgw_err*)+0xc4) [0x5df254] 8: (rgw_get_system_obj(RGWRados*, void*, rgw_bucket&, std::string const&, ceph::buffer::list&, RGWObjVersionTracker*, long*, std::map<std::string, ceph::buffer::list, std::less<std::string>, std::allocator<std::pair<std::string const, ceph::buffer::list> > >*, rgw_cache_entry_info*)+0x341) [0x5baee1] 9: (RGWRados::get_bucket_instance_from_oid(void*, std::string&, RGWBucketInfo&, long*, std::map<std::string, ceph::buffer::list, std::less<std::string>, std::allocator<std::pair<std::string const, ceph::buffer::list> > >*, rgw_cache_entry_info*)+0x123) [0x5c71e3] 10: (RGWRados::get_bucket_instance_info(void*, rgw_bucket&, RGWBucketInfo&, long*, std::map<std::string, ceph::buffer::list, std::less<std::string>, std::allocator<std::pair<std::string const, ceph::buffer::list> > >*)+0x67) [0x5c7617] 11: (RGWRados::open_bucket_index(rgw_bucket&, librados::IoCtx&, std::map<int, std::string, std::less<int>, std::allocator<std::pair<int const, std::string> > >&, int, std::map<int, std::string, std::less<int>, std::allocator<std::pair<int const, std::string> > >*)+0x13e) [0x5d224e] 12: (int RGWRados::open_bucket_index<rgw_cls_list_ret>(rgw_bucket&, librados::IoCtx&, std::map<int, std::string, std::less<int>, std::allocator<std::pair<int const, std::string> > >&, std::map<int, rgw_cls_list_ret, std::less<int>, std::allocator<std::pair<int const, rgw_cls_list_ret> > >&, int, std::map<int, std::string, std::less<int>, std::allocator<std::pair<int const, std::string> > >*)+0x29) [0x6014a9] 13: (RGWRados::cls_bucket_head(rgw_bucket&, std::map<std::string, rgw_bucket_dir_header, std::less<std::string>, std::allocator<std::pair<std::string const, rgw_bucket_dir_header> > >&, std::map<int, std::string, std::less<int>, std::allocator<std::pair<int const, std::string> > >*)+0xf3) [0x5ea283] 14: (RGWRados::cls_user_sync_bucket_stats(rgw_obj&, rgw_bucket&)+0x5e) [0x5eb01e] 15: (rgw_bucket_sync_user_stats(RGWRados*, std::string const&, rgw_bucket&)+0x58) [0x5a5838] 16: (RGWUserStatsCache::sync_bucket(std::string const&, rgw_bucket&)+0x23) [0x6501b3] 17: (RGWUserStatsCache::BucketsSyncThread::entry()+0x171) [0x654de1] 18: (()+0x8182) [0x8109182] 19: (clone()+0x6d) [0x99ed38d]
Could be that we're not waiting to or not shutting down the sync thread correctly.
Updated by Yehuda Sadeh over 9 years ago
- Status changed from New to Fix Under Review
- Assignee changed from Yehuda Sadeh to Josh Durgin
Updated by Josh Durgin over 9 years ago
- Status changed from Fix Under Review to Pending Backport
This is fixed in next in commit:de2e5fa048639de6c9ee004a93ab295625fa3b94
I tried merging next to master, but there are several conflicts in rgw. Could you resolve those Yehuda?
Updated by Yehuda Sadeh over 9 years ago
The fix for this is broken. When backporting, please also cherry-pick the follow up fix.
Updated by Sage Weil about 9 years ago
- Status changed from Pending Backport to Resolved
not going to backport this unless we see it on firefly.
Actions