Project

General

Profile

Actions
Copy link

Bug #62063

open

notification tests fail on 'radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph'

Added by Casey Bodley 10 months ago. Updated 12 days ago.

Status:
New
Priority:
Urgent
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

the 'user rm' command fails during cleanup:

2023-07-18T05:57:45.504 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph
2023-07-18T05:57:45.592 INFO:teuthology.orchestra.run.smithi092.stderr:ignoring --setuser ceph since I am not root
2023-07-18T05:57:45.593 INFO:teuthology.orchestra.run.smithi092.stderr:ignoring --setgroup ceph since I am not root
2023-07-18T06:47:45.612 INFO:teuthology.orchestra.run.smithi092.stderr:failed to fetch mon config (--no-mon-config to skip)
2023-07-18T06:47:45.613 DEBUG:teuthology.orchestra.run:got remote process result: 1

but the real issue appears to be an earlier crash from radosgw:

2023-07-18T05:44:38.067 INFO:tasks.rgw.client.0.smithi092.stdout:*** Caught signal (Segmentation fault) **
2023-07-18T05:44:38.067 INFO:tasks.rgw.client.0.smithi092.stdout: in thread 7f29832f8700 thread_name:radosgw
2023-07-18T05:44:38.070 INFO:tasks.rgw.client.0.smithi092.stdout: ceph version 18.0.0-4979-g5f1dece0 (5f1dece0019cdc7070e1396837e4fc3ade471a1d) reef (dev)
2023-07-18T05:44:38.070 INFO:tasks.rgw.client.0.smithi092.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f2ab62cacf0]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 2: radosgw(+0x941371) [0x55b745c75371]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 3: (RGWObjManifest::~RGWObjManifest()+0x1d3) [0x55b745cceb13]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 4: (std::_Rb_tree<rgw_obj, std::pair<rgw_obj const, RGWObjStateManifest>, std::_Select1st<std::pair<rgw_obj const, RGWObjStateManifest> >, std::less<rgw_obj>, std::allocator<std::pair<rgw_obj const, RGWObjStateManifest> > >::_M_erase(std::_Rb_tree_node<std::pair<rgw_obj const, RGWObjStateManifest> >*)+0x4f) [0x55b745fa843f]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 5: (rgw::sal::RadosObject::~RadosObject()+0x41) [0x55b745f78e51]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 6: (rgw::sal::RadosObject::~RadosObject()+0xd) [0x55b745f7901d]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 7: (req_state::~req_state()+0x71c) [0x55b745c1316c]
2023-07-18T05:44:38.071 INFO:tasks.rgw.client.0.smithi092.stdout: 8: (process_request(RGWProcessEnv const&, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RGWRestfulIO*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, int*)+0x207d) [0x55b745ab1abd]
2023-07-18T05:44:38.072 INFO:tasks.rgw.client.0.smithi092.stdout: 9: radosgw(+0x6b22be) [0x55b7459e62be]
2023-07-18T05:44:38.072 INFO:tasks.rgw.client.0.smithi092.stdout: 10: radosgw(+0x6b2f87) [0x55b7459e6f87]
2023-07-18T05:44:38.072 INFO:tasks.rgw.client.0.smithi092.stdout: 11: make_fcontext()

http://qa-proxy.ceph.com/teuthology/cbodley-2023-07-17_21:26:30-rgw-main-distro-default-smithi/7342641/teuthology.log

Actions
Copy link
 #1

Updated by Casey Bodley 8 months ago

  • Backport set to reef
Actions
Copy link
 #2

Updated by Casey Bodley 6 months ago

  • Status changed from New to Can't reproduce

not seeing this lately. we'll reopen if it pops up again

Actions
Copy link
 #3

Updated by Casey Bodley 12 days ago

  • Status changed from Can't reproduce to New

happening on quincy: https://qa-proxy.ceph.com/teuthology/yuriw-2024-04-20_15:31:09-rgw-wip-yuri4-testing-2024-04-19-0708-quincy-distro-default-smithi/7664647/teuthology.log

2024-04-20T15:58:29.356 INFO:tasks.rgw.client.0.smithi071.stdout:*** Caught signal (Segmentation fault) **
2024-04-20T15:58:29.356 INFO:tasks.rgw.client.0.smithi071.stdout: in thread 7f5fc4057700 thread_name:radosgw
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: ceph version 17.2.7-902-g7f4fa80c (7f4fa80c8c97f53f52b1189d2347b5761f6287e3) quincy (stable)
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7f60dfddacf0]
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 2: /lib64/libradosgw.so.2(+0x90c5b1) [0x7f60e289e5b1]
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 3: (RGWObjManifest::~RGWObjManifest()+0x1d3) [0x7f60e28b5233]
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 4: (RGWObjState::~RGWObjState()+0x1e3) [0x7f60e2acf883]
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 5: /lib64/libradosgw.so.2(+0x6e46ee) [0x7f60e26766ee]
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 6: (process_request(rgw::sal::Store*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSink*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*, std::chrono::duration<unsigned long, std::ratio<1l, 1000000000l> >*, std::shared_ptr<RateLimiter>, int*)+0x1702) [0x7f60e267a882]
2024-04-20T15:58:29.359 INFO:tasks.rgw.client.0.smithi071.stdout: 7: /lib64/libradosgw.so.2(+0x645765) [0x7f60e25d7765]
2024-04-20T15:58:29.360 INFO:tasks.rgw.client.0.smithi071.stdout: 8: /lib64/libradosgw.so.2(+0x646ba6) [0x7f60e25d8ba6]
2024-04-20T15:58:29.360 INFO:tasks.rgw.client.0.smithi071.stdout: 9: make_fcontext()
Actions
Copy link

Also available in: Atom PDF