Ceph » rgw

Hi,

We recently upgraded our small (4 node) production cluster from Nautilus to Pacific, and started seeing RadosGW crashes:

{
"backtrace": [
"/lib/x86_64-linux-gnu/libc.so.6(+0x3f040) [0x7f9b20524040]",
"(rgw_bucket::rgw_bucket(rgw_bucket const&)+0x1b) [0x7f9b210c8e2b]",
"(RGWInitMultipart::verify_permission(optional_yield)+0x82) [0x7f9b214473e2]",
"(rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, optional_yield, bool)+0xd38) [0x7f9b210a1048]",
"(process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSocket*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int)+0x3609) [0x7f9b210a68b9]",
"(RGWCivetWebFrontend::process(mg_connection*)+0x269) [0x7f9b20fc01f9]",
"/usr/lib/libradosgw.so.2(+0x640b62) [0x7f9b21186b62]",
"/usr/lib/libradosgw.so.2(+0x6427ef) [0x7f9b211887ef]",
"/usr/lib/libradosgw.so.2(+0x642ca8) [0x7f9b21188ca8]",
"/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f9b14db36db]",
"clone()"
],
"ceph_version": "16.2.1",
"crash_id": "2021-04-27T16:13:55.804495Z_b4b18ecf-9408-4e8b-9781-ae8c1918cede",
"entity_name": "client.rgw.ceph4.rgw0",
"os_id": "ubuntu",
"os_name": "Ubuntu",
"os_version": "18.04.5 LTS (Bionic Beaver)",
"os_version_id": "18.04",
"process_name": "radosgw",
"stack_sig": "8c23946311f29e1f658aafdf6fdde6cda1f21a8f422dbc9191ff6fe86aff390c",
"timestamp": "2021-04-27T16:13:55.804495Z",
"utsname_hostname": "ceph4",
"utsname_machine": "x86_64",
"utsname_release": "5.4.0-72-generic",
"utsname_sysname": "Linux",
"utsname_version": "#80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021"
}

We tracked it down, and it happens whenever we attempt a multipart upload to a bucket, that has a bucket policy set. Any policy on the bucket, even a trivial (empty) one, triggers the crash. Regular (non-multipart) uploads work fine.