Project

General

Profile

Bug #41370

[RGW] RGW in website mode: rgw_rados.h: 2150: FAILED assert(!obj.empty()

Added by Aleksandr Rudenko about 2 years ago. Updated about 1 year ago.

Status:
Duplicate
Priority:
Normal
Assignee:
Target version:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Hi guys!

Luminous 12.2.12

I have several instances in website mode. See their config:

[client]
    rgw crypt require ssl = false
    rgw dns name = xxx.xx
    rgw enable apis = s3, admin
    rgw dynamic resharding = true
    rgw enable usage log = false
    rgw enable ops log = false
    rgw enable gc threads = false
    rgw enable lc threads = false
    rgw remote addr param = HTTP_X_FORWARDED_FOR
    rgw max objs per shard = 256000
    rgw num rados handles = 2
    rgw thread pool size = 256
[client.xxx.x]
    host = xxx
    keyring = /var/lib/ceph/radosgw/xxx.x.keyring
    rgw dns s3website name = xxx.xx
    rgw enable apis = s3website
    rgw enable gc threads = true
    rgw enable lc threads = true
    rgw gc max objs = 24
    rgw frontends = civetweb num_threads=256 port=0.0.0.0:7481 enable_keep_alive=yes access_log_file=xxx.log error_log_file=xxx.log

All rgw instances in website mode are crashed a few time a day with this error:


2019-08-21 08:12:58.407798 7f2a10393700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.12/rpm/el7/BUILD/ceph-12.2.12/src/rgw/rgw_rados.h: In function 'void RGWObjectCtxImpl<T, S>::set_atomic(T&) [with T = rgw_obj; S = RGWObjState]' thread 7f2a10393700 time 2019-08-21 08:12:58.406053
/home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/12.2.12/rpm/el7/BUILD/ceph-12.2.12/src/rgw/rgw_rados.h: 2150: FAILED assert(!obj.empty())

 ceph version 12.2.12 (1436006594665279fe734b4c15d7e08c13ebd777) luminous (stable)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x110) [0x7f2a4301ffd0]
 2: (RGWObjectCtxImpl<rgw_obj, RGWObjState>::set_atomic(rgw_obj&)+0x238) [0x7f2a4d28b8b8]
 3: (RGWGetObj::verify_permission()+0x107) [0x7f2a4d25f907]
 4: (rgw_process_authenticated(RGWHandler_REST*, RGWOp*&, RGWRequest*, req_state*, bool)+0x100) [0x7f2a4d290990]
 5: (process_request(RGWRados*, RGWREST*, RGWRequest*, std::string const&, rgw::auth::StrategyRegistry const&, RGWRestfulIO*, OpsLogSocket*, int*)+0xb98) [0x7f2a4d2917b8]
 6: (RGWCivetWebFrontend::process(mg_connection*)+0x3a2) [0x7f2a4d11a8d2]
 7: (()+0x2afa7f) [0x7f2a4d189a7f]
 8: (()+0x2b140b) [0x7f2a4d18b40b]
 9: (()+0x7dc5) [0x7f2a4ba5bdc5]
 10: (clone()+0x6d) [0x7f2a3ffa3ced]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---

Related issues

Duplicates rgw - Bug #40678: RGW S3Website didn't do the necessary checking to the website configuration Resolved

History

#1 Updated by Casey Bodley about 2 years ago

  • Assignee set to Casey Bodley

#2 Updated by Aleksandr Rudenko about 2 years ago

It happens if website mode was enabled but index page was not set.
After this, all requests for index page will crash rgw instance with this error.

#3 Updated by Robin Johnson about 1 year ago

  • Pull request ID set to 28904

PR 28904 fixed this in master.

commits:
c96f415dafe176b1b8d10ff9456d13fb76c79baa
355f392ad26631f44dac250296e96f421d86fb8f

#4 Updated by Nathan Cutler about 1 year ago

  • Duplicates Bug #40678: RGW S3Website didn't do the necessary checking to the website configuration added

#5 Updated by Nathan Cutler about 1 year ago

  • Status changed from New to Duplicate

Thanks, @Robin

Also available in: Atom PDF