Project

General

Profile

Bug #47722 ยป ceph-issue.txt

Arnaud Lefebvre, 10/01/2020 05:53 PM

 
2020-10-01 16:44:12.764 7fd9e81f8700 1 ====== starting new request req=0x7fd9c43b9070 =====
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s initializing for trans_id = tx000000000000000000487-005f76075c-8d0b095-default
2020-10-01 16:44:12.764 7fd9e81f8700 10 rgw api priority: s3=2 s3website=-1
2020-10-01 16:44:12.764 7fd9e81f8700 10 host=<our domain>
2020-10-01 16:44:12.764 7fd9e81f8700 20 subdomain= domain=<our domain> in_hosted_domain=1 in_hosted_domain_s3website=0
2020-10-01 16:44:12.764 7fd9e81f8700 20 final domain/bucket subdomain= domain=<our domain> in_hosted_domain=1 in_hosted_domain_s3website=0 s->info.domain=<our domain> s->info.request_uri=/<the_file>
2020-10-01 16:44:12.764 7fd9e81f8700 20 get_handler handler=22RGWHandler_REST_Obj_S3
2020-10-01 16:44:12.764 7fd9e81f8700 10 handler=22RGWHandler_REST_Obj_S3
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s getting op 0
2020-10-01 16:44:12.764 7fd9e81f8700 10 req 1159 0.000s s3:get_obj scheduling with dmclock client=2 cost=1
2020-10-01 16:44:12.764 7fd9e81f8700 10 op=21RGWGetObj_ObjStore_S3
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s s3:get_obj verifying requester
2020-10-01 16:44:12.764 7fd9e81f8700 20 req 1159 0.000s s3:get_obj rgw::auth::StrategyRegistry::s3_main_strategy_t: trying rgw::auth::s3::AWSAuthStrategy
2020-10-01 16:44:12.764 7fd9e81f8700 20 req 1159 0.000s s3:get_obj rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::S3AnonymousEngine
2020-10-01 16:44:12.764 7fd9e81f8700 20 req 1159 0.000s s3:get_obj rgw::auth::s3::S3AnonymousEngine denied with reason=-1
2020-10-01 16:44:12.764 7fd9e81f8700 20 req 1159 0.000s s3:get_obj rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::LocalEngine
2020-10-01 16:44:12.764 7fd9e81f8700 10 get_canon_resource(): dest=/<the_file>
2020-10-01 16:44:12.764 7fd9e81f8700 10 string_to_sign:
GET


1601573483
/<the_file>
2020-10-01 16:44:12.764 7fd9e81f8700 15 req 1159 0.000s s3:get_obj string_to_sign=GET


1601573483
/<the_file>
2020-10-01 16:44:12.764 7fd9e81f8700 15 req 1159 0.000s s3:get_obj server signature=kL81aM7FpEQxVJHHaGSCGyciBnE=
2020-10-01 16:44:12.764 7fd9e81f8700 15 req 1159 0.000s s3:get_obj client signature=kL81aM7FpEQxVJHHaGSCGyciBnE=
2020-10-01 16:44:12.764 7fd9e81f8700 15 req 1159 0.000s s3:get_obj compare=0
2020-10-01 16:44:12.764 7fd9e81f8700 20 req 1159 0.000s s3:get_obj rgw::auth::s3::LocalEngine granted access
2020-10-01 16:44:12.764 7fd9e81f8700 20 req 1159 0.000s s3:get_obj rgw::auth::s3::AWSAuthStrategy granted access
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s s3:get_obj normalizing buckets and tenants
2020-10-01 16:44:12.764 7fd9e81f8700 10 s->object=<the_file> s->bucket=<the bucket>
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s s3:get_obj init permissions
2020-10-01 16:44:12.764 7fd9e81f8700 15 decode_policy Read AccessControlPolicy<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID><some id></ID><DisplayName><some other id></DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID><some id></ID><DisplayName><some other id></DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
2020-10-01 16:44:12.764 7fd9e81f8700 20 get_system_obj_state: rctx=0x7fd9c43b6b80 obj=default.rgw.meta:users.uid:<some id> state=0x7fda0a12c0a0 s->prefetch_data=0
2020-10-01 16:44:12.764 7fd9e81f8700 10 cache get: name=default.rgw.meta+users.uid+<some id> : hit (requested=0x6, cached=0x17)
2020-10-01 16:44:12.764 7fd9e81f8700 20 get_system_obj_state: s->obj_tag was set empty
2020-10-01 16:44:12.764 7fd9e81f8700 20 Read xattr: user.rgw.idtag
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s s3:get_obj recalculating target
2020-10-01 16:44:12.764 7fd9e81f8700 2 req 1159 0.000s s3:get_obj reading permissions
2020-10-01 16:44:12.764 7fd9e81f8700 20 get_obj_state: rctx=0x7fd9c43b7c60 obj=<the_file> state=0x7fda0a00ee08 s->prefetch_data=0
2020-10-01 16:44:12.766 7fd9e81f8700 10 manifest: total_size = 414177017
2020-10-01 16:44:12.766 7fd9e81f8700 20 get_obj_state: setting s->obj_tag to _lCobmExLYMXttLkVu7re5UqnUEVHWMp
2020-10-01 16:44:12.766 7fd9e81f8700 15 decode_policy Read AccessControlPolicy<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID><some id></ID><DisplayName><some other id></DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID><some id></ID><DisplayName><some other id></DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
2020-10-01 16:44:12.766 7fd9e81f8700 2 req 1159 0.002s s3:get_obj init op
2020-10-01 16:44:12.766 7fd9e81f8700 2 req 1159 0.002s s3:get_obj verifying op mask
2020-10-01 16:44:12.766 7fd9e81f8700 20 req 1159 0.002s s3:get_obj required_mask= 1 user.op_mask=7
2020-10-01 16:44:12.766 7fd9e81f8700 2 req 1159 0.002s s3:get_obj verifying op permissions
2020-10-01 16:44:12.766 7fd9e81f8700 20 req 1159 0.002s s3:get_obj -- Getting permissions begin with perm_mask=49
2020-10-01 16:44:12.766 7fd9e81f8700 5 req 1159 0.002s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=<some id>, acct_name=<some other id>, subuser=, perm_mask=15, is_admin=0) mask=49
2020-10-01 16:44:12.766 7fd9e81f8700 5 Searching permissions for uid=<some id>
2020-10-01 16:44:12.766 7fd9e81f8700 5 Found permission: 15
2020-10-01 16:44:12.766 7fd9e81f8700 5 Searching permissions for group=1 mask=49
2020-10-01 16:44:12.766 7fd9e81f8700 5 Permissions for group not found
2020-10-01 16:44:12.766 7fd9e81f8700 5 Searching permissions for group=2 mask=49
2020-10-01 16:44:12.766 7fd9e81f8700 5 Permissions for group not found
2020-10-01 16:44:12.766 7fd9e81f8700 5 req 1159 0.002s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=<some id>, acct_name=<some other id>, subuser=, perm_mask=15, is_admin=0), owner=<some id>, perm=1
2020-10-01 16:44:12.766 7fd9e81f8700 10 req 1159 0.002s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=<some id>, acct_name=<some other id>, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
2020-10-01 16:44:12.766 7fd9e81f8700 2 req 1159 0.002s s3:get_obj verifying op params
2020-10-01 16:44:12.766 7fd9e81f8700 2 req 1159 0.002s s3:get_obj pre-executing
2020-10-01 16:44:12.766 7fd9e81f8700 2 req 1159 0.002s s3:get_obj executing
2020-10-01 16:44:12.766 7fd9e81f8700 20 get_obj_state: rctx=0x7fd9c43b7c60 obj=<the_file> state=0x7fda0a00ee08 s->prefetch_data=1
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.acl
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.content_type
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.etag
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.idtag
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.manifest
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.pg_ver
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.source_zone
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.tail_tag
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.x-amz-acl
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.x-amz-date
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.x-amz-meta-s3b-last-modified
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.x-amz-metadata-directive
2020-10-01 16:44:12.766 7fd9e81f8700 20 Read xattr: user.rgw.x-amz-storage-class
2020-10-01 16:44:12.766 7fd9e81f8700 15 Encryption mode:
2020-10-01 16:44:12.766 7fd9e81f8700 20 get_obj_state: rctx=0x7fd9c43b7c60 obj=<the_file> state=0x7fda0a00ee08 s->prefetch_data=1
2020-10-01 16:44:12.766 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__multipart_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.25 obj-ofs=254935040 read_ofs=3276800 len=917504
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=255852544 part_ofs=251658240 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=255852544 stripe_ofs=255852544 part_ofs=251658240 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__shadow_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.25_1 obj-ofs=255852544 read_ofs=0 len=4194304
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=260046848 part_ofs=251658240 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=260046848 stripe_ofs=260046848 part_ofs=251658240 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__shadow_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.25_2 obj-ofs=260046848 read_ofs=0 len=2097152
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=264241152 part_ofs=251658240 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=262144000 stripe_ofs=262144000 part_ofs=262144000 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__multipart_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.26 obj-ofs=262144000 read_ofs=0 len=4194304
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=266338304 part_ofs=262144000 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=266338304 stripe_ofs=266338304 part_ofs=262144000 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__shadow_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.26_1 obj-ofs=266338304 read_ofs=0 len=4194304
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=270532608 part_ofs=262144000 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=270532608 stripe_ofs=270532608 part_ofs=262144000 rule->part_size=10485760
2020-10-01 16:44:12.766 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__shadow_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.26_2 obj-ofs=270532608 read_ofs=0 len=2097152
2020-10-01 16:44:12.792 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.792 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=274726912 part_ofs=262144000 rule->part_size=10485760
2020-10-01 16:44:12.792 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=272629760 stripe_ofs=272629760 part_ofs=272629760 rule->part_size=10485760
2020-10-01 16:44:12.792 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__multipart_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.27 obj-ofs=272629760 read_ofs=0 len=4194304
2020-10-01 16:44:12.820 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.820 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=276824064 part_ofs=272629760 rule->part_size=10485760
2020-10-01 16:44:12.820 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=276824064 stripe_ofs=276824064 part_ofs=272629760 rule->part_size=10485760
2020-10-01 16:44:12.820 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__shadow_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.27_1 obj-ofs=276824064 read_ofs=0 len=4194304
2020-10-01 16:44:12.835 7fd9e81f8700 20 RGWObjManifest::operator++(): rule->part_size=10485760 rules.size()=2
2020-10-01 16:44:12.835 7fd9e81f8700 20 RGWObjManifest::operator++(): stripe_ofs=281018368 part_ofs=272629760 rule->part_size=10485760
2020-10-01 16:44:12.835 7fd9e81f8700 20 RGWObjManifest::operator++(): result: ofs=281018368 stripe_ofs=281018368 part_ofs=272629760 rule->part_size=10485760
2020-10-01 16:44:12.835 7fd9e81f8700 20 rados->get_obj_iterate_cb oid=d7f493dc-9dec-4bc3-96f8-c5efe76c79f3.4196.2__shadow_<the_file>.2~xS5GP1e28GTJSb3qR-dQQtjlXVHR9dE.27_2 obj-ofs=281018368 read_ofs=0 len=2097152
2020-10-01 16:44:19.372 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:44:41.371 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:45:03.372 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:45:25.372 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:45:47.373 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:46:09.372 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:46:31.373 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
2020-10-01 16:46:53.373 7fda07bfb700 2 RGWDataChangesLog::ChangesRenewThread: start
    (1-1/1)