2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[78]: shard_id=78 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[78]: shard_id=78 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b6456a700 20 run: stack=0x3b963c0 is io blocked 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003072-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header: 2018-11-23 15:34:33.865 7f3b6456a700 20 cr:s=0x3b968c0:op=0x3dc2c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.865 7f3b6456a700 20 cr:s=0x3b968c0:op=0x3ef5e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[61]: shard_id=61 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[61]: shard_id=61 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b6456a700 20 run: stack=0x3b968c0 is io blocked 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000306a-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header: 2018-11-23 15:34:33.865 7f3b6456a700 20 cr:s=0x352e780:op=0x38caa00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.865 7f3b6456a700 20 cr:s=0x352e780:op=0x3f0de00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000306d-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[10]: shard_id=10 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[10]: shard_id=10 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.865 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.865 7f3b63568700 10 received header: 2018-11-23 15:34:33.865 7f3b6456a700 20 run: stack=0x352e780 is io blocked 2018-11-23 15:34:33.865 7f3b6456a700 20 cr:s=0x3b96f00:op=0x51c2400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.865 7f3b6456a700 20 cr:s=0x3b96f00:op=0x3b64800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[64]: shard_id=64 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b6456a700 20 RGW-SYNC:data:sync:shard[64]: shard_id=64 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.865 7f3b6456a700 20 run: stack=0x3b96f00 is io blocked 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x3c8f400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3ebca00:op=0x3c8e000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 run: stack=0x3b98580 is io blocked 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3ebca00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98f80:op=0x5090a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98300:op=0x3f16800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x38e8000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3ebca00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 run: stack=0x3ebca00 is io blocked 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98f80:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98300:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98f80:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 run: stack=0x3b98f80 is io blocked 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98300:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 run: stack=0x3b98300 is io blocked 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 run: stack=0x3b99ac0 is io blocked 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x3c8f400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 RGW-SYNC:data:sync:shard[87]: read error repo, got 0 entries 2018-11-23 15:34:33.869 7f3b6456a700 20 cr:s=0x3b98580:op=0x51c2400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.869 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.869 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.869 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.869 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.869 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=87&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.869 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=303, curl_handle=0x4085ae0 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b98580 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebca00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebca00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x5091e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x50d4900 req_data->id=303, curl_handle=0x4085ae0 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebca00:op=0x3c8e000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[108]: read error repo, got 0 entries 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x38e8000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[96]: read error repo, got 0 entries 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebca00:op=0x38caa00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.873 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.873 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=108&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.873 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=304, curl_handle=0x4f98d80 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3ebca00 is io blocked 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x4cf4400 req_data->id=304, curl_handle=0x4f98d80 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3ebc780 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x3dc2c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.873 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.873 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=96&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.873 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=305, curl_handle=0x3b2de80 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b99ac0 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b28bc0:op=0x38c9200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b28bc0:op=0x38e9400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[29]: shard_id=29 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x50d6640 req_data->id=305, curl_handle=0x3b2de80 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[29]: shard_id=29 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b28bc0 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b28300:op=0x38cb600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b28300:op=0x3ee6800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[25]: shard_id=25 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[25]: shard_id=25 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b28300 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98f80:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98f80:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98f80:op=0x5090a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[79]: read error repo, got 0 entries 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x3c8fe00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98f80:op=0x38cb600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.873 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.873 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=79&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.873 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=306, curl_handle=0x3fdf580 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b98f80 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98300:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98300:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x40e18c0 req_data->id=306, curl_handle=0x3fdf580 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b995c0 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98300:op=0x3f16800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[71]: read error repo, got 0 entries 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b98300:op=0x38c9200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.873 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.873 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=71&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.873 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=307, curl_handle=0x5234e00 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b98300 is io blocked 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x3bd3b00 req_data->id=307, curl_handle=0x5234e00 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x5091e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[106]: read error repo, got 0 entries 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3ebc780:op=0x3dc2600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.873 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.873 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=106&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.873 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=308, curl_handle=0x3fdec80 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3ebc780 is io blocked 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x3c8fe00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.873 7f3b6456a700 20 RGW-SYNC:data:sync:shard[92]: read error repo, got 0 entries 2018-11-23 15:34:33.873 7f3b6456a700 20 cr:s=0x3b995c0:op=0x38c9e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x4df3f80 req_data->id=308, curl_handle=0x3fdec80 2018-11-23 15:34:33.873 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.873 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.873 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.873 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=92&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.873 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=309, curl_handle=0x52cb620 2018-11-23 15:34:33.873 7f3b6456a700 20 run: stack=0x3b995c0 is io blocked 2018-11-23 15:34:33.873 7f3b63568700 20 link_request req_data=0x4cf4880 req_data->id=309, curl_handle=0x52cb620 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003074-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header: 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003076-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header: 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003077-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header: 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003075-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header: 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003078-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.873 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.873 7f3b63568700 10 received header: 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x508c000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000307a-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header: 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3b981c0 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x3c0e000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x5091400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebcb40 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3b99c00 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x3f2a800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebc000 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50b0a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5090000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebd2c0 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebd540 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x508c000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[75]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b981c0:op=0x5120c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003079-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.877 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.877 7f3b63568700 10 received header: 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=75&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=310, curl_handle=0x3fb2d20 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3b981c0 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x50ec6c0 req_data->id=310, curl_handle=0x3fb2d20 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x3c0e000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[109]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x4f8d800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=109&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=311, curl_handle=0x4bcfbe0 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebcb40 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x5091400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[97]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99c00:op=0x4f8cc00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x381f3c0 req_data->id=311, curl_handle=0x4bcfbe0 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=97&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=312, curl_handle=0x4085c60 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3b99c00 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x3f2a800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[100]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebc000:op=0x5122400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=100&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=313, curl_handle=0x3eeff60 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebc000 is io blocked 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x3f93600 req_data->id=312, curl_handle=0x4085c60 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x50ef840 req_data->id=313, curl_handle=0x3eeff60 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50b1e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x3c0fe00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebe080 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebd7c0 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50b0a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[115]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x3dc3200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=115&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=314, curl_handle=0x3fb2320 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebd2c0 is io blocked 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x4cf3b00 req_data->id=314, curl_handle=0x3fb2320 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5090000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[117]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd540:op=0x51c1200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=117&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=315, curl_handle=0x4bd1440 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebd540 is io blocked 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x4df4400 req_data->id=315, curl_handle=0x4bd1440 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x3c90800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3b99200 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x3c0fe00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[119]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x51c2a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=119&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=316, curl_handle=0x3fb3940 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebd7c0 is io blocked 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x40e2880 req_data->id=316, curl_handle=0x3fb3940 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50b1e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[126]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3ebe080:op=0x51c0600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=126&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=317, curl_handle=0x52cad80 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3ebe080 is io blocked 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x50d5b00 req_data->id=317, curl_handle=0x52cad80 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x3c90800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 RGW-SYNC:data:sync:shard[89]: read error repo, got 0 entries 2018-11-23 15:34:33.877 7f3b6456a700 20 cr:s=0x3b99200:op=0x51c3000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.877 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.877 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.877 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.877 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=89&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.877 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=318, curl_handle=0x52c8c80 2018-11-23 15:34:33.877 7f3b6456a700 20 run: stack=0x3b99200 is io blocked 2018-11-23 15:34:33.877 7f3b63568700 20 link_request req_data=0x50ed680 req_data->id=318, curl_handle=0x52c8c80 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000307b-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000307e-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000307c-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000307f-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000307d-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003080-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003081-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003082-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3ebd540:op=0x51c1200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3ebd540:op=0x5090000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[117]: shard_id=117 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[117]: shard_id=117 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 run: stack=0x3ebd540 is io blocked 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x51c2a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3ebe080:op=0x51c0600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3ebd7c0:op=0x3c0fe00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[119]: shard_id=119 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[119]: shard_id=119 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 run: stack=0x3ebd7c0 is io blocked 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3ebe080:op=0x50b1e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[126]: shard_id=126 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[126]: shard_id=126 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 run: stack=0x3ebe080 is io blocked 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003083-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.881 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.881 7f3b63568700 10 received header: 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3b99200:op=0x51c3000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 cr:s=0x3b99200:op=0x3c90800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[89]: shard_id=89 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 RGW-SYNC:data:sync:shard[89]: shard_id=89 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.881 7f3b6456a700 20 run: stack=0x3b99200 is io blocked 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x3f0f200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x3efca00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 run: stack=0x3b29c00 is io blocked 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 run: stack=0x352e500 is io blocked 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x3f0f200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 RGW-SYNC:data:sync:shard[32]: read error repo, got 0 entries 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x3b29c00:op=0x51c3000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.885 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.885 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.885 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.885 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=32&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.885 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=319, curl_handle=0x52c8c80 2018-11-23 15:34:33.885 7f3b6456a700 20 run: stack=0x3b29c00 is io blocked 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x3efca00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.885 7f3b63568700 20 link_request req_data=0x40e2880 req_data->id=319, curl_handle=0x52c8c80 2018-11-23 15:34:33.885 7f3b6456a700 20 RGW-SYNC:data:sync:shard[20]: read error repo, got 0 entries 2018-11-23 15:34:33.885 7f3b6456a700 20 cr:s=0x352e500:op=0x51c0600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.885 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.885 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.885 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.885 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.885 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=20&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.885 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=320, curl_handle=0x52cad80 2018-11-23 15:34:33.885 7f3b6456a700 20 run: stack=0x352e500 is io blocked 2018-11-23 15:34:33.885 7f3b63568700 20 link_request req_data=0x50ed680 req_data->id=320, curl_handle=0x52cad80 2018-11-23 15:34:33.885 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.885 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.885 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.885 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003085-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.885 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.885 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.885 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.885 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.885 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.885 7f3b63568700 10 received header: 2018-11-23 15:34:33.889 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.889 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.889 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.889 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003084-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.889 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.889 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.889 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.889 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.889 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.889 7f3b63568700 10 received header: 2018-11-23 15:34:33.889 7f3b6456a700 20 cr:s=0x3b28580:op=0x5120000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.889 7f3b6456a700 20 cr:s=0x3b28580:op=0x3f3d400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.889 7f3b6456a700 20 RGW-SYNC:data:sync:shard[34]: shard_id=34 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.889 7f3b6456a700 20 RGW-SYNC:data:sync:shard[34]: shard_id=34 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.889 7f3b6456a700 20 run: stack=0x3b28580 is io blocked 2018-11-23 15:34:33.889 7f3b6456a700 20 cr:s=0x352f7c0:op=0x4f8de00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.889 7f3b6456a700 20 cr:s=0x352f7c0:op=0x4dd3e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.889 7f3b6456a700 20 RGW-SYNC:data:sync:shard[8]: shard_id=8 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.889 7f3b6456a700 20 RGW-SYNC:data:sync:shard[8]: shard_id=8 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.889 7f3b6456a700 20 run: stack=0x352f7c0 is io blocked 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b977c0:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b977c0:op=0x3f32800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 RGW-SYNC:data:sync:shard[74]: shard_id=74 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.893 7f3b6456a700 20 RGW-SYNC:data:sync:shard[74]: shard_id=74 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.893 7f3b6456a700 20 run: stack=0x3b977c0 is io blocked 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x3f05400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x3f30a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 run: stack=0x35310c0 is io blocked 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 run: stack=0x3b990c0 is io blocked 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x3f30a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 RGW-SYNC:data:sync:shard[88]: read error repo, got 0 entries 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x3b990c0:op=0x5123000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.893 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.893 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.893 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.893 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=88&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.893 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=321, curl_handle=0x4f99d20 2018-11-23 15:34:33.893 7f3b6456a700 20 run: stack=0x3b990c0 is io blocked 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x3f05400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 RGW-SYNC:data:sync:shard[31]: read error repo, got 0 entries 2018-11-23 15:34:33.893 7f3b63568700 20 link_request req_data=0x5158000 req_data->id=321, curl_handle=0x4f99d20 2018-11-23 15:34:33.893 7f3b6456a700 20 cr:s=0x35310c0:op=0x5121e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.893 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.893 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.893 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.893 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.893 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=31&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.893 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=322, curl_handle=0x3b2ada0 2018-11-23 15:34:33.893 7f3b6456a700 20 run: stack=0x35310c0 is io blocked 2018-11-23 15:34:33.893 7f3b63568700 20 link_request req_data=0x4df53c0 req_data->id=322, curl_handle=0x3b2ada0 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003086-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header: 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003087-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.893 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.893 7f3b63568700 10 received header: 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x3ef1400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x3f07200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x3ee5e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 run: stack=0x3b96c80 is io blocked 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 run: stack=0x3b98800 is io blocked 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 run: stack=0x3b29d40 is io blocked 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x3ee5e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 RGW-SYNC:data:sync:shard[39]: read error repo, got 0 entries 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b29d40:op=0x5121200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.897 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.897 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.897 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.897 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=39&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.897 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=323, curl_handle=0x3fb2ec0 2018-11-23 15:34:33.897 7f3b6456a700 20 run: stack=0x3b29d40 is io blocked 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x3f07200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 RGW-SYNC:data:sync:shard[85]: read error repo, got 0 entries 2018-11-23 15:34:33.897 7f3b63568700 20 link_request req_data=0x4cf41c0 req_data->id=323, curl_handle=0x3fb2ec0 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x3ef1400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 RGW-SYNC:data:sync:shard[60]: read error repo, got 0 entries 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b98800:op=0x5120600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.897 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.897 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.897 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.897 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.897 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=85&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.897 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=324, curl_handle=0x3fb3940 2018-11-23 15:34:33.897 7f3b6456a700 20 run: stack=0x3b98800 is io blocked 2018-11-23 15:34:33.897 7f3b6456a700 20 cr:s=0x3b96c80:op=0x5123600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.897 7f3b63568700 20 link_request req_data=0x3f92d00 req_data->id=324, curl_handle=0x3fb3940 2018-11-23 15:34:33.897 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.897 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.897 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.897 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.897 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=60&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.897 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=325, curl_handle=0x4bd1440 2018-11-23 15:34:33.897 7f3b6456a700 20 run: stack=0x3b96c80 is io blocked 2018-11-23 15:34:33.897 7f3b63568700 20 link_request req_data=0x50d5b00 req_data->id=325, curl_handle=0x4bd1440 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003088-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header: 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003089-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header: 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000308a-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.897 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.897 7f3b63568700 10 received header: 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98940:op=0x51c0000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98940:op=0x3f14000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[84]: shard_id=84 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[84]: shard_id=84 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b98940 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b27680:op=0x5122a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b27680:op=0x3b79200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[55]: shard_id=55 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[55]: shard_id=55 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b27680 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x3edf200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b28940 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x3edf200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[53]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28940:op=0x5122a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=53&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=326, curl_handle=0x3eefa20 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b28940 is io blocked 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x50d5440 req_data->id=326, curl_handle=0x3eefa20 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x3ee8a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x38eb200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x3edc000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b98bc0 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x3ef3200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x352ea00 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b96280 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b28080 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x3f04000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b29980 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x3f15e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b98d00 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x3b62a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b99840 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x3ee8a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[82]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x3f17200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x51c0000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=82&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=327, curl_handle=0x3eeee20 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b98bc0 is io blocked 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x3881440 req_data->id=327, curl_handle=0x3eeee20 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x3ef5400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3ebcc80 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x3edc000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[46]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x38eb200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[9]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b96280:op=0x5120000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=46&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=328, curl_handle=0x3fb0c40 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b96280 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x3ef3200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[48]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x3f93a80 req_data->id=328, curl_handle=0x3fb0c40 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b99d40 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x352ea00:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=9&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=329, curl_handle=0x3fb31e0 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x352ea00 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b28080:op=0x4f8c000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x50efa80 req_data->id=329, curl_handle=0x3fb31e0 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=48&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=330, curl_handle=0x4bcfce0 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b28080 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x3f04000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[54]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x3f15e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[81]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x3b62a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[94]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b29980:op=0x4f8d200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x4df4400 req_data->id=330, curl_handle=0x4bcfce0 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=54&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=331, curl_handle=0x4bcfd00 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b29980 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b98d00:op=0x4f8c600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x3f93180 req_data->id=331, curl_handle=0x4bcfd00 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=81&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=332, curl_handle=0x4bd1b40 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b98d00 is io blocked 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99840:op=0x4f8e400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=94&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=333, curl_handle=0x4bd0b00 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b99840 is io blocked 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x50efcc0 req_data->id=332, curl_handle=0x4bd1b40 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x50d46c0 req_data->id=333, curl_handle=0x4bd0b00 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x3f17200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[110]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x4f8ea00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.901 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.901 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000308b-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.901 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.901 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.901 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.901 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.901 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.901 7f3b63568700 10 received header: 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=110&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=334, curl_handle=0x52cb0e0 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3ebcc80 is io blocked 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x50d6d00 req_data->id=334, curl_handle=0x52cb0e0 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x3ef5400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 RGW-SYNC:data:sync:shard[98]: read error repo, got 0 entries 2018-11-23 15:34:33.901 7f3b6456a700 20 cr:s=0x3b99d40:op=0x4f8f000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.901 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.901 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.901 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.901 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=98&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.901 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=335, curl_handle=0x52cb860 2018-11-23 15:34:33.901 7f3b6456a700 20 run: stack=0x3b99d40 is io blocked 2018-11-23 15:34:33.901 7f3b63568700 20 link_request req_data=0x3f246c0 req_data->id=335, curl_handle=0x52cb860 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000308c-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x4dd5200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x3f28a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x508d400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b29200 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b97540 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b97400 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x3ee1e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x3ef7200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b277c0 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b29340 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x3f35e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x3f00a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x3ee4a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x3eff200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99340 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b295c0 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99700 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b98080 is io blocked 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000308d-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x5100800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x508d400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[76]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x50e8800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97400:op=0x4f8f600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=76&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=336, curl_handle=0x52c9800 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b97400 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x3ee1e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[27]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b277c0:op=0x4f8de00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x3f92880 req_data->id=336, curl_handle=0x52c9800 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=27&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=337, curl_handle=0x3fb31e0 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b277c0 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x4dd5200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[33]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x352ea00:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29200:op=0x51c1800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x4cbe480 req_data->id=337, curl_handle=0x3fb31e0 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=33&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=338, curl_handle=0x52c9e20 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b29200 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x3f35e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[90]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003092-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x3f28a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[73]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x352ea00:op=0x38eb200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[9]: shard_id=9 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[9]: shard_id=9 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x352ea00 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x3ef7200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[30]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99340:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x50efa80 req_data->id=338, curl_handle=0x52c9e20 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=90&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=339, curl_handle=0x52c8b80 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99340 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b97540:op=0x51c1e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x4e6b440 req_data->id=339, curl_handle=0x52c8b80 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=73&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=340, curl_handle=0x52cabc0 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b97540 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29340:op=0x51c0c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000308e-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=30&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=341, curl_handle=0x52c9540 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b29340 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x3ee4a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[93]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x3eff200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[70]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99700:op=0x51c3600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x3bd3680 req_data->id=340, curl_handle=0x52cabc0 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x40e1d40 req_data->id=341, curl_handle=0x52c9540 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=93&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=342, curl_handle=0x52cb860 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99700 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98080:op=0x51c1200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003091-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=70&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=343, curl_handle=0x52c9860 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b98080 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x3f00a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[21]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99d40:op=0x4f8f000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b295c0:op=0x51c2a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=21&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x40e2d00 req_data->id=342, curl_handle=0x52cb860 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x3bd2d80 req_data->id=343, curl_handle=0x52c9860 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=344, curl_handle=0x52caae0 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b295c0 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99d40:op=0x3ef5400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[98]: shard_id=98 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[98]: shard_id=98 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99d40 is io blocked 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003090-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x3f246c0 req_data->id=344, curl_handle=0x52caae0 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b28080:op=0x4f8c000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b28080:op=0x3ef3200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[48]: shard_id=48 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[48]: shard_id=48 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b28080 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29980:op=0x4f8d200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003094-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x3eeb200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003093-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b29980:op=0x3f04000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[54]: shard_id=54 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[54]: shard_id=54 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b29980 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99480 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98d00:op=0x4f8c600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b98d00:op=0x3f15e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[81]: shard_id=81 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[81]: shard_id=81 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b98d00 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99840:op=0x4f8e400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x4f8ea00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99840:op=0x3b62a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[94]: shard_id=94 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[94]: shard_id=94 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99840 is io blocked 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3ebcc80:op=0x3f17200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[110]: shard_id=110 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[110]: shard_id=110 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3ebcc80 is io blocked 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003095-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.905 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.905 7f3b63568700 10 received header: 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x3eeb200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 RGW-SYNC:data:sync:shard[91]: read error repo, got 0 entries 2018-11-23 15:34:33.905 7f3b6456a700 20 cr:s=0x3b99480:op=0x4f8ea00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.905 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.905 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.905 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.905 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=91&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.905 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=345, curl_handle=0x52cb0e0 2018-11-23 15:34:33.905 7f3b6456a700 20 run: stack=0x3b99480 is io blocked 2018-11-23 15:34:33.905 7f3b63568700 20 link_request req_data=0x50d6d00 req_data->id=345, curl_handle=0x52cb0e0 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x508f200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 run: stack=0x3ebc3c0 is io blocked 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003098-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003099-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x508f200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[103]: read error repo, got 0 entries 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x4f8e400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.909 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.909 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000309a-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=103&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.909 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=346, curl_handle=0x4bd0b00 2018-11-23 15:34:33.909 7f3b6456a700 20 run: stack=0x3ebc3c0 is io blocked 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003097-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b63568700 20 link_request req_data=0x50d46c0 req_data->id=346, curl_handle=0x4bd0b00 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b277c0:op=0x4f8de00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b277c0:op=0x3ee1e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[27]: shard_id=27 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[27]: shard_id=27 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 run: stack=0x3b277c0 is io blocked 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b97540:op=0x51c1e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b97540:op=0x3f28a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[73]: shard_id=73 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[73]: shard_id=73 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 run: stack=0x3b97540 is io blocked 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000309b-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx000000000000000003096-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b29200:op=0x51c1800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b29200:op=0x4dd5200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[33]: shard_id=33 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[33]: shard_id=33 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 run: stack=0x3b29200 is io blocked 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000309c-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000308f-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b96280:op=0x5120000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 cr:s=0x3b96280:op=0x3edc000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[46]: shard_id=46 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 RGW-SYNC:data:sync:shard[46]: shard_id=46 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.909 7f3b6456a700 20 run: stack=0x3b96280 is io blocked 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000309d-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.909 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.909 7f3b63568700 10 received header: 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebca00:op=0x38caa00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebca00:op=0x3c8e000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[108]: shard_id=108 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[108]: shard_id=108 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebca00 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x3dc2c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b98f80:op=0x38cb600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b98300:op=0x38c9200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b99ac0:op=0x38e8000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[96]: shard_id=96 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[96]: shard_id=96 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3b99ac0 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b98f80:op=0x5090a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[79]: shard_id=79 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[79]: shard_id=79 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3b98f80 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b98300:op=0x3f16800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[71]: shard_id=71 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[71]: shard_id=71 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3b98300 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b98580:op=0x51c2400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b98580:op=0x3c8f400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[87]: shard_id=87 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[87]: shard_id=87 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3b98580 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x3c8ea00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x3c0ea00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebda40 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50b2800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x4df6a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50b0000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebcdc0 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x5092800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x508de00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebcf00 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebd040 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebdcc0 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebd900 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebc280 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50ea800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50b2800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[112]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x51c2400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=112&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=347, curl_handle=0x4085ae0 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebcf00 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x3c8ea00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[121]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x3c0ea00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[111]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebda40:op=0x38c9200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x3bd3680 req_data->id=347, curl_handle=0x4085ae0 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=121&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=348, curl_handle=0x5234e00 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebda40 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x38cb600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=111&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=349, curl_handle=0x3fdf580 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebcdc0 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x508de00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[102]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x4df6a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[113]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebc280:op=0x3dc2c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=102&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=350, curl_handle=0x3b2de80 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebc280 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd040:op=0x38caa00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=113&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x40e18c0 req_data->id=348, curl_handle=0x5234e00 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x3bd3b00 req_data->id=349, curl_handle=0x3fdf580 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x50d4900 req_data->id=350, curl_handle=0x3b2de80 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=351, curl_handle=0x4f98d80 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebd040 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50b0000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[123]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x5120000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=123&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=352, curl_handle=0x3fb0c40 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebdcc0 is io blocked 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x5092800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[120]: read error repo, got 0 entries 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3ebd900:op=0x51c1800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.913 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.913 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.913 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=120&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.913 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=353, curl_handle=0x52c9e20 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3ebd900 is io blocked 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x50d6640 req_data->id=351, curl_handle=0x4f98d80 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x50efa80 req_data->id=352, curl_handle=0x3fb0c40 2018-11-23 15:34:33.913 7f3b63568700 20 link_request req_data=0x4cf4400 req_data->id=353, curl_handle=0x52c9e20 2018-11-23 15:34:33.913 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.913 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.913 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.913 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000309f-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.913 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.913 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.913 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.913 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.913 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.913 7f3b63568700 10 received header: 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b29340:op=0x51c0c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 cr:s=0x3b29340:op=0x3ef7200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[30]: shard_id=30 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 RGW-SYNC:data:sync:shard[30]: shard_id=30 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.913 7f3b6456a700 20 run: stack=0x3b29340 is io blocked 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx00000000000000000309e-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3ebc780:op=0x3dc2600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3ebc780:op=0x5091e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[106]: shard_id=106 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[106]: shard_id=106 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.917 7f3b6456a700 20 run: stack=0x3ebc780 is io blocked 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b995c0:op=0x38c9e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b295c0:op=0x51c2a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b995c0:op=0x3c8fe00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[92]: shard_id=92 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[92]: shard_id=92 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.917 7f3b6456a700 20 run: stack=0x3b995c0 is io blocked 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b295c0:op=0x3f00a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[21]: shard_id=21 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[21]: shard_id=21 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.917 7f3b6456a700 20 run: stack=0x3b295c0 is io blocked 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x4dd4800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 run: stack=0x3b29e80 is io blocked 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a0-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a3-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x4dd4800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 RGW-SYNC:data:sync:shard[1]: read error repo, got 0 entries 2018-11-23 15:34:33.917 7f3b6456a700 20 cr:s=0x3b29e80:op=0x51c2a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.917 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.917 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.917 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.917 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.917 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=1&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.917 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=354, curl_handle=0x52caae0 2018-11-23 15:34:33.917 7f3b6456a700 20 run: stack=0x3b29e80 is io blocked 2018-11-23 15:34:33.917 7f3b63568700 20 link_request req_data=0x4cf4880 req_data->id=354, curl_handle=0x52caae0 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a2-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a6-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a1-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a4-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a5-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.917 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.917 7f3b63568700 10 received header: 2018-11-23 15:34:33.921 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.921 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.921 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.921 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a7-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.921 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.921 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.921 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.921 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.921 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.921 7f3b63568700 10 received header: 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3b99c00:op=0x4f8cc00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3b99c00:op=0x5091400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[97]: shard_id=97 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[97]: shard_id=97 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x3b99c00 is io blocked 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x3dc3200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3ebd2c0:op=0x50b0a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[115]: shard_id=115 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[115]: shard_id=115 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x3ebd2c0 is io blocked 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3b981c0:op=0x5120c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3b981c0:op=0x508c000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[75]: shard_id=75 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[75]: shard_id=75 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x3b981c0 is io blocked 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x4f8d800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3ebc000:op=0x5122400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3ebcb40:op=0x3c0e000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[109]: shard_id=109 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[109]: shard_id=109 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x3ebcb40 is io blocked 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x3ebc000:op=0x3f2a800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[100]: shard_id=100 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[100]: shard_id=100 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x3ebc000 is io blocked 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x3ee9e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x35306c0 is io blocked 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x3ee9e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 RGW-SYNC:data:sync:shard[16]: read error repo, got 0 entries 2018-11-23 15:34:33.921 7f3b6456a700 20 cr:s=0x35306c0:op=0x5122400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.921 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.921 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.921 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.921 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.921 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=16&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.921 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=355, curl_handle=0x3eeff60 2018-11-23 15:34:33.921 7f3b6456a700 20 run: stack=0x35306c0 is io blocked 2018-11-23 15:34:33.921 7f3b63568700 20 link_request req_data=0x381f3c0 req_data->id=355, curl_handle=0x3eeff60 2018-11-23 15:34:33.921 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.925 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.925 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.925 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a8-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.925 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.925 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.925 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.925 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.925 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.925 7f3b63568700 10 received header: 2018-11-23 15:34:33.925 7f3b6456a700 20 cr:s=0x352e500:op=0x51c0600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.925 7f3b6456a700 20 cr:s=0x352e500:op=0x3efca00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.925 7f3b6456a700 20 RGW-SYNC:data:sync:shard[20]: shard_id=20 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.925 7f3b6456a700 20 RGW-SYNC:data:sync:shard[20]: shard_id=20 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.925 7f3b6456a700 20 run: stack=0x352e500 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b29c00:op=0x51c3000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b29c00:op=0x3f0f200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[32]: shard_id=32 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[32]: shard_id=32 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b29c00 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x3f38a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b28440 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x3f36800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x3f22800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x38e9e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x3f33200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b28e40 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b27e00 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b27900 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3530940 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x3f36800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[35]: read error repo, got 0 entries 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28e40:op=0x51c3000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.929 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.929 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=35&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.929 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=356, curl_handle=0x52c8c80 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b28e40 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x3f38a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[50]: read error repo, got 0 entries 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b28440:op=0x51c0600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.929 7f3b63568700 20 link_request req_data=0x3f93600 req_data->id=356, curl_handle=0x52c8c80 2018-11-23 15:34:33.929 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.929 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.929 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=50&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.929 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=357, curl_handle=0x52cad80 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b28440 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x3f22800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[56]: read error repo, got 0 entries 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x38e9e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[42]: read error repo, got 0 entries 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27e00:op=0x4f8d800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.929 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.929 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=56&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.929 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=358, curl_handle=0x4bcfbe0 2018-11-23 15:34:33.929 7f3b63568700 20 link_request req_data=0x40e2880 req_data->id=357, curl_handle=0x52cad80 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b27e00 is io blocked 2018-11-23 15:34:33.929 7f3b63568700 20 link_request req_data=0x50ec6c0 req_data->id=358, curl_handle=0x4bcfbe0 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3b27900:op=0x5120c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.929 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.929 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=42&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.929 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=359, curl_handle=0x3fb2d20 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3b27900 is io blocked 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x3f33200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 RGW-SYNC:data:sync:shard[18]: read error repo, got 0 entries 2018-11-23 15:34:33.929 7f3b6456a700 20 cr:s=0x3530940:op=0x3dc3200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.929 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.929 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.929 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.929 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=18&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.929 7f3b63568700 20 link_request req_data=0x50ef840 req_data->id=359, curl_handle=0x3fb2d20 2018-11-23 15:34:33.929 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=360, curl_handle=0x3fb2320 2018-11-23 15:34:33.929 7f3b6456a700 20 run: stack=0x3530940 is io blocked 2018-11-23 15:34:33.929 7f3b63568700 20 link_request req_data=0x50ed680 req_data->id=360, curl_handle=0x3fb2320 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030a9-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030ad-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030ac-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030ab-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b990c0:op=0x5123000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b990c0:op=0x3f30a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[88]: shard_id=88 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[88]: shard_id=88 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3b990c0 is io blocked 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x35310c0:op=0x5121e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x35310c0:op=0x3f05400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[31]: shard_id=31 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[31]: shard_id=31 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x35310c0 is io blocked 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x3f04a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x3f37200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x3ee4000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030aa-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3b96a00 is io blocked 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3ebde00 is io blocked 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3b27f40 is io blocked 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x3f04a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[57]: read error repo, got 0 entries 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b96a00:op=0x5121e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.933 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.933 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.933 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.933 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=57&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.933 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=361, curl_handle=0x3b2ada0 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3b96a00 is io blocked 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x3ee4000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[49]: read error repo, got 0 entries 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x3f37200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 RGW-SYNC:data:sync:shard[124]: read error repo, got 0 entries 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3b27f40:op=0x5123000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.933 7f3b63568700 20 link_request req_data=0x3f246c0 req_data->id=361, curl_handle=0x3b2ada0 2018-11-23 15:34:33.933 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.933 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.933 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.933 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.933 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=49&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.933 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=362, curl_handle=0x4f99d20 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3b27f40 is io blocked 2018-11-23 15:34:33.933 7f3b6456a700 20 cr:s=0x3ebde00:op=0x4f8cc00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.933 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.933 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.933 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.933 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.933 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=124&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.933 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=363, curl_handle=0x4085c60 2018-11-23 15:34:33.933 7f3b6456a700 20 run: stack=0x3ebde00 is io blocked 2018-11-23 15:34:33.933 7f3b63568700 20 link_request req_data=0x4df53c0 req_data->id=362, curl_handle=0x4f99d20 2018-11-23 15:34:33.933 7f3b63568700 20 link_request req_data=0x5158000 req_data->id=363, curl_handle=0x4085c60 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030af-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030ae-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b0-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.933 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.933 7f3b63568700 10 received header: 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b98800:op=0x5120600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b98800:op=0x3f07200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[85]: shard_id=85 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[85]: shard_id=85 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b98800 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96c80:op=0x5123600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96c80:op=0x3ef1400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[60]: shard_id=60 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[60]: shard_id=60 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b96c80 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b29d40:op=0x5121200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b29d40:op=0x3ee5e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[39]: shard_id=39 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[39]: shard_id=39 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b29d40 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x3f06800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x3ee5400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x4dd2a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b28d00 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3530080 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b96500 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x3f06800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[52]: read error repo, got 0 entries 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b28d00:op=0x5121200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.937 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.937 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.937 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.937 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=52&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.937 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=364, curl_handle=0x3fb2ec0 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b28d00 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x3ee5400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b63568700 20 link_request req_data=0x4cf3b00 req_data->id=364, curl_handle=0x3fb2ec0 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[2]: read error repo, got 0 entries 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x4dd2a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 RGW-SYNC:data:sync:shard[59]: read error repo, got 0 entries 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3530080:op=0x5123600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.937 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.937 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.937 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.937 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=2&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.937 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=365, curl_handle=0x4bd1440 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3530080 is io blocked 2018-11-23 15:34:33.937 7f3b6456a700 20 cr:s=0x3b96500:op=0x5120600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.937 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.937 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.937 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.937 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.937 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=59&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.937 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=366, curl_handle=0x3fb3940 2018-11-23 15:34:33.937 7f3b6456a700 20 run: stack=0x3b96500 is io blocked 2018-11-23 15:34:33.937 7f3b63568700 20 link_request req_data=0x4cf41c0 req_data->id=365, curl_handle=0x4bd1440 2018-11-23 15:34:33.937 7f3b63568700 20 link_request req_data=0x3f92d00 req_data->id=366, curl_handle=0x3fb3940 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b1-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header: 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b3-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header: 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b2-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.937 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.937 7f3b63568700 10 received header: 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b28940:op=0x5122a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b28940:op=0x3edf200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[53]: shard_id=53 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[53]: shard_id=53 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b28940 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x3f34a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b27540 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x508e800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x3eea800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x4b5b400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x3f1e800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b972c0 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b97e00 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b29ac0 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b97900 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x4b5b400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[51]: read error repo, got 0 entries 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x5122a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.941 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.941 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=51&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.941 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=367, curl_handle=0x3eefa20 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b29ac0 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x3f34a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[41]: read error repo, got 0 entries 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b27540:op=0x38c9e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.941 7f3b63568700 20 link_request req_data=0x4cbe480 req_data->id=367, curl_handle=0x3eefa20 2018-11-23 15:34:33.941 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.941 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.941 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=41&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.941 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=368, curl_handle=0x52cb620 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b27540 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x3f1e800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[69]: read error repo, got 0 entries 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97900:op=0x3dc2600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.941 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.941 7f3b63568700 20 link_request req_data=0x40e1d40 req_data->id=368, curl_handle=0x52cb620 2018-11-23 15:34:33.941 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=69&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.941 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=369, curl_handle=0x3fdec80 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b97900 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x508e800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[68]: read error repo, got 0 entries 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x3eea800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 RGW-SYNC:data:sync:shard[67]: read error repo, got 0 entries 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b972c0:op=0x51c0c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.941 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.941 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.941 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=68&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.941 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=370, curl_handle=0x52c9540 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b972c0 is io blocked 2018-11-23 15:34:33.941 7f3b6456a700 20 cr:s=0x3b97e00:op=0x51c1e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.941 7f3b63568700 20 link_request req_data=0x50d5440 req_data->id=369, curl_handle=0x3fdec80 2018-11-23 15:34:33.941 7f3b63568700 20 link_request req_data=0x3f93a80 req_data->id=370, curl_handle=0x52c9540 2018-11-23 15:34:33.941 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.941 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.941 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.941 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=67&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.941 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=371, curl_handle=0x52cabc0 2018-11-23 15:34:33.941 7f3b6456a700 20 run: stack=0x3b97e00 is io blocked 2018-11-23 15:34:33.941 7f3b63568700 20 link_request req_data=0x4df3f80 req_data->id=371, curl_handle=0x52cabc0 2018-11-23 15:34:33.941 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.941 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.941 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.941 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b6-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.941 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.941 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.941 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.941 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.941 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.941 7f3b63568700 10 received header: 2018-11-23 15:34:33.941 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b4-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header: 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b5-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header: 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b7-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header: 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x51c0000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b98bc0:op=0x3ee8a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 RGW-SYNC:data:sync:shard[82]: shard_id=82 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.945 7f3b6456a700 20 RGW-SYNC:data:sync:shard[82]: shard_id=82 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.945 7f3b6456a700 20 run: stack=0x3b98bc0 is io blocked 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b97400:op=0x4f8f600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b97400:op=0x508d400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 RGW-SYNC:data:sync:shard[76]: shard_id=76 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.945 7f3b6456a700 20 RGW-SYNC:data:sync:shard[76]: shard_id=76 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.945 7f3b6456a700 20 run: stack=0x3b97400 is io blocked 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b8-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header: 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x508ca00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 run: stack=0x3b986c0 is io blocked 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x508ca00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 RGW-SYNC:data:sync:shard[86]: read error repo, got 0 entries 2018-11-23 15:34:33.945 7f3b6456a700 20 cr:s=0x3b986c0:op=0x4f8f600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.945 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.945 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.945 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.945 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.945 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=86&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.945 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=372, curl_handle=0x52c9800 2018-11-23 15:34:33.945 7f3b6456a700 20 run: stack=0x3b986c0 is io blocked 2018-11-23 15:34:33.945 7f3b63568700 20 link_request req_data=0x3881440 req_data->id=372, curl_handle=0x52c9800 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030b9-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.945 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.945 7f3b63568700 10 received header: 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99480:op=0x4f8ea00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99480:op=0x3eeb200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[91]: shard_id=91 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[91]: shard_id=91 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b99480 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x4f8e400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x3f15400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50b1400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99340:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebc3c0:op=0x508f200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[103]: shard_id=103 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[103]: shard_id=103 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebc3c0 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99700:op=0x51c3600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b98080:op=0x51c1200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99340:op=0x3f35e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[90]: shard_id=90 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[90]: shard_id=90 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b99340 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x3c10800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x50b3200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99700:op=0x3ee4a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[93]: shard_id=93 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[93]: shard_id=93 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b99700 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b98080:op=0x3eff200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[70]: shard_id=70 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[70]: shard_id=70 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b98080 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b99e80 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebdf40 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x3ef0000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebd680 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebd400 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b27cc0 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x3c10800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[118]: read error repo, got 0 entries 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd680:op=0x51c1200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.949 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.949 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=118&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.949 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=373, curl_handle=0x52c9860 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebd680 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x3f15400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[99]: read error repo, got 0 entries 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b99e80:op=0x51c3600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b63568700 20 link_request req_data=0x3f92880 req_data->id=373, curl_handle=0x52c9860 2018-11-23 15:34:33.949 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.949 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.949 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=99&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.949 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=374, curl_handle=0x52cb860 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b99e80 is io blocked 2018-11-23 15:34:33.949 7f3b63568700 20 link_request req_data=0x3bd2d80 req_data->id=374, curl_handle=0x52cb860 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50b1400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[125]: read error repo, got 0 entries 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.949 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.949 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=125&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.949 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=375, curl_handle=0x52c8b80 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebdf40 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.949 7f3b63568700 20 link_request req_data=0x50d46c0 req_data->id=375, curl_handle=0x52c8b80 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x50b3200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[116]: read error repo, got 0 entries 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x3ef0000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 RGW-SYNC:data:sync:shard[40]: read error repo, got 0 entries 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3ebd400:op=0x4f8e400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.949 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.949 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=116&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.949 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=376, curl_handle=0x4bd0b00 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3ebd400 is io blocked 2018-11-23 15:34:33.949 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x4f8ea00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.949 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.949 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.949 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.949 7f3b63568700 20 link_request req_data=0x50d6d00 req_data->id=376, curl_handle=0x4bd0b00 2018-11-23 15:34:33.949 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.949 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=40&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.949 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=377, curl_handle=0x52cb0e0 2018-11-23 15:34:33.949 7f3b6456a700 20 run: stack=0x3b27cc0 is io blocked 2018-11-23 15:34:33.949 7f3b63568700 20 link_request req_data=0x4e6b440 req_data->id=377, curl_handle=0x52cb0e0 2018-11-23 15:34:33.949 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.949 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.949 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.949 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030ba-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.949 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.949 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.949 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.949 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.949 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.949 7f3b63568700 10 received header: 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030be-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header: 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x3c0f400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebe1c0 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x4df7400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x4df6000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x5093200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x3c91200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebd180 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebc500 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebc8c0 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebc640 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x3c11200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030bb-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header: 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebdb80 is io blocked 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030bc-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header: 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030bd-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.953 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.953 7f3b63568700 10 received header: 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x50eb800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x3c91200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 RGW-SYNC:data:sync:shard[105]: read error repo, got 0 entries 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x50e6000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc640:op=0x51c0000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.953 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.953 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=105&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.953 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=378, curl_handle=0x3eeee20 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebc640 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x4df6000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 RGW-SYNC:data:sync:shard[104]: read error repo, got 0 entries 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc500:op=0x4f8de00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.953 7f3b63568700 20 link_request req_data=0x50d7600 req_data->id=378, curl_handle=0x3eeee20 2018-11-23 15:34:33.953 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.953 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=104&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.953 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=379, curl_handle=0x3fb31e0 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebc500 is io blocked 2018-11-23 15:34:33.953 7f3b63568700 20 link_request req_data=0x3f93180 req_data->id=379, curl_handle=0x3fb31e0 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x50fd800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x3c0f400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 RGW-SYNC:data:sync:shard[127]: read error repo, got 0 entries 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x4f8c600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.953 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.953 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=127&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.953 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=380, curl_handle=0x4bd1b40 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebe1c0 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x5101800:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x4df7400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b63568700 20 link_request req_data=0x4df4400 req_data->id=380, curl_handle=0x4bd1b40 2018-11-23 15:34:33.953 7f3b6456a700 20 RGW-SYNC:data:sync:shard[114]: read error repo, got 0 entries 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebd180:op=0x4f8d200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.953 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.953 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=114&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.953 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=381, curl_handle=0x4bcfd00 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebd180 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x50fc000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x5103000:21RGWRadosGetOmapKeysCR: operate() returned r=-2 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x5093200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 RGW-SYNC:data:sync:shard[107]: read error repo, got 0 entries 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x3c11200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 RGW-SYNC:data:sync:shard[122]: read error repo, got 0 entries 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x4f8c000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.953 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.953 7f3b63568700 20 link_request req_data=0x40e2d00 req_data->id=381, curl_handle=0x4bcfd00 2018-11-23 15:34:33.953 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.953 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=107&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.953 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=382, curl_handle=0x4bcfce0 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebc8c0 is io blocked 2018-11-23 15:34:33.953 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x4f8f000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.953 7f3b6456a700 20 > HTTP_DATE -> Fri, 23 Nov 2018 08:34:33 +0000 2018-11-23 15:34:33.953 7f3b6456a700 10 get_canon_resource(): dest=/admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 10 generated canonical header: GET Fri, 23 Nov 2018 08:34:33 +0000 /admin/log/ 2018-11-23 15:34:33.953 7f3b6456a700 15 generated auth header: AWS CCO3A2ABT5S2HCJ1T9QL:rhSEl5Z5aePC8M0cTSAyELAxPtM= 2018-11-23 15:34:33.953 7f3b6456a700 20 sending request to http://10.5.8.118:80/admin/log/?type=data&id=122&info&rgwx-zonegroup=c6658002-6d73-4b66-afbe-41710f34840b 2018-11-23 15:34:33.953 7f3b6456a700 20 register_request mgr=0x380d630 req_data->id=383, curl_handle=0x52cbe80 2018-11-23 15:34:33.953 7f3b6456a700 20 run: stack=0x3ebdb80 is io blocked 2018-11-23 15:34:33.953 7f3b63568700 20 link_request req_data=0x50d5b00 req_data->id=382, curl_handle=0x4bcfce0 2018-11-23 15:34:33.953 7f3b63568700 20 link_request req_data=0x4cf2900 req_data->id=383, curl_handle=0x52cbe80 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030c0-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header: 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x38cb600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebcdc0:op=0x3c0ea00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[111]: shard_id=111 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[111]: shard_id=111 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebcdc0 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebda40:op=0x38c9200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebda40:op=0x3c8ea00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[121]: shard_id=121 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[121]: shard_id=121 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebda40 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebc280:op=0x3dc2c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebd040:op=0x38caa00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030bf-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header: 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x5120000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebc280:op=0x508de00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[102]: shard_id=102 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[102]: shard_id=102 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebc280 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebd040:op=0x4df6a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[113]: shard_id=113 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[113]: shard_id=113 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebd040 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebdcc0:op=0x50b0000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[123]: shard_id=123 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[123]: shard_id=123 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebdcc0 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x51c2400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebd900:op=0x51c1800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebcf00:op=0x50b2800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[112]: shard_id=112 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[112]: shard_id=112 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebcf00 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebd900:op=0x5092800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[120]: shard_id=120 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[120]: shard_id=120 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebd900 is io blocked 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030c1-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header: 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebc500:op=0x4f8de00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebc500:op=0x4df6000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[104]: shard_id=104 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[104]: shard_id=104 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebc500 is io blocked 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030c3-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header: 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x4f8c000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebc8c0:op=0x5093200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[107]: shard_id=107 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[107]: shard_id=107 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebc8c0 is io blocked 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030c2-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header: 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:HTTP/1.1 200 OK 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:x-amz-request-id: tx0000000000000000030c4-005bf7bb99-1a0f7-hn-1 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Content-Length: 38 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header:Date: Fri, 23 Nov 2018 08:34:33 GMT 2018-11-23 15:34:33.957 7f3b63568700 10 receive_http_header 2018-11-23 15:34:33.957 7f3b63568700 10 received header: 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebd180:op=0x4f8d200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebd180:op=0x4df7400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[114]: shard_id=114 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[114]: shard_id=114 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebd180 is io blocked 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x4f8f000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 cr:s=0x3ebdb80:op=0x3c11200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[122]: shard_id=122 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 RGW-SYNC:data:sync:shard[122]: shard_id=122 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.957 7f3b6456a700 20 run: stack=0x3ebdb80 is io blocked 2018-11-23 15:34:33.961 7f3b6456a700 20 cr:s=0x3b29e80:op=0x51c2a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.961 7f3b6456a700 20 cr:s=0x3b29e80:op=0x4dd4800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.961 7f3b6456a700 20 RGW-SYNC:data:sync:shard[1]: shard_id=1 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.961 7f3b6456a700 20 RGW-SYNC:data:sync:shard[1]: shard_id=1 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.961 7f3b6456a700 20 run: stack=0x3b29e80 is io blocked 2018-11-23 15:34:33.961 7f3b6456a700 20 cr:s=0x35306c0:op=0x5122400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.961 7f3b6456a700 20 cr:s=0x35306c0:op=0x3ee9e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.961 7f3b6456a700 20 RGW-SYNC:data:sync:shard[16]: shard_id=16 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.961 7f3b6456a700 20 RGW-SYNC:data:sync:shard[16]: shard_id=16 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.961 7f3b6456a700 20 run: stack=0x35306c0 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b27e00:op=0x4f8d800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b27e00:op=0x3f22800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[56]: shard_id=56 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[56]: shard_id=56 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3b27e00 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3530940:op=0x3dc3200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b96a00:op=0x5121e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b27f40:op=0x5123000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3ebde00:op=0x4f8cc00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3530940:op=0x3f33200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[18]: shard_id=18 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[18]: shard_id=18 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3530940 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b28e40:op=0x51c3000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b28440:op=0x51c0600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b27900:op=0x5120c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b96a00:op=0x3f04a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[57]: shard_id=57 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[57]: shard_id=57 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3b96a00 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b27f40:op=0x3ee4000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[49]: shard_id=49 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[49]: shard_id=49 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3b27f40 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3ebde00:op=0x3f37200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[124]: shard_id=124 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[124]: shard_id=124 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3ebde00 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b28e40:op=0x3f36800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[35]: shard_id=35 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[35]: shard_id=35 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3b28e40 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b28440:op=0x3f38a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[50]: shard_id=50 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[50]: shard_id=50 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3b28440 is io blocked 2018-11-23 15:34:33.973 7f3b6456a700 20 cr:s=0x3b27900:op=0x38e9e00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[42]: shard_id=42 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 RGW-SYNC:data:sync:shard[42]: shard_id=42 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.973 7f3b6456a700 20 run: stack=0x3b27900 is io blocked 2018-11-23 15:34:33.977 7f3b6456a700 20 cr:s=0x3b96500:op=0x5120600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.977 7f3b6456a700 20 cr:s=0x3b96500:op=0x4dd2a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.977 7f3b6456a700 20 RGW-SYNC:data:sync:shard[59]: shard_id=59 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.977 7f3b6456a700 20 RGW-SYNC:data:sync:shard[59]: shard_id=59 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.977 7f3b6456a700 20 run: stack=0x3b96500 is io blocked 2018-11-23 15:34:33.977 7f3b6456a700 20 cr:s=0x3b28d00:op=0x5121200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.977 7f3b6456a700 20 cr:s=0x3b28d00:op=0x3f06800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.977 7f3b6456a700 20 RGW-SYNC:data:sync:shard[52]: shard_id=52 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.977 7f3b6456a700 20 RGW-SYNC:data:sync:shard[52]: shard_id=52 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.977 7f3b6456a700 20 run: stack=0x3b28d00 is io blocked 2018-11-23 15:34:33.977 7f3b6456a700 20 cr:s=0x3530080:op=0x5123600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.977 7f3b6456a700 20 cr:s=0x3530080:op=0x3ee5400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.977 7f3b6456a700 20 RGW-SYNC:data:sync:shard[2]: shard_id=2 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.977 7f3b6456a700 20 RGW-SYNC:data:sync:shard[2]: shard_id=2 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.977 7f3b6456a700 20 run: stack=0x3530080 is io blocked 2018-11-23 15:34:33.981 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x5122a00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.981 7f3b6456a700 20 cr:s=0x3b29ac0:op=0x4b5b400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.981 7f3b6456a700 20 RGW-SYNC:data:sync:shard[51]: shard_id=51 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.981 7f3b6456a700 20 RGW-SYNC:data:sync:shard[51]: shard_id=51 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.981 7f3b6456a700 20 run: stack=0x3b29ac0 is io blocked 2018-11-23 15:34:33.981 7f3b6456a700 20 cr:s=0x3b97900:op=0x3dc2600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.981 7f3b6456a700 20 cr:s=0x3b97900:op=0x3f1e800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.981 7f3b6456a700 20 RGW-SYNC:data:sync:shard[69]: shard_id=69 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.981 7f3b6456a700 20 RGW-SYNC:data:sync:shard[69]: shard_id=69 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.981 7f3b6456a700 20 run: stack=0x3b97900 is io blocked 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b986c0:op=0x4f8f600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b986c0:op=0x508ca00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[86]: shard_id=86 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[86]: shard_id=86 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 run: stack=0x3b986c0 is io blocked 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b27540:op=0x38c9e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b972c0:op=0x51c0c00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b97e00:op=0x51c1e00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b27540:op=0x3f34a00:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[41]: shard_id=41 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[41]: shard_id=41 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 run: stack=0x3b27540 is io blocked 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b972c0:op=0x508e800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[68]: shard_id=68 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[68]: shard_id=68 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 run: stack=0x3b972c0 is io blocked 2018-11-23 15:34:33.985 7f3b6456a700 20 cr:s=0x3b97e00:op=0x3eea800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[67]: shard_id=67 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 RGW-SYNC:data:sync:shard[67]: shard_id=67 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.985 7f3b6456a700 20 run: stack=0x3b97e00 is io blocked 2018-11-23 15:34:33.989 7f3b6456a700 20 cr:s=0x3ebd680:op=0x51c1200:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.989 7f3b6456a700 20 cr:s=0x3ebd680:op=0x3c10800:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.989 7f3b6456a700 20 RGW-SYNC:data:sync:shard[118]: shard_id=118 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.989 7f3b6456a700 20 RGW-SYNC:data:sync:shard[118]: shard_id=118 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.989 7f3b6456a700 20 run: stack=0x3ebd680 is io blocked 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3b99e80:op=0x51c3600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3b99e80:op=0x3f15400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[99]: shard_id=99 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[99]: shard_id=99 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 run: stack=0x3b99e80 is io blocked 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x5121800:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3ebd400:op=0x4f8e400:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x4f8ea00:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3ebdf40:op=0x50b1400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[125]: shard_id=125 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[125]: shard_id=125 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 run: stack=0x3ebdf40 is io blocked 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3ebd400:op=0x50b3200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[116]: shard_id=116 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[116]: shard_id=116 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 run: stack=0x3ebd400 is io blocked 2018-11-23 15:34:33.993 7f3b6456a700 20 cr:s=0x3b27cc0:op=0x3ef0000:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[40]: shard_id=40 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 RGW-SYNC:data:sync:shard[40]: shard_id=40 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.993 7f3b6456a700 20 run: stack=0x3b27cc0 is io blocked 2018-11-23 15:34:33.997 7f3b6456a700 20 cr:s=0x3ebc640:op=0x51c0000:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.997 7f3b6456a700 20 cr:s=0x3ebc640:op=0x3c91200:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.997 7f3b6456a700 20 RGW-SYNC:data:sync:shard[105]: shard_id=105 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.997 7f3b6456a700 20 RGW-SYNC:data:sync:shard[105]: shard_id=105 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.997 7f3b6456a700 20 run: stack=0x3ebc640 is io blocked 2018-11-23 15:34:33.997 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x4f8c600:31RGWReadRemoteDataLogShardInfoCR: operate() 2018-11-23 15:34:33.997 7f3b6456a700 20 cr:s=0x3ebe1c0:op=0x3c0f400:18RGWDataSyncShardCR: operate() 2018-11-23 15:34:33.997 7f3b6456a700 20 RGW-SYNC:data:sync:shard[127]: shard_id=127 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.997 7f3b6456a700 20 RGW-SYNC:data:sync:shard[127]: shard_id=127 datalog_marker= sync_marker.marker= 2018-11-23 15:34:33.997 7f3b6456a700 20 run: stack=0x3ebe1c0 is io blocked 2018-11-23 15:34:34.409 7f3b78d93700 2 RGWDataChangesLog::ChangesRenewThread: start