Ceph » rgw

2015-07-01 10:24:54.311787 7ff95edb57c0  0 ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047), process radosgw, pid 26980

2015-07-01 10:24:54.793573 7ff95edb57c0  0 framework: fastcgi

2015-07-01 10:24:54.793693 7ff95edb57c0  0 starting handler: fastcgi

2015-07-01 10:24:54.815589 7ff92effd700  0 ERROR: can't read user header: ret=-2

2015-07-01 10:24:54.815604 7ff92effd700  0 ERROR: sync_user() failed, user=tw-west ret=-2

2015-07-01 10:25:22.237515 7ff927fef700  1 ====== starting new request req=0x7ff94000bfd0 =====

2015-07-01 10:25:22.243321 7ff927fef700  1 ====== req done req=0x7ff94000bfd0 http_status=200 ======

2015-07-01 10:34:38.656534 7ff9147c8700  1 ====== starting new request req=0x7ff94006a620 =====

2015-07-01 10:34:38.660966 7ff9147c8700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:39.629302 7ff9147c8700  0 > HTTP_DATE -> Wed Jul  1 02:34:39 2015

2015-07-01 10:34:39.629304 7ff9147c8700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:39.733257 7ff9147c8700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:39.733353 7ff9147c8700  1 ====== req done req=0x7ff94006a620 http_status=500 ======

2015-07-01 10:34:40.291928 7ff8fff9f700  1 ====== starting new request req=0x7ff94005b9e0 =====

2015-07-01 10:34:40.292565 7ff8fff9f700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:40.304926 7ff8fff9f700  0 > HTTP_DATE -> Wed Jul  1 02:34:40 2015

2015-07-01 10:34:40.304928 7ff8fff9f700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:40.317349 7ff8fff9f700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:40.317401 7ff8fff9f700  1 ====== req done req=0x7ff94005b9e0 http_status=500 ======

2015-07-01 10:34:41.903723 7ff9087b0700  1 ====== starting new request req=0x7ff940034db0 =====

2015-07-01 10:34:41.904485 7ff9087b0700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:41.922114 7ff9087b0700  0 > HTTP_DATE -> Wed Jul  1 02:34:41 2015

2015-07-01 10:34:41.922118 7ff9087b0700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:41.931607 7ff9087b0700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:41.931678 7ff9087b0700  1 ====== req done req=0x7ff940034db0 http_status=500 ======

2015-07-01 10:34:44.485520 7ff902fa5700  1 ====== starting new request req=0x7ff94000dde0 =====

2015-07-01 10:34:44.486038 7ff902fa5700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:44.508509 7ff902fa5700  0 > HTTP_DATE -> Wed Jul  1 02:34:44 2015

2015-07-01 10:34:44.508511 7ff902fa5700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:44.549616 7ff902fa5700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:44.549777 7ff902fa5700  1 ====== req done req=0x7ff94000dde0 http_status=500 ======

2015-07-01 10:34:46.394077 7ff918fd1700  1 ====== starting new request req=0x7ff9400397b0 =====

2015-07-01 10:34:46.398290 7ff918fd1700  1 ====== req done req=0x7ff9400397b0 http_status=200 ======

2015-07-01 10:34:46.404346 7ff9027a4700  1 ====== starting new request req=0x7ff940027840 =====

2015-07-01 10:34:46.404853 7ff9027a4700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:46.482770 7ff9027a4700  0 > HTTP_DATE -> Wed Jul  1 02:34:46 2015

2015-07-01 10:34:46.482771 7ff9027a4700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:46.552883 7ff9027a4700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:46.552971 7ff9027a4700  1 ====== req done req=0x7ff940027840 http_status=500 ======

2015-07-01 10:34:47.550229 7ff90d7ba700  1 ====== starting new request req=0x7ff940015950 =====

2015-07-01 10:34:47.550688 7ff90d7ba700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:47.558518 7ff90d7ba700  0 > HTTP_DATE -> Wed Jul  1 02:34:47 2015

2015-07-01 10:34:47.558520 7ff90d7ba700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:47.566081 7ff90d7ba700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:47.566126 7ff90d7ba700  1 ====== req done req=0x7ff940015950 http_status=500 ======

2015-07-01 10:34:48.932688 7ff913fc7700  1 ====== starting new request req=0x7ff94001c370 =====

2015-07-01 10:34:48.933439 7ff913fc7700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:48.946175 7ff913fc7700  0 > HTTP_DATE -> Wed Jul  1 02:34:48 2015

2015-07-01 10:34:48.946179 7ff913fc7700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:48.956077 7ff913fc7700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:48.956261 7ff913fc7700  1 ====== req done req=0x7ff94001c370 http_status=500 ======

2015-07-01 10:34:51.904977 7ff929ff3700  1 ====== starting new request req=0x7ff94005b9e0 =====

2015-07-01 10:34:51.905556 7ff929ff3700  0 WARNING: couldn't find acl header for bucket, generating default

2015-07-01 10:34:51.922573 7ff929ff3700  0 > HTTP_DATE -> Wed Jul  1 02:34:51 2015

2015-07-01 10:34:51.922575 7ff929ff3700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

2015-07-01 10:34:51.932717 7ff929ff3700  0 WARNING: set_req_state_err err_no=5 resorting to 500

2015-07-01 10:34:51.932838 7ff929ff3700  1 ====== req done req=0x7ff94005b9e0 http_status=500 ======

2015-07-01 10:34:54.096336 7ff91c7d8700  1 ====== starting new request req=0x7ff94000dde0 =====

2015-07-01 10:34:54.099972 7ff91c7d8700  1 ====== req done req=0x7ff94000dde0 http_status=200 ======

2015-07-01 10:34:54.112960 7ff91b7d6700  1 ====== starting new request req=0x7ff940034db0 =====

2015-07-01 10:34:54.116891 7ff91b7d6700 -1 *** Caught signal (Aborted) **

 in thread 7ff91b7d6700

 ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047)

 1: (()+0x1f18af) [0x7ff95efcb8af]

 2: (()+0x10340) [0x7ff95cfcd340]

 3: (gsignal()+0x39) [0x7ff95c0d4cc9]

 4: (abort()+0x148) [0x7ff95c0d80d8]

 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff95c9df535]

 6: (()+0x5e6d6) [0x7ff95c9dd6d6]

 7: (()+0x5e703) [0x7ff95c9dd703]

 8: (()+0x5e922) [0x7ff95c9dd922]

 9: (bool JSONDecoder::decode_json<utime_t>(char const*, utime_t&, JSONObj*, bool)+0x204) [0x7ff95eed35b4]

 10: (void decode_json_obj<cls_replica_log_item_marker>(std::list<cls_replica_log_item_marker, std::allocator<cls_replica_log_item_marker> >&, JSONObj*)+0x73) [0x7ff95eea9e13]

 11: (int rgw_rest_get_json_input<std::list<cls_replica_log_item_marker, std::allocator<cls_replica_log_item_marker> > >(CephContext*, req_state*, std::list<cls_replica_log_item_marker, std::allocator<cls_replica_log_item_marker> >&, int, bool*)+0xa5) [0x7ff95eea9f75]

 12: (RGWOp_BILog_SetBounds::execute()+0x390) [0x7ff95eea8e20]

 13: (()+0xde63d) [0x7ff95eeb863d]

 14: (RGWFCGXProcess::handle_request(RGWRequest*)+0x80) [0x7ff95eeb8d30]

 15: (RGWProcess::RGWWQ::_process(RGWRequest*)+0x30) [0x7ff95eeba340]

 16: (ThreadPool::worker(ThreadPool::WorkThread*)+0xaf1) [0x7ff95deca531]

 17: (ThreadPool::WorkThread::entry()+0x10) [0x7ff95decb420]

 18: (()+0x8182) [0x7ff95cfc5182]

 19: (clone()+0x6d) [0x7ff95c19847d]

 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---

 -9999> 2015-07-01 10:33:49.893762 7ff9067ac700  2 req 6306:0.000109::POST /admin/log:unlock_datalog_object:authorizing

 -9998> 2015-07-01 10:33:49.893948 7ff9067ac700  2 req 6306:0.000294::POST /admin/log:unlock_datalog_object:reading permissions

 -9997> 2015-07-01 10:33:49.893954 7ff9067ac700  2 req 6306:0.000301::POST /admin/log:unlock_datalog_object:init op

 -9996> 2015-07-01 10:33:49.893957 7ff9067ac700  2 req 6306:0.000303::POST /admin/log:unlock_datalog_object:verifying op mask

 -9995> 2015-07-01 10:33:49.893959 7ff9067ac700  2 req 6306:0.000306::POST /admin/log:unlock_datalog_object:verifying op permissions

 -9994> 2015-07-01 10:33:49.893962 7ff9067ac700  2 overriding permissions due to system operation

 -9993> 2015-07-01 10:33:49.893963 7ff9067ac700  2 req 6306:0.000310::POST /admin/log:unlock_datalog_object:verifying op params

 -9992> 2015-07-01 10:33:49.893965 7ff9067ac700  2 req 6306:0.000312::POST /admin/log:unlock_datalog_object:executing

 -9991> 2015-07-01 10:33:49.894001 7ff9067ac700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:6478 data_log.106 [call lock.unlock] 59.d2dd3d41 ondisk+write e425) v4 -- ?+0 0x7ff950021c70 con 0x7ff95f92e520

 -9990> 2015-07-01 10:33:49.902968 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2522 ==== osd_op_reply(6478 data_log.106 [call] v425'3938 uv3938 ondisk = 0) v6 ==== 179+0+0 (2424285253 0 0) 0x7ff9380101b0 con 0x7ff95f92e520

 -9989> 2015-07-01 10:33:49.903068 7ff9067ac700  2 req 6306:0.009414::POST /admin/log:unlock_datalog_object:http status=200

 -9988> 2015-07-01 10:33:49.903076 7ff9067ac700  1 ====== req done req=0x7ff940028200 http_status=200 ======

  -307> 2015-07-01 10:34:38.588833 7ff91efdd700  1 ====== starting new request req=0x7ff94000dde0 =====

  -306> 2015-07-01 10:34:38.588850 7ff91efdd700  2 req 6949:0.000017::GET /admin/replica_log::initializing

  -305> 2015-07-01 10:34:38.588877 7ff91efdd700  2 req 6949:0.000045::GET /admin/replica_log::getting op

  -304> 2015-07-01 10:34:38.588880 7ff91efdd700  2 req 6949:0.000048::GET /admin/replica_log:replicabilog_getbounds:authorizing

  -303> 2015-07-01 10:34:38.588998 7ff91efdd700  2 req 6949:0.000166::GET /admin/replica_log:replicabilog_getbounds:reading permissions

  -302> 2015-07-01 10:34:38.589003 7ff91efdd700  2 req 6949:0.000171::GET /admin/replica_log:replicabilog_getbounds:init op

  -301> 2015-07-01 10:34:38.589005 7ff91efdd700  2 req 6949:0.000172::GET /admin/replica_log:replicabilog_getbounds:verifying op mask

  -300> 2015-07-01 10:34:38.589006 7ff91efdd700  2 req 6949:0.000174::GET /admin/replica_log:replicabilog_getbounds:verifying op permissions

  -299> 2015-07-01 10:34:38.589008 7ff91efdd700  2 overriding permissions due to system operation

  -298> 2015-07-01 10:34:38.589009 7ff91efdd700  2 req 6949:0.000177::GET /admin/replica_log:replicabilog_getbounds:verifying op params

  -297> 2015-07-01 10:34:38.589011 7ff91efdd700  2 req 6949:0.000179::GET /admin/replica_log:replicabilog_getbounds:executing

  -296> 2015-07-01 10:34:38.589071 7ff91efdd700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7121 replica_log.BASKET1 [call replica_log.get] 59.933a1b8d ack+read e425) v4 -- ?+0 0x7ff93c042930 con 0x7ff95f9302e0

  -295> 2015-07-01 10:34:38.590105 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1797 ==== osd_op_reply(7121 replica_log.BASKET1 [call] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 186+0+0 (2444263206 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -294> 2015-07-01 10:34:38.590208 7ff91efdd700  2 req 6949:0.001376::GET /admin/replica_log:replicabilog_getbounds:http status=404

  -293> 2015-07-01 10:34:38.590214 7ff91efdd700  1 ====== req done req=0x7ff94000dde0 http_status=404 ======

  -292> 2015-07-01 10:34:38.612810 7ff91e7dc700  1 ====== starting new request req=0x7ff9400261b0 =====

  -291> 2015-07-01 10:34:38.612847 7ff91e7dc700  2 req 6950:0.000036::GET /admin/replica_log::initializing

  -290> 2015-07-01 10:34:38.612899 7ff91e7dc700  2 req 6950:0.000089::GET /admin/replica_log::getting op

  -289> 2015-07-01 10:34:38.612905 7ff91e7dc700  2 req 6950:0.000094::GET /admin/replica_log:replicabilog_getbounds:authorizing

  -288> 2015-07-01 10:34:38.613124 7ff91e7dc700  2 req 6950:0.000313::GET /admin/replica_log:replicabilog_getbounds:reading permissions

  -287> 2015-07-01 10:34:38.613134 7ff91e7dc700  2 req 6950:0.000324::GET /admin/replica_log:replicabilog_getbounds:init op

  -286> 2015-07-01 10:34:38.613137 7ff91e7dc700  2 req 6950:0.000327::GET /admin/replica_log:replicabilog_getbounds:verifying op mask

  -285> 2015-07-01 10:34:38.613140 7ff91e7dc700  2 req 6950:0.000330::GET /admin/replica_log:replicabilog_getbounds:verifying op permissions

  -284> 2015-07-01 10:34:38.613145 7ff91e7dc700  2 overriding permissions due to system operation

  -283> 2015-07-01 10:34:38.613148 7ff91e7dc700  2 req 6950:0.000338::GET /admin/replica_log:replicabilog_getbounds:verifying op params

  -282> 2015-07-01 10:34:38.613151 7ff91e7dc700  2 req 6950:0.000341::GET /admin/replica_log:replicabilog_getbounds:executing

  -281> 2015-07-01 10:34:38.613255 7ff91e7dc700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7122 replica_log.BASKET1 [call replica_log.get] 59.933a1b8d ack+read e425) v4 -- ?+0 0x7ff948045710 con 0x7ff95f9302e0

  -280> 2015-07-01 10:34:38.614654 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1798 ==== osd_op_reply(7122 replica_log.BASKET1 [call] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 186+0+0 (2444263206 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -279> 2015-07-01 10:34:38.614788 7ff91e7dc700  2 req 6950:0.001978::GET /admin/replica_log:replicabilog_getbounds:http status=404

  -278> 2015-07-01 10:34:38.614796 7ff91e7dc700  1 ====== req done req=0x7ff9400261b0 http_status=404 ======

  -277> 2015-07-01 10:34:38.635376 7ff9057aa700  1 ====== starting new request req=0x7ff9400397b0 =====

  -276> 2015-07-01 10:34:38.635411 7ff9057aa700  2 req 6951:0.000035::GET /admin/replica_log::initializing

  -275> 2015-07-01 10:34:38.635460 7ff9057aa700  2 req 6951:0.000084::GET /admin/replica_log::getting op

  -274> 2015-07-01 10:34:38.635466 7ff9057aa700  2 req 6951:0.000091::GET /admin/replica_log:replicabilog_getbounds:authorizing

  -273> 2015-07-01 10:34:38.635680 7ff9057aa700  2 req 6951:0.000304::GET /admin/replica_log:replicabilog_getbounds:reading permissions

  -272> 2015-07-01 10:34:38.635690 7ff9057aa700  2 req 6951:0.000315::GET /admin/replica_log:replicabilog_getbounds:init op

  -271> 2015-07-01 10:34:38.635693 7ff9057aa700  2 req 6951:0.000318::GET /admin/replica_log:replicabilog_getbounds:verifying op mask

  -270> 2015-07-01 10:34:38.635697 7ff9057aa700  2 req 6951:0.000322::GET /admin/replica_log:replicabilog_getbounds:verifying op permissions

  -269> 2015-07-01 10:34:38.635700 7ff9057aa700  2 overriding permissions due to system operation

  -268> 2015-07-01 10:34:38.635702 7ff9057aa700  2 req 6951:0.000327::GET /admin/replica_log:replicabilog_getbounds:verifying op params

  -267> 2015-07-01 10:34:38.635705 7ff9057aa700  2 req 6951:0.000330::GET /admin/replica_log:replicabilog_getbounds:executing

  -266> 2015-07-01 10:34:38.635800 7ff9057aa700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.75:6800/21154 -- osd_op(client.7211.0:7123 .bucket.meta.BASKET1:tw-east.6930.1 [call version.read,getxattrs,stat] 55.b017dbf8 ack+read e425) v4 -- ?+0 0x7ff93403d9b0 con 0x7ff94405f860

  -265> 2015-07-01 10:34:38.638411 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.4 10.35.64.75:6800/21154 12 ==== osd_op_reply(7123 .bucket.meta.BASKET1:tw-east.6930.1 [call,getxattrs,stat] v0'0 uv3 ondisk = 0) v6 ==== 286+0+322 (3643147009 0 1252381838) 0x7ff94006a1e0 con 0x7ff94405f860

  -264> 2015-07-01 10:34:38.638614 7ff9057aa700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7124 replica_log.BASKET1 [call replica_log.get] 59.933a1b8d ack+read e425) v4 -- ?+0 0x7ff934018ca0 con 0x7ff95f9302e0

  -263> 2015-07-01 10:34:38.640517 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1799 ==== osd_op_reply(7124 replica_log.BASKET1 [call] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 186+0+0 (2444263206 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -262> 2015-07-01 10:34:38.640644 7ff9057aa700  2 req 6951:0.005269::GET /admin/replica_log:replicabilog_getbounds:http status=404

  -261> 2015-07-01 10:34:38.640652 7ff9057aa700  1 ====== req done req=0x7ff9400397b0 http_status=404 ======

  -260> 2015-07-01 10:34:38.656534 7ff9147c8700  1 ====== starting new request req=0x7ff94006a620 =====

  -259> 2015-07-01 10:34:38.656585 7ff9147c8700  2 req 6952:0.000052::PUT /BASKET1/tw.json::initializing

  -258> 2015-07-01 10:34:38.656669 7ff9147c8700  2 req 6952:0.000136:s3:PUT /BASKET1/tw.json::getting op

  -257> 2015-07-01 10:34:38.656705 7ff9147c8700  2 req 6952:0.000173:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

  -256> 2015-07-01 10:34:38.656993 7ff9147c8700  2 req 6952:0.000460:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

  -255> 2015-07-01 10:34:38.657131 7ff9147c8700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7125 BASKET1 [call version.read,getxattrs,stat] 55.385b1c1c ack+read e425) v4 -- ?+0 0x7ff93c042930 con 0x7ff95f92e520

  -254> 2015-07-01 10:34:38.660558 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2779 ==== osd_op_reply(7125 BASKET1 [call,getxattrs,stat] v0'0 uv6 ondisk = 0) v6 ==== 258+0+139 (3447838219 0 2364625330) 0x7ff938036ec0 con 0x7ff95f92e520

  -253> 2015-07-01 10:34:38.660871 7ff9147c8700  2 req 6952:0.004338:s3:PUT /BASKET1/tw.json:copy_obj:init op

  -252> 2015-07-01 10:34:38.660883 7ff9147c8700  2 req 6952:0.004351:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

  -251> 2015-07-01 10:34:38.660886 7ff9147c8700  2 req 6952:0.004354:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

  -250> 2015-07-01 10:34:38.660966 7ff9147c8700  0 WARNING: couldn't find acl header for bucket, generating default

  -249> 2015-07-01 10:34:38.661032 7ff9147c8700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7126 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e425) v4 -- ?+0 0x7ff93c042930 con 0x7ff95f9302e0

  -248> 2015-07-01 10:34:38.662749 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1800 ==== osd_op_reply(7126 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (535621043 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -247> 2015-07-01 10:34:38.662928 7ff9147c8700  2 overriding permissions due to system operation

  -246> 2015-07-01 10:34:38.662936 7ff9147c8700  2 req 6952:0.006404:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

  -245> 2015-07-01 10:34:38.662943 7ff9147c8700  2 req 6952:0.006411:s3:PUT /BASKET1/tw.json:copy_obj:executing

  -244> 2015-07-01 10:34:38.663271 7ff9147c8700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

  -243> 2015-07-01 10:34:38.663347 7ff9147c8700 10 monclient: _send_mon_message to mon.node3 at 10.35.64.73:6789/0

  -242> 2015-07-01 10:34:38.663358 7ff9147c8700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6789/0 -- pool_op(create pool 0 auid 0 tid 7127 name .rgw.buckets v424) v4 -- ?+0 0x7ff93c012540 con 0x7ff95f926bf0

  -241> 2015-07-01 10:34:39.380079 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.72:6800/1502 -- ping v1 -- ?+0 0x7ff93000d760 con 0x7ff95f9240b0

  -240> 2015-07-01 10:34:39.380131 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- ping v1 -- ?+0 0x7ff93001cb20 con 0x7ff95f9302e0

  -239> 2015-07-01 10:34:39.380151 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.74:6800/27167 -- ping v1 -- ?+0 0x7ff93001ccc0 con 0x7ff95f934520

  -238> 2015-07-01 10:34:39.380170 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- ping v1 -- ?+0 0x7ff930045eb0 con 0x7ff95f92e520

  -237> 2015-07-01 10:34:39.397337 7ff954816700  1 -- 10.35.64.71:0/1026982 <== mon.2 10.35.64.73:6789/0 18 ==== osd_map(426..426 src has 1..426) v3 ==== 417+0+0 (3467348124 0 0) 0x7ff940034240 con 0x7ff95f926bf0

  -236> 2015-07-01 10:34:39.397490 7ff954816700  1 -- 10.35.64.71:0/1026982 <== mon.2 10.35.64.73:6789/0 19 ==== pool_op_reply(tid 7127 (0) Success v426) v1 ==== 43+0+0 (3279357725 0 0) 0x7ff94001fde0 con 0x7ff95f926bf0

  -235> 2015-07-01 10:34:39.397829 7ff9147c8700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7128 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e426) v4 -- ?+0 0x7ff93c042930 con 0x7ff95f92e520

  -234> 2015-07-01 10:34:39.629124 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2780 ==== osd_op_reply(7128 statelog.obj_opstate.68 [call] v426'4701 uv4701 ondisk = 0) v6 ==== 190+0+0 (1892154604 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -233> 2015-07-01 10:34:39.629302 7ff9147c8700  0 > HTTP_DATE -> Wed Jul  1 02:34:39 2015

  -232> 2015-07-01 10:34:39.629304 7ff9147c8700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

  -231> 2015-07-01 10:34:39.631885 7ff9147c8700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7129 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e426) v4 -- ?+0 0x7ff93c042930 con 0x7ff95f92e520

  -230> 2015-07-01 10:34:39.733125 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2781 ==== osd_op_reply(7129 statelog.obj_opstate.68 [call] v426'4702 uv4702 ondisk = 0) v6 ==== 190+0+0 (3841990531 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -229> 2015-07-01 10:34:39.733257 7ff9147c8700  0 WARNING: set_req_state_err err_no=5 resorting to 500

  -228> 2015-07-01 10:34:39.733334 7ff9147c8700  2 req 6952:1.076802:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

  -227> 2015-07-01 10:34:39.733353 7ff9147c8700  1 ====== req done req=0x7ff94006a620 http_status=500 ======

  -226> 2015-07-01 10:34:40.291928 7ff8fff9f700  1 ====== starting new request req=0x7ff94005b9e0 =====

  -225> 2015-07-01 10:34:40.291975 7ff8fff9f700  2 req 6953:0.000048::PUT /BASKET1/tw.json::initializing

  -224> 2015-07-01 10:34:40.292056 7ff8fff9f700  2 req 6953:0.000129:s3:PUT /BASKET1/tw.json::getting op

  -223> 2015-07-01 10:34:40.292091 7ff8fff9f700  2 req 6953:0.000164:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

  -222> 2015-07-01 10:34:40.292376 7ff8fff9f700  2 req 6953:0.000449:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

  -221> 2015-07-01 10:34:40.292507 7ff8fff9f700  2 req 6953:0.000580:s3:PUT /BASKET1/tw.json:copy_obj:init op

  -220> 2015-07-01 10:34:40.292513 7ff8fff9f700  2 req 6953:0.000586:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

  -219> 2015-07-01 10:34:40.292516 7ff8fff9f700  2 req 6953:0.000588:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

  -218> 2015-07-01 10:34:40.292565 7ff8fff9f700  0 WARNING: couldn't find acl header for bucket, generating default

  -217> 2015-07-01 10:34:40.292654 7ff8fff9f700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7130 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e426) v4 -- ?+0 0x7ff93c012540 con 0x7ff95f9302e0

  -216> 2015-07-01 10:34:40.294338 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1801 ==== osd_op_reply(7130 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (1239691441 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -215> 2015-07-01 10:34:40.294503 7ff8fff9f700  2 overriding permissions due to system operation

  -214> 2015-07-01 10:34:40.294509 7ff8fff9f700  2 req 6953:0.002582:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

  -213> 2015-07-01 10:34:40.294514 7ff8fff9f700  2 req 6953:0.002587:s3:PUT /BASKET1/tw.json:copy_obj:executing

  -212> 2015-07-01 10:34:40.294579 7ff8fff9f700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

  -211> 2015-07-01 10:34:40.294686 7ff8fff9f700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7131 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e426) v4 -- ?+0 0x7ff93c012540 con 0x7ff95f92e520

  -210> 2015-07-01 10:34:40.304814 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2782 ==== osd_op_reply(7131 statelog.obj_opstate.68 [call] v426'4703 uv4703 ondisk = 0) v6 ==== 190+0+0 (2520956582 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -209> 2015-07-01 10:34:40.304926 7ff8fff9f700  0 > HTTP_DATE -> Wed Jul  1 02:34:40 2015

  -208> 2015-07-01 10:34:40.304928 7ff8fff9f700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

  -207> 2015-07-01 10:34:40.307663 7ff8fff9f700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7132 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e426) v4 -- ?+0 0x7ff93c012540 con 0x7ff95f92e520

  -206> 2015-07-01 10:34:40.317280 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2783 ==== osd_op_reply(7132 statelog.obj_opstate.68 [call] v426'4704 uv4704 ondisk = 0) v6 ==== 190+0+0 (699648542 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -205> 2015-07-01 10:34:40.317349 7ff8fff9f700  0 WARNING: set_req_state_err err_no=5 resorting to 500

  -204> 2015-07-01 10:34:40.317385 7ff8fff9f700  2 req 6953:0.025458:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

  -203> 2015-07-01 10:34:40.317401 7ff8fff9f700  1 ====== req done req=0x7ff94005b9e0 http_status=500 ======

  -202> 2015-07-01 10:34:41.903723 7ff9087b0700  1 ====== starting new request req=0x7ff940034db0 =====

  -201> 2015-07-01 10:34:41.903778 7ff9087b0700  2 req 6954:0.000057::PUT /BASKET1/tw.json::initializing

  -200> 2015-07-01 10:34:41.903880 7ff9087b0700  2 req 6954:0.000159:s3:PUT /BASKET1/tw.json::getting op

  -199> 2015-07-01 10:34:41.903892 7ff9087b0700  2 req 6954:0.000171:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

  -198> 2015-07-01 10:34:41.904225 7ff9087b0700  2 req 6954:0.000504:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

  -197> 2015-07-01 10:34:41.904406 7ff9087b0700  2 req 6954:0.000685:s3:PUT /BASKET1/tw.json:copy_obj:init op

  -196> 2015-07-01 10:34:41.904414 7ff9087b0700  2 req 6954:0.000693:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

  -195> 2015-07-01 10:34:41.904418 7ff9087b0700  2 req 6954:0.000697:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

  -194> 2015-07-01 10:34:41.904485 7ff9087b0700  0 WARNING: couldn't find acl header for bucket, generating default

  -193> 2015-07-01 10:34:41.904597 7ff9087b0700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7133 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e426) v4 -- ?+0 0x7ff9380569a0 con 0x7ff95f9302e0

  -192> 2015-07-01 10:34:41.906680 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1802 ==== osd_map(427..427 src has 1..427) v3 ==== 238+0+0 (1866214008 0 0) 0x7ff95f97d180 con 0x7ff95f9302e0

  -191> 2015-07-01 10:34:41.906838 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1803 ==== osd_op_reply(7133 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (2078508111 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -190> 2015-07-01 10:34:41.907032 7ff9087b0700  2 overriding permissions due to system operation

  -189> 2015-07-01 10:34:41.907041 7ff9087b0700  2 req 6954:0.003319:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

  -188> 2015-07-01 10:34:41.907047 7ff9087b0700  2 req 6954:0.003326:s3:PUT /BASKET1/tw.json:copy_obj:executing

  -187> 2015-07-01 10:34:41.907338 7ff9087b0700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

  -186> 2015-07-01 10:34:41.907508 7ff9087b0700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7134 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff93805c050 con 0x7ff95f92e520

  -185> 2015-07-01 10:34:41.921940 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2784 ==== osd_op_reply(7134 statelog.obj_opstate.68 [call] v427'4705 uv4705 ondisk = 0) v6 ==== 190+0+0 (794285493 0 0) 0x7ff93805c050 con 0x7ff95f92e520

  -184> 2015-07-01 10:34:41.922114 7ff9087b0700  0 > HTTP_DATE -> Wed Jul  1 02:34:41 2015

  -183> 2015-07-01 10:34:41.922118 7ff9087b0700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

  -182> 2015-07-01 10:34:41.925432 7ff9087b0700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7135 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff938048240 con 0x7ff95f92e520

  -181> 2015-07-01 10:34:41.931486 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2785 ==== osd_op_reply(7135 statelog.obj_opstate.68 [call] v427'4706 uv4706 ondisk = 0) v6 ==== 190+0+0 (3130013402 0 0) 0x7ff938056780 con 0x7ff95f92e520

  -180> 2015-07-01 10:34:41.931607 7ff9087b0700  0 WARNING: set_req_state_err err_no=5 resorting to 500

  -179> 2015-07-01 10:34:41.931666 7ff9087b0700  2 req 6954:0.027945:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

  -178> 2015-07-01 10:34:41.931678 7ff9087b0700  1 ====== req done req=0x7ff940034db0 http_status=500 ======

  -177> 2015-07-01 10:34:44.380483 7ff94e6fb700 10 monclient: renew_subs

  -176> 2015-07-01 10:34:44.380531 7ff94e6fb700 10 monclient: _send_mon_message to mon.node3 at 10.35.64.73:6789/0

  -175> 2015-07-01 10:34:44.380558 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6789/0 -- mon_subscribe({monmap=2+,osdmap=428}) v2 -- ?+0 0x7ff93000d760 con 0x7ff95f926bf0

  -174> 2015-07-01 10:34:44.380597 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.72:6800/1502 -- ping v1 -- ?+0 0x7ff93001cb20 con 0x7ff95f9240b0

  -173> 2015-07-01 10:34:44.380617 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- ping v1 -- ?+0 0x7ff93001ccc0 con 0x7ff95f9302e0

  -172> 2015-07-01 10:34:44.380634 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.74:6800/27167 -- ping v1 -- ?+0 0x7ff930045eb0 con 0x7ff95f934520

  -171> 2015-07-01 10:34:44.380652 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- ping v1 -- ?+0 0x7ff9300460d0 con 0x7ff95f92e520

  -170> 2015-07-01 10:34:44.383326 7ff954816700  1 -- 10.35.64.71:0/1026982 <== mon.2 10.35.64.73:6789/0 20 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (2045553213 0 0) 0x7ff94001c4a0 con 0x7ff95f926bf0

  -169> 2015-07-01 10:34:44.383361 7ff954816700 10 monclient: handle_subscribe_ack sent 2015-07-01 10:34:44.380522 renew after 2015-07-01 10:37:14.380522

  -168> 2015-07-01 10:34:44.485520 7ff902fa5700  1 ====== starting new request req=0x7ff94000dde0 =====

  -167> 2015-07-01 10:34:44.485567 7ff902fa5700  2 req 6955:0.000048::PUT /BASKET1/tw.json::initializing

  -166> 2015-07-01 10:34:44.485636 7ff902fa5700  2 req 6955:0.000117:s3:PUT /BASKET1/tw.json::getting op

  -165> 2015-07-01 10:34:44.485643 7ff902fa5700  2 req 6955:0.000123:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

  -164> 2015-07-01 10:34:44.485871 7ff902fa5700  2 req 6955:0.000351:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

  -163> 2015-07-01 10:34:44.485987 7ff902fa5700  2 req 6955:0.000468:s3:PUT /BASKET1/tw.json:copy_obj:init op

  -162> 2015-07-01 10:34:44.485992 7ff902fa5700  2 req 6955:0.000473:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

  -161> 2015-07-01 10:34:44.485994 7ff902fa5700  2 req 6955:0.000475:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

  -160> 2015-07-01 10:34:44.486038 7ff902fa5700  0 WARNING: couldn't find acl header for bucket, generating default

  -159> 2015-07-01 10:34:44.486109 7ff902fa5700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7136 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e427) v4 -- ?+0 0x7ff93c012540 con 0x7ff95f9302e0

  -158> 2015-07-01 10:34:44.487426 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1804 ==== osd_op_reply(7136 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (2078508111 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -157> 2015-07-01 10:34:44.487563 7ff902fa5700  2 overriding permissions due to system operation

  -156> 2015-07-01 10:34:44.487569 7ff902fa5700  2 req 6955:0.002050:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

  -155> 2015-07-01 10:34:44.487573 7ff902fa5700  2 req 6955:0.002054:s3:PUT /BASKET1/tw.json:copy_obj:executing

  -154> 2015-07-01 10:34:44.487639 7ff902fa5700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

  -153> 2015-07-01 10:34:44.487732 7ff902fa5700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7137 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff93c014ff0 con 0x7ff95f92e520

  -152> 2015-07-01 10:34:44.508404 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2786 ==== osd_op_reply(7137 statelog.obj_opstate.68 [call] v427'4707 uv4707 ondisk = 0) v6 ==== 190+0+0 (3386021887 0 0) 0x7ff938056b10 con 0x7ff95f92e520

  -151> 2015-07-01 10:34:44.508509 7ff902fa5700  0 > HTTP_DATE -> Wed Jul  1 02:34:44 2015

  -150> 2015-07-01 10:34:44.508511 7ff902fa5700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

  -149> 2015-07-01 10:34:44.512194 7ff902fa5700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7138 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff93c012540 con 0x7ff95f92e520

  -148> 2015-07-01 10:34:44.549538 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2787 ==== osd_op_reply(7138 statelog.obj_opstate.68 [call] v427'4708 uv4708 ondisk = 0) v6 ==== 190+0+0 (2498987765 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -147> 2015-07-01 10:34:44.549616 7ff902fa5700  0 WARNING: set_req_state_err err_no=5 resorting to 500

  -146> 2015-07-01 10:34:44.549737 7ff902fa5700  2 req 6955:0.064218:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

  -145> 2015-07-01 10:34:44.549777 7ff902fa5700  1 ====== req done req=0x7ff94000dde0 http_status=500 ======

  -144> 2015-07-01 10:34:46.394077 7ff918fd1700  1 ====== starting new request req=0x7ff9400397b0 =====

  -143> 2015-07-01 10:34:46.394144 7ff918fd1700  2 req 6956:0.000069::GET /admin/opstate::initializing

  -142> 2015-07-01 10:34:46.394220 7ff918fd1700  2 req 6956:0.000145::GET /admin/opstate::getting op

  -141> 2015-07-01 10:34:46.394226 7ff918fd1700  2 req 6956:0.000151::GET /admin/opstate:opstate_list:authorizing

  -140> 2015-07-01 10:34:46.394567 7ff918fd1700  2 req 6956:0.000491::GET /admin/opstate:opstate_list:reading permissions

  -139> 2015-07-01 10:34:46.394576 7ff918fd1700  2 req 6956:0.000501::GET /admin/opstate:opstate_list:init op

  -138> 2015-07-01 10:34:46.394580 7ff918fd1700  2 req 6956:0.000505::GET /admin/opstate:opstate_list:verifying op mask

  -137> 2015-07-01 10:34:46.394583 7ff918fd1700  2 req 6956:0.000508::GET /admin/opstate:opstate_list:verifying op permissions

  -136> 2015-07-01 10:34:46.394588 7ff918fd1700  2 overriding permissions due to system operation

  -135> 2015-07-01 10:34:46.394590 7ff918fd1700  2 req 6956:0.000515::GET /admin/opstate:opstate_list:verifying op params

  -134> 2015-07-01 10:34:46.394593 7ff918fd1700  2 req 6956:0.000518::GET /admin/opstate:opstate_list:executing

  -133> 2015-07-01 10:34:46.394725 7ff918fd1700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7139 statelog.obj_opstate.68 [call statelog.list] 59.868ada28 ack+read e427) v4 -- ?+0 0x7ff93403d540 con 0x7ff95f92e520

  -132> 2015-07-01 10:34:46.398127 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2788 ==== osd_op_reply(7139 statelog.obj_opstate.68 [call] v0'0 uv4708 ondisk = 0) v6 ==== 190+0+90 (366987647 0 1510955586) 0x7ff938052680 con 0x7ff95f92e520

  -131> 2015-07-01 10:34:46.398283 7ff918fd1700  2 req 6956:0.004208::GET /admin/opstate:opstate_list:http status=200

  -130> 2015-07-01 10:34:46.398290 7ff918fd1700  1 ====== req done req=0x7ff9400397b0 http_status=200 ======

  -129> 2015-07-01 10:34:46.404346 7ff9027a4700  1 ====== starting new request req=0x7ff940027840 =====

  -128> 2015-07-01 10:34:46.404375 7ff9027a4700  2 req 6957:0.000030::PUT /BASKET1/tw.json::initializing

  -127> 2015-07-01 10:34:46.404435 7ff9027a4700  2 req 6957:0.000090:s3:PUT /BASKET1/tw.json::getting op

  -126> 2015-07-01 10:34:46.404451 7ff9027a4700  2 req 6957:0.000106:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

  -125> 2015-07-01 10:34:46.404640 7ff9027a4700  2 req 6957:0.000295:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

  -124> 2015-07-01 10:34:46.404789 7ff9027a4700  2 req 6957:0.000443:s3:PUT /BASKET1/tw.json:copy_obj:init op

  -123> 2015-07-01 10:34:46.404795 7ff9027a4700  2 req 6957:0.000449:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

  -122> 2015-07-01 10:34:46.404798 7ff9027a4700  2 req 6957:0.000453:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

  -121> 2015-07-01 10:34:46.404853 7ff9027a4700  0 WARNING: couldn't find acl header for bucket, generating default

  -120> 2015-07-01 10:34:46.404914 7ff9027a4700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7140 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e427) v4 -- ?+0 0x7ff950019e10 con 0x7ff95f9302e0

  -119> 2015-07-01 10:34:46.406435 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1805 ==== osd_op_reply(7140 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (2078508111 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

  -118> 2015-07-01 10:34:46.406592 7ff9027a4700  2 overriding permissions due to system operation

  -117> 2015-07-01 10:34:46.406598 7ff9027a4700  2 req 6957:0.002252:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

  -116> 2015-07-01 10:34:46.406603 7ff9027a4700  2 req 6957:0.002258:s3:PUT /BASKET1/tw.json:copy_obj:executing

  -115> 2015-07-01 10:34:46.406682 7ff9027a4700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

  -114> 2015-07-01 10:34:46.406799 7ff9027a4700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7141 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff950019e10 con 0x7ff95f92e520

  -113> 2015-07-01 10:34:46.482671 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2789 ==== osd_op_reply(7141 statelog.obj_opstate.68 [call] v427'4709 uv4709 ondisk = 0) v6 ==== 190+0+0 (3887146960 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -112> 2015-07-01 10:34:46.482770 7ff9027a4700  0 > HTTP_DATE -> Wed Jul  1 02:34:46 2015

  -111> 2015-07-01 10:34:46.482771 7ff9027a4700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

  -110> 2015-07-01 10:34:46.484974 7ff9027a4700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7142 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff950019e10 con 0x7ff95f92e520

  -109> 2015-07-01 10:34:46.552722 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2790 ==== osd_op_reply(7142 statelog.obj_opstate.68 [call] v427'4710 uv4710 ondisk = 0) v6 ==== 190+0+0 (1920386239 0 0) 0x7ff938052680 con 0x7ff95f92e520

  -108> 2015-07-01 10:34:46.552883 7ff9027a4700  0 WARNING: set_req_state_err err_no=5 resorting to 500

  -107> 2015-07-01 10:34:46.552950 7ff9027a4700  2 req 6957:0.148604:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

  -106> 2015-07-01 10:34:46.552971 7ff9027a4700  1 ====== req done req=0x7ff940027840 http_status=500 ======

  -105> 2015-07-01 10:34:47.347840 7ff94f7fe700 10 monclient: tick

  -104> 2015-07-01 10:34:47.347907 7ff94f7fe700 10 monclient: renew subs? (now: 2015-07-01 10:34:47.347907; renew after: 2015-07-01 10:37:14.380522) -- no

  -103> 2015-07-01 10:34:47.550229 7ff90d7ba700  1 ====== starting new request req=0x7ff940015950 =====

  -102> 2015-07-01 10:34:47.550275 7ff90d7ba700  2 req 6958:0.000046::PUT /BASKET1/tw.json::initializing

  -101> 2015-07-01 10:34:47.550333 7ff90d7ba700  2 req 6958:0.000104:s3:PUT /BASKET1/tw.json::getting op

  -100> 2015-07-01 10:34:47.550340 7ff90d7ba700  2 req 6958:0.000111:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

   -99> 2015-07-01 10:34:47.550544 7ff90d7ba700  2 req 6958:0.000315:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

   -98> 2015-07-01 10:34:47.550644 7ff90d7ba700  2 req 6958:0.000415:s3:PUT /BASKET1/tw.json:copy_obj:init op

   -97> 2015-07-01 10:34:47.550648 7ff90d7ba700  2 req 6958:0.000419:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

   -96> 2015-07-01 10:34:47.550651 7ff90d7ba700  2 req 6958:0.000421:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

   -95> 2015-07-01 10:34:47.550688 7ff90d7ba700  0 WARNING: couldn't find acl header for bucket, generating default

   -94> 2015-07-01 10:34:47.550752 7ff90d7ba700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7143 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e427) v4 -- ?+0 0x7ff940074d00 con 0x7ff95f9302e0

   -93> 2015-07-01 10:34:47.552349 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1806 ==== osd_op_reply(7143 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (2078508111 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

   -92> 2015-07-01 10:34:47.552454 7ff90d7ba700  2 overriding permissions due to system operation

   -91> 2015-07-01 10:34:47.552458 7ff90d7ba700  2 req 6958:0.002228:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

   -90> 2015-07-01 10:34:47.552461 7ff90d7ba700  2 req 6958:0.002232:s3:PUT /BASKET1/tw.json:copy_obj:executing

   -89> 2015-07-01 10:34:47.552520 7ff90d7ba700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

   -88> 2015-07-01 10:34:47.552594 7ff90d7ba700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7144 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff940074d00 con 0x7ff95f92e520

   -87> 2015-07-01 10:34:47.558448 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2791 ==== osd_op_reply(7144 statelog.obj_opstate.68 [call] v427'4711 uv4711 ondisk = 0) v6 ==== 190+0+0 (20211098 0 0) 0x7ff938052680 con 0x7ff95f92e520

   -86> 2015-07-01 10:34:47.558518 7ff90d7ba700  0 > HTTP_DATE -> Wed Jul  1 02:34:47 2015

   -85> 2015-07-01 10:34:47.558520 7ff90d7ba700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

   -84> 2015-07-01 10:34:47.560962 7ff90d7ba700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7145 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff940074d00 con 0x7ff95f92e520

   -83> 2015-07-01 10:34:47.565839 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2792 ==== osd_op_reply(7145 statelog.obj_opstate.68 [call] v427'4712 uv4712 ondisk = 0) v6 ==== 190+0+0 (3358910123 0 0) 0x7ff938052680 con 0x7ff95f92e520

   -82> 2015-07-01 10:34:47.566081 7ff90d7ba700  0 WARNING: set_req_state_err err_no=5 resorting to 500

   -81> 2015-07-01 10:34:47.566119 7ff90d7ba700  2 req 6958:0.015890:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

   -80> 2015-07-01 10:34:47.566126 7ff90d7ba700  1 ====== req done req=0x7ff940015950 http_status=500 ======

   -79> 2015-07-01 10:34:48.336661 7ff94d6f9700  2 RGWDataChangesLog::ChangesRenewThread: start

   -78> 2015-07-01 10:34:48.932688 7ff913fc7700  1 ====== starting new request req=0x7ff94001c370 =====

   -77> 2015-07-01 10:34:48.932755 7ff913fc7700  2 req 6959:0.000069::PUT /BASKET1/tw.json::initializing

   -76> 2015-07-01 10:34:48.932849 7ff913fc7700  2 req 6959:0.000162:s3:PUT /BASKET1/tw.json::getting op

   -75> 2015-07-01 10:34:48.932860 7ff913fc7700  2 req 6959:0.000174:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

   -74> 2015-07-01 10:34:48.933188 7ff913fc7700  2 req 6959:0.000501:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

   -73> 2015-07-01 10:34:48.933362 7ff913fc7700  2 req 6959:0.000676:s3:PUT /BASKET1/tw.json:copy_obj:init op

   -72> 2015-07-01 10:34:48.933370 7ff913fc7700  2 req 6959:0.000683:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

   -71> 2015-07-01 10:34:48.933373 7ff913fc7700  2 req 6959:0.000687:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

   -70> 2015-07-01 10:34:48.933439 7ff913fc7700  0 WARNING: couldn't find acl header for bucket, generating default

   -69> 2015-07-01 10:34:48.933536 7ff913fc7700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7146 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e427) v4 -- ?+0 0x7ff938049650 con 0x7ff95f9302e0

   -68> 2015-07-01 10:34:48.935930 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1807 ==== osd_op_reply(7146 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (2078508111 0 0) 0x7ff95f944240 con 0x7ff95f9302e0

   -67> 2015-07-01 10:34:48.936103 7ff913fc7700  2 overriding permissions due to system operation

   -66> 2015-07-01 10:34:48.936110 7ff913fc7700  2 req 6959:0.003424:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

   -65> 2015-07-01 10:34:48.936116 7ff913fc7700  2 req 6959:0.003430:s3:PUT /BASKET1/tw.json:copy_obj:executing

   -64> 2015-07-01 10:34:48.936209 7ff913fc7700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

   -63> 2015-07-01 10:34:48.936346 7ff913fc7700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7147 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff938048c60 con 0x7ff95f92e520

   -62> 2015-07-01 10:34:48.945999 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2793 ==== osd_op_reply(7147 statelog.obj_opstate.68 [call] v427'4713 uv4713 ondisk = 0) v6 ==== 190+0+0 (3145090958 0 0) 0x7ff9380491d0 con 0x7ff95f92e520

   -61> 2015-07-01 10:34:48.946175 7ff913fc7700  0 > HTTP_DATE -> Wed Jul  1 02:34:48 2015

   -60> 2015-07-01 10:34:48.946179 7ff913fc7700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

   -59> 2015-07-01 10:34:48.949680 7ff913fc7700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7148 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff938054010 con 0x7ff95f92e520

   -58> 2015-07-01 10:34:48.955950 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2794 ==== osd_op_reply(7148 statelog.obj_opstate.68 [call] v427'4714 uv4714 ondisk = 0) v6 ==== 190+0+0 (783394017 0 0) 0x7ff938054010 con 0x7ff95f92e520

   -57> 2015-07-01 10:34:48.956077 7ff913fc7700  0 WARNING: set_req_state_err err_no=5 resorting to 500

   -56> 2015-07-01 10:34:48.956219 7ff913fc7700  2 req 6959:0.023532:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

   -55> 2015-07-01 10:34:48.956261 7ff913fc7700  1 ====== req done req=0x7ff94001c370 http_status=500 ======

   -54> 2015-07-01 10:34:49.380928 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.72:6800/1502 -- ping v1 -- ?+0 0x7ff930045eb0 con 0x7ff95f9240b0

   -53> 2015-07-01 10:34:49.380977 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- ping v1 -- ?+0 0x7ff9300460d0 con 0x7ff95f9302e0

   -52> 2015-07-01 10:34:49.381002 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.74:6800/27167 -- ping v1 -- ?+0 0x7ff9300462f0 con 0x7ff95f934520

   -51> 2015-07-01 10:34:49.381019 7ff94e6fb700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- ping v1 -- ?+0 0x7ff93000d760 con 0x7ff95f92e520

   -50> 2015-07-01 10:34:51.904977 7ff929ff3700  1 ====== starting new request req=0x7ff94005b9e0 =====

   -49> 2015-07-01 10:34:51.905033 7ff929ff3700  2 req 6960:0.000057::PUT /BASKET1/tw.json::initializing

   -48> 2015-07-01 10:34:51.905106 7ff929ff3700  2 req 6960:0.000130:s3:PUT /BASKET1/tw.json::getting op

   -47> 2015-07-01 10:34:51.905114 7ff929ff3700  2 req 6960:0.000139:s3:PUT /BASKET1/tw.json:copy_obj:authorizing

   -46> 2015-07-01 10:34:51.905370 7ff929ff3700  2 req 6960:0.000394:s3:PUT /BASKET1/tw.json:copy_obj:reading permissions

   -45> 2015-07-01 10:34:51.905500 7ff929ff3700  2 req 6960:0.000524:s3:PUT /BASKET1/tw.json:copy_obj:init op

   -44> 2015-07-01 10:34:51.905505 7ff929ff3700  2 req 6960:0.000530:s3:PUT /BASKET1/tw.json:copy_obj:verifying op mask

   -43> 2015-07-01 10:34:51.905508 7ff929ff3700  2 req 6960:0.000533:s3:PUT /BASKET1/tw.json:copy_obj:verifying op permissions

   -42> 2015-07-01 10:34:51.905556 7ff929ff3700  0 WARNING: couldn't find acl header for bucket, generating default

   -41> 2015-07-01 10:34:51.905684 7ff929ff3700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.71:6800/25970 -- osd_op(client.7211.0:7149 tw-east [getxattrs,stat] 57.24dd5f83 ack+read e427) v4 -- ?+0 0x7ff934014bd0 con 0x7ff95f9302e0

   -40> 2015-07-01 10:34:51.907012 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.0 10.35.64.71:6800/25970 1808 ==== osd_op_reply(7149 tw-east [getxattrs,stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v6 ==== 216+0+0 (2078508111 0 0) 0x7ff95f97c820 con 0x7ff95f9302e0

   -39> 2015-07-01 10:34:51.907303 7ff929ff3700  2 overriding permissions due to system operation

   -38> 2015-07-01 10:34:51.907310 7ff929ff3700  2 req 6960:0.002335:s3:PUT /BASKET1/tw.json:copy_obj:verifying op params

   -37> 2015-07-01 10:34:51.907315 7ff929ff3700  2 req 6960:0.002340:s3:PUT /BASKET1/tw.json:copy_obj:executing

   -36> 2015-07-01 10:34:51.907387 7ff929ff3700  5 Copy object BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json => BASKET1(@{i=.rgw.buckets.index}.rgw.buckets[tw-east.6930.1]):tw.json

   -35> 2015-07-01 10:34:51.907490 7ff929ff3700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7150 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff934014bd0 con 0x7ff95f92e520

   -34> 2015-07-01 10:34:51.922481 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2795 ==== osd_op_reply(7150 statelog.obj_opstate.68 [call] v427'4715 uv4715 ondisk = 0) v6 ==== 190+0+0 (1576207812 0 0) 0x7ff9380492a0 con 0x7ff95f92e520

   -33> 2015-07-01 10:34:51.922573 7ff929ff3700  0 > HTTP_DATE -> Wed Jul  1 02:34:51 2015

   -32> 2015-07-01 10:34:51.922575 7ff929ff3700  0 > HTTP_X_AMZ_COPY_SOURCE -> BASKET1/tw.json

   -31> 2015-07-01 10:34:51.925367 7ff929ff3700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7151 statelog.obj_opstate.68 [call statelog.add] 59.868ada28 ondisk+write e427) v4 -- ?+0 0x7ff934014bd0 con 0x7ff95f92e520

   -30> 2015-07-01 10:34:51.932631 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2796 ==== osd_op_reply(7151 statelog.obj_opstate.68 [call] v427'4716 uv4716 ondisk = 0) v6 ==== 190+0+0 (13767886 0 0) 0x7ff938052680 con 0x7ff95f92e520

   -29> 2015-07-01 10:34:51.932717 7ff929ff3700  0 WARNING: set_req_state_err err_no=5 resorting to 500

   -28> 2015-07-01 10:34:51.932812 7ff929ff3700  2 req 6960:0.027837:s3:PUT /BASKET1/tw.json:copy_obj:http status=500

   -27> 2015-07-01 10:34:51.932838 7ff929ff3700  1 ====== req done req=0x7ff94005b9e0 http_status=500 ======

   -26> 2015-07-01 10:34:54.096336 7ff91c7d8700  1 ====== starting new request req=0x7ff94000dde0 =====

   -25> 2015-07-01 10:34:54.096395 7ff91c7d8700  2 req 6961:0.000060::GET /admin/opstate::initializing

   -24> 2015-07-01 10:34:54.096482 7ff91c7d8700  2 req 6961:0.000147::GET /admin/opstate::getting op

   -23> 2015-07-01 10:34:54.096488 7ff91c7d8700  2 req 6961:0.000154::GET /admin/opstate:opstate_list:authorizing

   -22> 2015-07-01 10:34:54.096809 7ff91c7d8700  2 req 6961:0.000474::GET /admin/opstate:opstate_list:reading permissions

   -21> 2015-07-01 10:34:54.096818 7ff91c7d8700  2 req 6961:0.000484::GET /admin/opstate:opstate_list:init op

   -20> 2015-07-01 10:34:54.096823 7ff91c7d8700  2 req 6961:0.000489::GET /admin/opstate:opstate_list:verifying op mask

   -19> 2015-07-01 10:34:54.096827 7ff91c7d8700  2 req 6961:0.000493::GET /admin/opstate:opstate_list:verifying op permissions

   -18> 2015-07-01 10:34:54.096832 7ff91c7d8700  2 overriding permissions due to system operation

   -17> 2015-07-01 10:34:54.096835 7ff91c7d8700  2 req 6961:0.000501::GET /admin/opstate:opstate_list:verifying op params

   -16> 2015-07-01 10:34:54.096838 7ff91c7d8700  2 req 6961:0.000504::GET /admin/opstate:opstate_list:executing

   -15> 2015-07-01 10:34:54.096964 7ff91c7d8700  1 -- 10.35.64.71:0/1026982 --> 10.35.64.73:6800/29951 -- osd_op(client.7211.0:7152 statelog.obj_opstate.68 [call statelog.list] 59.868ada28 ack+read e427) v4 -- ?+0 0x7ff930049f60 con 0x7ff95f92e520

   -14> 2015-07-01 10:34:54.099798 7ff954816700  1 -- 10.35.64.71:0/1026982 <== osd.2 10.35.64.73:6800/29951 2797 ==== osd_op_reply(7152 statelog.obj_opstate.68 [call] v0'0 uv4716 ondisk = 0) v6 ==== 190+0+90 (1631025820 0 1691987791) 0x7ff938052680 con 0x7ff95f92e520

   -13> 2015-07-01 10:34:54.099959 7ff91c7d8700  2 req 6961:0.003625::GET /admin/opstate:opstate_list:http status=200

   -12> 2015-07-01 10:34:54.099972 7ff91c7d8700  1 ====== req done req=0x7ff94000dde0 http_status=200 ======

   -11> 2015-07-01 10:34:54.112960 7ff91b7d6700  1 ====== starting new request req=0x7ff940034db0 =====

   -10> 2015-07-01 10:34:54.112996 7ff91b7d6700  2 req 6962:0.000036::POST /admin/replica_log::initializing

    -9> 2015-07-01 10:34:54.113064 7ff91b7d6700  2 req 6962:0.000105::POST /admin/replica_log::getting op

    -8> 2015-07-01 10:34:54.113071 7ff91b7d6700  2 req 6962:0.000112::POST /admin/replica_log:replicabilog_updatebounds:authorizing

    -7> 2015-07-01 10:34:54.113293 7ff91b7d6700  2 req 6962:0.000334::POST /admin/replica_log:replicabilog_updatebounds:reading permissions

    -6> 2015-07-01 10:34:54.113303 7ff91b7d6700  2 req 6962:0.000344::POST /admin/replica_log:replicabilog_updatebounds:init op

    -5> 2015-07-01 10:34:54.113307 7ff91b7d6700  2 req 6962:0.000348::POST /admin/replica_log:replicabilog_updatebounds:verifying op mask

    -4> 2015-07-01 10:34:54.113311 7ff91b7d6700  2 req 6962:0.000352::POST /admin/replica_log:replicabilog_updatebounds:verifying op permissions

    -3> 2015-07-01 10:34:54.113316 7ff91b7d6700  2 overriding permissions due to system operation

    -2> 2015-07-01 10:34:54.113318 7ff91b7d6700  2 req 6962:0.000359::POST /admin/replica_log:replicabilog_updatebounds:verifying op params

    -1> 2015-07-01 10:34:54.113321 7ff91b7d6700  2 req 6962:0.000362::POST /admin/replica_log:replicabilog_updatebounds:executing

     0> 2015-07-01 10:34:54.116891 7ff91b7d6700 -1 *** Caught signal (Aborted) **

 in thread 7ff91b7d6700

 ceph version 0.80.9 (b5a67f0e1d15385bc0d60a6da6e7fc810bde6047)

 1: (()+0x1f18af) [0x7ff95efcb8af]

 2: (()+0x10340) [0x7ff95cfcd340]

 3: (gsignal()+0x39) [0x7ff95c0d4cc9]

 4: (abort()+0x148) [0x7ff95c0d80d8]

 5: (__gnu_cxx::__verbose_terminate_handler()+0x155) [0x7ff95c9df535]

 6: (()+0x5e6d6) [0x7ff95c9dd6d6]

 7: (()+0x5e703) [0x7ff95c9dd703]

 8: (()+0x5e922) [0x7ff95c9dd922]

 9: (bool JSONDecoder::decode_json<utime_t>(char const*, utime_t&, JSONObj*, bool)+0x204) [0x7ff95eed35b4]

 10: (void decode_json_obj<cls_replica_log_item_marker>(std::list<cls_replica_log_item_marker, std::allocator<cls_replica_log_item_marker> >&, JSONObj*)+0x73) [0x7ff95eea9e13]

 11: (int rgw_rest_get_json_input<std::list<cls_replica_log_item_marker, std::allocator<cls_replica_log_item_marker> > >(CephContext*, req_state*, std::list<cls_replica_log_item_marker, std::allocator<cls_replica_log_item_marker> >&, int, bool*)+0xa5) [0x7ff95eea9f75]

 12: (RGWOp_BILog_SetBounds::execute()+0x390) [0x7ff95eea8e20]

 13: (()+0xde63d) [0x7ff95eeb863d]

 14: (RGWFCGXProcess::handle_request(RGWRequest*)+0x80) [0x7ff95eeb8d30]

 15: (RGWProcess::RGWWQ::_process(RGWRequest*)+0x30) [0x7ff95eeba340]

 16: (ThreadPool::worker(ThreadPool::WorkThread*)+0xaf1) [0x7ff95deca531]

 17: (ThreadPool::WorkThread::entry()+0x10) [0x7ff95decb420]

 18: (()+0x8182) [0x7ff95cfc5182]

 19: (clone()+0x6d) [0x7ff95c19847d]

 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---

   0/ 5 none

   0/ 1 lockdep

   0/ 1 context

   1/ 1 crush

   1/ 5 mds

   1/ 5 mds_balancer

   1/ 5 mds_locker

   1/ 5 mds_log

   1/ 5 mds_log_expire

   1/ 5 mds_migrator

   0/ 1 buffer

   0/ 1 timer

   0/ 1 filer

   0/ 1 striper

   0/ 1 objecter

   0/ 5 rados

   0/ 5 rbd

   0/ 5 journaler

   0/ 5 objectcacher

   0/ 5 client

   0/ 5 osd

   0/ 5 optracker

   0/ 5 objclass

   1/ 3 filestore

   1/ 3 keyvaluestore

   1/ 3 journal

   0/ 5 ms

   1/ 5 mon

   0/10 monc

   1/ 5 paxos

   0/ 5 tp

   1/ 5 auth

   1/ 5 crypto

   1/ 1 finisher

   1/ 5 heartbeatmap

   1/ 5 perfcounter

   1/ 5 rgw

   1/10 civetweb

   1/ 5 javaclient

   1/ 5 asok

   1/ 1 throttle

  -2/-2 (syslog threshold)

  -1/-1 (stderr threshold)

  max_recent     10000

  max_new         1000

  log_file /var/log/ceph/radosgw.node1.log

--- end dump of recent events ---