Project

General

Profile

Bug #50467 » rgw.log

Jan Harkes, 04/21/2021 03:21 PM

 
2021-04-21 09:48:26.853 7f2e2b3fc9c0 0 deferred set uid:gid to 64045:64045 (ceph:ceph)
2021-04-21 09:48:26.853 7f2e2b3fc9c0 0 ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable), process radosgw, pid 102195
2021-04-21 09:48:26.869 7f2e2b3fc9c0 10 cannot find current period zonegroup using local zonegroup
2021-04-21 09:48:26.873 7f2e2b3fc9c0 10 Cannot find current period zone using local zone
2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 all 8 watchers are set, enabling cache
2021-04-21 09:48:27.073 7f2e0a06d700 2 RGWDataChangesLog::ChangesRenewThread: start
2021-04-21 09:48:27.077 7f2e0906b700 2 object expiration: start
2021-04-21 09:48:27.077 7f2e0986c700 2 garbage collection: garbage collection: start
2021-04-21 09:48:27.081 7f2df5043700 5 lifecycle: schedule life cycle next start time: Thu Apr 22 04:00:00 2021
2021-04-21 09:48:27.081 7f2df4041700 10 ERROR: can't get key: ret=-2
2021-04-21 09:48:27.081 7f2df4041700 5 ERROR: sync_all_users() returned ret=-2
2021-04-21 09:48:27.081 7f2e2b3fc9c0 0 starting handler: beast
2021-04-21 09:48:27.081 7f2e2b3fc9c0 4 frontend listening on 0.0.0.0:7480
2021-04-21 09:48:27.081 7f2e2b3fc9c0 4 frontend listening on [::]:7480
2021-04-21 09:48:27.085 7f2e2b3fc9c0 0 set uid:gid to 64045:64045 (ceph:ceph)
2021-04-21 09:48:27.085 7f2e2b3fc9c0 4 frontend spawning 512 threads
2021-04-21 09:48:27.089 7f2e0906b700 5 process_single_shard(): failed to acquire lock on obj_delete_at_hint.0000000000
2021-04-21 09:48:27.101 7f2e2b3fc9c0 1 mgrc service_daemon_register rgw.storage1 metadata {arch=x86_64,ceph_release=nautilus,ceph_version=ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable),ceph_version_short=14.2.20,cpu=Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz,distro=ubuntu,distro_description=Ubuntu 18.04.5 LTS,distro_version=18.04,frontend_config#0=beast port=7480,frontend_type#0=beast,hostname=storage1,kernel_description=#146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021,kernel_version=4.15.0-142-generic,mem_swap_kb=9804796,mem_total_kb=65916304,num_handles=1,os=Linux,pid=102195,zone_id=default,zone_name=default,zonegroup_id=default,zonegroup_name=default}
2021-04-21 09:48:27.101 7f2e2b3fc9c0 4 rgw realm watcher: No realm, disabling dynamic reconfiguration.
2021-04-21 09:48:27.129 7f2df3840700 0 RGWReshardLock::lock failed to acquire lock on reshard.0000000000 ret=-16
2021-04-21 09:48:27.129 7f2df3840700 5 process_single_logshard(): failed to acquire lock on reshard.0000000000, ret = -16
2021-04-21 09:48:27.701 7f2e0986c700 10 garbage collection: RGWGC::process failed to acquire lock on gc.24
2021-04-21 09:48:28.729 7f2e0986c700 2 garbage collection: garbage collection: stop
2021-04-21 09:48:35.305 7f2e0906b700 2 object expiration: stop
2021-04-21 09:48:49.073 7f2e0a06d700 2 RGWDataChangesLog::ChangesRenewThread: start
2021-04-21 09:49:01.962 7f2df203d700 1 ====== starting new request req=0x55b8e3a36880 =====
2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s initializing for trans_id = tx000000000000000000001-0060802d4d-4c54ae2-default
2021-04-21 09:49:01.962 7f2df203d700 10 rgw api priority: s3=8 s3website=7
2021-04-21 09:49:01.962 7f2df203d700 10 host=<.........>
2021-04-21 09:49:01.962 7f2df203d700 10 handler=22RGWHandler_REST_Obj_S3
2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s getting op 0
2021-04-21 09:49:01.962 7f2df203d700 10 req 1 0.000s s3:get_obj scheduling with dmclock client=2 cost=1
2021-04-21 09:49:01.962 7f2df203d700 10 op=21RGWGetObj_ObjStore_S3
2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj verifying requester
2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj normalizing buckets and tenants
2021-04-21 09:49:01.962 7f2df203d700 10 s->object=testfile s->bucket=test
2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj init permissions
2021-04-21 09:49:01.962 7f2df203d700 10 cache get: name=.rgw++test : miss
2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++test info.flags=0x16
2021-04-21 09:49:01.966 7f2df203d700 10 adding .rgw++test to cache LRU end
2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++test : type miss (requested=0x13, cached=0x16)
2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++test info.flags=0x13
2021-04-21 09:49:01.966 7f2df203d700 10 moving .rgw++test to cache LRU end
2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++.bucket.meta.test:default.1021717.1 : miss
2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++.bucket.meta.test:default.1021717.1 info.flags=0x16
2021-04-21 09:49:01.966 7f2df203d700 10 adding .rgw++.bucket.meta.test:default.1021717.1 to cache LRU end
2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=user.rgw.acl bl.length()=149
2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++.bucket.meta.test:default.1021717.1 : type miss (requested=0x13, cached=0x16)
2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++.bucket.meta.test:default.1021717.1 info.flags=0x13
2021-04-21 09:49:01.966 7f2df203d700 10 moving .rgw++.bucket.meta.test:default.1021717.1 to cache LRU end
2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=user.rgw.acl bl.length()=149
2021-04-21 09:49:01.966 7f2df203d700 10 chain_cache_entry: cache_locator=.rgw++test
2021-04-21 09:49:01.966 7f2df203d700 10 chain_cache_entry: cache_locator=.rgw++.bucket.meta.test:default.1021717.1
2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.users.uid++anonymous : miss
2021-04-21 09:49:01.970 7f2df203d700 10 cache put: name=.users.uid++anonymous info.flags=0x0
2021-04-21 09:49:01.970 7f2df203d700 10 adding .users.uid++anonymous to cache LRU end
2021-04-21 09:49:01.970 7f2df203d700 2 req 1 0.008s s3:get_obj recalculating target
2021-04-21 09:49:01.970 7f2df203d700 2 req 1 0.008s s3:get_obj reading permissions
2021-04-21 09:49:01.974 7f2df203d700 10 manifest: total_size = 12
2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj init op
2021-04-21 09:49:01.974 7f2df203d700 10 cache get: name=.users.uid++jaharkes : miss
2021-04-21 09:49:01.974 7f2df203d700 10 cache put: name=.users.uid++jaharkes info.flags=0x6
2021-04-21 09:49:01.974 7f2df203d700 10 adding .users.uid++jaharkes to cache LRU end
2021-04-21 09:49:01.974 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
2021-04-21 09:49:01.974 7f2df203d700 10 cache get: name=.users.uid++jaharkes : type miss (requested=0x1, cached=0x6)
2021-04-21 09:49:01.974 7f2df203d700 10 cache put: name=.users.uid++jaharkes info.flags=0x1
2021-04-21 09:49:01.974 7f2df203d700 10 moving .users.uid++jaharkes to cache LRU end
2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op mask
2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op permissions
2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) mask=49
2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for uid=anonymous
2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for user not found
2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for group=1 mask=49
2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for group not found
2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0), owner=jaharkes, perm=0
2021-04-21 09:49:01.974 7f2df203d700 10 req 1 0.012s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=0, user_perm_mask=1, acl perm=0
2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) mask=49
2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for uid=anonymous
2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for user not found
2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for group=1 mask=49
2021-04-21 09:49:01.974 7f2df203d700 5 Found permission: 1
2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0), owner=jaharkes, perm=1
2021-04-21 09:49:01.974 7f2df203d700 10 req 1 0.012s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op params
2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj pre-executing
2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj executing
tcmalloc: large alloc 94252581314560 bytes == (nil) @ 0x7f2e2afc31e7 0x7f2e21e94815 0x7f2e21f12497 0x55b8e009d04f 0x55b8e02c5823 0x55b8e02ef33c 0x55b8e02ef5b9 0x55b8e026ed4c 0x55b8dff96210 0x55b8dff99278 0x55b8dfee14c7 0x55b8dfee29d9 0x55b8e04bc3ef
terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::current_exception_std_exception_wrapper<std::bad_alloc> >'
what(): std::bad_alloc
*** Caught signal (Aborted) **
in thread 7f2df203d700 thread_name:radosgw
ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable)
1: (()+0x12980) [0x7f2e1ffcc980]
2: (gsignal()+0xc7) [0x7f2e1f869fb7]
3: (abort()+0x141) [0x7f2e1f86b921]
4: (()+0x340ef9) [0x7f2e2185cef9]
5: (__cxxabiv1::__terminate(void (*)())+0x6) [0x7f2e21e94226]
6: (()+0x978291) [0x7f2e21e94291]
7: (()+0x9783e5) [0x7f2e21e943e5]
8: (virtual thunk to boost::exception_detail::clone_impl<boost::exception_detail::current_exception_std_exception_wrapper<std::bad_alloc> >::rethrow() const+0x6c) [0x55b8dfe236d8]
9: (boost::rethrow_exception(boost::exception_ptr const&)+0x12) [0x55b8dfef1e72]
10: (boost::coroutines::push_coroutine<void>::operator()()+0x69) [0x55b8dfef1f19]
11: (boost::asio::detail::executor_function<ceph::async::ForwardingHandler<ceph::async::CompletionHandler<boost::asio::detail::coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > >, std::tuple<boost::system::error_code, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > > > >, std::allocator<void> >::do_complete(boost::asio::detail::executor_function_base*, bool)+0x16a) [0x55b8dfefa00a]
12: (boost::asio::detail::executor_op<boost::asio::executor::function, std::allocator<void>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x5c) [0x55b8dff4a22c]
13: (boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>::operator()()+0x85) [0x55b8dff0a6a5]
14: (boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>, boost::asio::detail::recycling_allocator<void, boost::asio::detail::thread_info_base::default_tag>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x96) [0x55b8dff0aa06]
15: (boost::asio::detail::scheduler::run(boost::system::error_code&)+0x4f6) [0x55b8dfef6fe6]
16: (()+0x31c682) [0x55b8dfed6682]
17: (()+0x9f8e70) [0x7f2e21f14e70]
18: (()+0x76db) [0x7f2e1ffc16db]
19: (clone()+0x3f) [0x7f2e1f94c71f]
2021-04-21 09:49:01.982 7f2df203d700 -1 *** Caught signal (Aborted) **
in thread 7f2df203d700 thread_name:radosgw

ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable)
1: (()+0x12980) [0x7f2e1ffcc980]
2: (gsignal()+0xc7) [0x7f2e1f869fb7]
3: (abort()+0x141) [0x7f2e1f86b921]
4: (()+0x340ef9) [0x7f2e2185cef9]
5: (__cxxabiv1::__terminate(void (*)())+0x6) [0x7f2e21e94226]
6: (()+0x978291) [0x7f2e21e94291]
7: (()+0x9783e5) [0x7f2e21e943e5]
8: (virtual thunk to boost::exception_detail::clone_impl<boost::exception_detail::current_exception_std_exception_wrapper<std::bad_alloc> >::rethrow() const+0x6c) [0x55b8dfe236d8]
9: (boost::rethrow_exception(boost::exception_ptr const&)+0x12) [0x55b8dfef1e72]
10: (boost::coroutines::push_coroutine<void>::operator()()+0x69) [0x55b8dfef1f19]
11: (boost::asio::detail::executor_function<ceph::async::ForwardingHandler<ceph::async::CompletionHandler<boost::asio::detail::coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > >, std::tuple<boost::system::error_code, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > > > >, std::allocator<void> >::do_complete(boost::asio::detail::executor_function_base*, bool)+0x16a) [0x55b8dfefa00a]
12: (boost::asio::detail::executor_op<boost::asio::executor::function, std::allocator<void>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x5c) [0x55b8dff4a22c]
13: (boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>::operator()()+0x85) [0x55b8dff0a6a5]
14: (boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>, boost::asio::detail::recycling_allocator<void, boost::asio::detail::thread_info_base::default_tag>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x96) [0x55b8dff0aa06]
15: (boost::asio::detail::scheduler::run(boost::system::error_code&)+0x4f6) [0x55b8dfef6fe6]
16: (()+0x31c682) [0x55b8dfed6682]
17: (()+0x9f8e70) [0x7f2e21f14e70]
18: (()+0x76db) [0x7f2e1ffc16db]
19: (clone()+0x3f) [0x7f2e1f94c71f]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
-603> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command assert hook 0x55b8e286a5b0
-602> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command abort hook 0x55b8e286a5b0
-601> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perfcounters_dump hook 0x55b8e286a5b0
-600> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command 1 hook 0x55b8e286a5b0
-599> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf dump hook 0x55b8e286a5b0
-598> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perfcounters_schema hook 0x55b8e286a5b0
-597> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf histogram dump hook 0x55b8e286a5b0
-596> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command 2 hook 0x55b8e286a5b0
-595> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf schema hook 0x55b8e286a5b0
-594> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf histogram schema hook 0x55b8e286a5b0
-593> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf reset hook 0x55b8e286a5b0
-592> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config show hook 0x55b8e286a5b0
-591> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config help hook 0x55b8e286a5b0
-590> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config set hook 0x55b8e286a5b0
-589> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config unset hook 0x55b8e286a5b0
-588> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config get hook 0x55b8e286a5b0
-587> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config diff hook 0x55b8e286a5b0
-586> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config diff get hook 0x55b8e286a5b0
-585> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command log flush hook 0x55b8e286a5b0
-584> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command log dump hook 0x55b8e286a5b0
-583> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command log reopen hook 0x55b8e286a5b0
-582> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command dump_mempools hook 0x55b8e351c068
-581> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 0 framework: beast
-580> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 0 framework conf key: port, val: 7480
-579> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: get_monmap_and_config
-578> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-577> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2021-04-21 09:48:26.849950
created 2021-04-21 09:48:26.849950
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-576> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding auth protocol: cephx
-575> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding auth protocol: cephx
-574> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding auth protocol: cephx
-573> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-572> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-571> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-570> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-569> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-568> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-567> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-566> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-565> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-564> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-563> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-562> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-561> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-560> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: init
-559> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding auth protocol: cephx
-558> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding auth protocol: cephx
-557> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding auth protocol: cephx
-556> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-555> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-554> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-553> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-552> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-551> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-550> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-549> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-548> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-547> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-546> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-545> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-544> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-543> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-542> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-541> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-a con 0x55b8e3600000 addr [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0]
-540> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e3600480 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-539> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-d con 0x55b8e3600900 addr [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0]
-538> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-537> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-536> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-535> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-534> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:26.851505
-533> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e3600000 auth_method 0
-532> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-531> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): _init_auth method 2
-530> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-529> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-528> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-527> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-526> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e3600480 auth_method 0
-525> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-524> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): _init_auth method 2
-523> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-522> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e3600900 auth_method 0
-521> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-520> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): _init_auth method 2
-519> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-518> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-517> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-516> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-515> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_done global_id 80029120 payload 290
-514> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient: _finish_hunting 0
-513> 2021-04-21 09:48:26.849 7f2e16886700 1 monclient: found mon.noname-a
-512> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient: _send_mon_message to mon.noname-a at v2:10.1.1.95:3300/0
-511> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: handle_monmap mon_map magic: 0 v1
-510> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: got monmap 17 from mon.noname-a (according to old e17)
-509> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-508> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: _finish_auth 0
-507> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: handle_config config(17 keys) v1
-506> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: handle_monmap mon_map magic: 0 v1
-505> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: got monmap 17 from mon.storage1 (according to old e17)
-504> 2021-04-21 09:48:26.849 7f2e14081700 4 set_mon_vals no callback set
-503> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-502> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals device_failure_prediction_mode = local
-501> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_crush_update_on_start = false
-500> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_pg_autoscale_mode = on
-499> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_pg_num = 16
-498> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_pgp_num = 16
-497> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_size = 3
-496> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rbd_cache = true
-495> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rbd_cache_writethrough_until_flush = true
-494> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rbd_default_format = 2
-493> 2021-04-21 09:48:26.849 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80029120
-492> 2021-04-21 09:48:26.849 7f2e2b3fc9c0 10 monclient: get_monmap_and_config success
-491> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rgw_dns_name = <.....>
-490> 2021-04-21 09:48:26.849 7f2e2b3fc9c0 10 monclient: shutdown
-489> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 0 deferred set uid:gid to 64045:64045 (ceph:ceph)
-488> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 0 ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable), process radosgw, pid 102195
-487> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) init /var/run/ceph/ceph-client.rgw.storage1.102195.94252564165832.asok
-486> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) bind_and_listen /var/run/ceph/ceph-client.rgw.storage1.102195.94252564165832.asok
-485> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command 0 hook 0x55b8e28691c8
-484> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command version hook 0x55b8e28691c8
-483> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command git_version hook 0x55b8e28691c8
-482> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command help hook 0x55b8e286a9a0
-481> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command get_command_descriptions hook 0x55b8e286a990
-480> 2021-04-21 09:48:26.853 7f2e14882700 5 asok(0x55b8e29b0000) entry start
-479> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: get_monmap_and_config
-478> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-477> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-476> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-475> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-474> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-473> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-472> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-471> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-470> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-469> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-468> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-467> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-466> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-465> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-464> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-463> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-462> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-461> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-460> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: init
-459> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding auth protocol: cephx
-458> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding auth protocol: cephx
-457> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding auth protocol: cephx
-456> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-455> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-454> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-453> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-452> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-451> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-450> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-449> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-448> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-447> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-446> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-445> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-444> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-443> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-442> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-441> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-c con 0x55b8e3600900 addr [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0]
-440> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-b con 0x55b8e3600480 addr [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0]
-439> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e3600000 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-438> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-437> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-436> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-435> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-434> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:26.859296
-433> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e3600480 auth_method 0
-432> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-431> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): _init_auth method 2
-430> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-429> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e3600900 auth_method 0
-428> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-427> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): _init_auth method 2
-426> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-425> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e3600000 auth_method 0
-424> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-423> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): _init_auth method 2
-422> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-421> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-420> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-419> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-418> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-417> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-416> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-415> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-414> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-413> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-412> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_done global_id 80058227 payload 290
-411> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient: _finish_hunting 0
-410> 2021-04-21 09:48:26.857 7f2e16886700 1 monclient: found mon.noname-c
-409> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient: _send_mon_message to mon.noname-c at v2:10.1.1.98:3300/0
-408> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: handle_monmap mon_map magic: 0 v1
-407> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: got monmap 17 from mon.noname-c (according to old e17)
-406> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-405> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: _finish_auth 0
-404> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80058227
-403> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: handle_config config(17 keys) v1
-402> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: get_monmap_and_config success
-401> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: shutdown
-400> 2021-04-21 09:48:26.857 7f2e1087a700 4 set_mon_vals no callback set
-399> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: handle_monmap mon_map magic: 0 v1
-398> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: got monmap 17 from mon.storage3 (according to old e17)
-397> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-396> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-395> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-394> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-393> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-392> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-391> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-390> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-389> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-388> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-387> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-386> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-385> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-384> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-383> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-382> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-381> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-380> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-379> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-378> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: starting msgr at
-377> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: starting objecter
-376> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command objecter_requests hook 0x55b8e286a330
-375> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: setting wanted keys
-374> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: calling monclient init
-373> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: init
-372> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding auth protocol: cephx
-371> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding auth protocol: cephx
-370> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding auth protocol: cephx
-369> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-368> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-367> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-366> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-365> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-364> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-363> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-362> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-361> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-360> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-359> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-358> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-357> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-356> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-355> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-354> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-b con 0x55b8e3600900 addr [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0]
-353> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-c con 0x55b8e3600480 addr [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0]
-352> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e3600000 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-351> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-350> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-349> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-348> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-347> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:26.863315
-346> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e3600900 auth_method 0
-345> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-344> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): _init_auth method 2
-343> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-342> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e3600000 auth_method 0
-341> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-340> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): _init_auth method 2
-339> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-338> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e3600480 auth_method 0
-337> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-336> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): _init_auth method 2
-335> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-334> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-333> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-332> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-331> 2021-04-21 09:48:26.861 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-330> 2021-04-21 09:48:26.861 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-329> 2021-04-21 09:48:26.861 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-328> 2021-04-21 09:48:26.861 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-327> 2021-04-21 09:48:26.861 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-326> 2021-04-21 09:48:26.861 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-325> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient(hunting): handle_auth_done global_id 80038626 payload 872
-324> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient: _finish_hunting 0
-323> 2021-04-21 09:48:26.861 7f2e16886700 1 monclient: found mon.noname-b
-322> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient: _send_mon_message to mon.noname-b at v2:10.1.1.97:3300/0
-321> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: handle_monmap mon_map magic: 0 v1
-320> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: got monmap 17 from mon.noname-b (according to old e17)
-319> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-318> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: _finish_auth 0
-317> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: handle_config config(17 keys) v1
-316> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: handle_monmap mon_map magic: 0 v1
-315> 2021-04-21 09:48:26.861 7f2e10079700 4 set_mon_vals no callback set
-314> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: got monmap 17 from mon.storage2 (according to old e17)
-313> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-312> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80038626
-311> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _renew_subs
-310> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage2 at v2:10.1.1.97:3300/0
-309> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _renew_subs
-308> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage2 at v2:10.1.1.97:3300/0
-307> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 1 librados: init done
-306> 2021-04-21 09:48:26.865 7f2e1187c700 4 mgrc handle_mgr_map Got map version 1741
-305> 2021-04-21 09:48:26.865 7f2e1187c700 4 mgrc handle_mgr_map Active mgr is now [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-304> 2021-04-21 09:48:26.865 7f2e1187c700 4 mgrc reconnect Starting new session with [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-303> 2021-04-21 09:48:26.865 7f2e16085700 10 monclient: get_auth_request con 0x55b8e3602880 auth_method 0
-302> 2021-04-21 09:48:26.865 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3600d80 auth_method 0
-301> 2021-04-21 09:48:26.869 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3601200 auth_method 0
-300> 2021-04-21 09:48:26.869 7f2e2b3fc9c0 10 cannot find current period zonegroup using local zonegroup
-299> 2021-04-21 09:48:26.873 7f2e16085700 10 monclient: get_auth_request con 0x55b8e3601680 auth_method 0
-298> 2021-04-21 09:48:26.873 7f2e2b3fc9c0 10 Cannot find current period zone using local zone
-297> 2021-04-21 09:48:26.877 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3601f80 auth_method 0
-296> 2021-04-21 09:48:26.905 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3602400 auth_method 0
-295> 2021-04-21 09:48:26.965 7f2e16085700 10 monclient: get_auth_request con 0x55b8e36f1200 auth_method 0
-294> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 all 8 watchers are set, enabling cache
-293> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache list hook 0x55b8e29b5800
-292> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache inspect hook 0x55b8e29b5800
-291> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache erase hook 0x55b8e29b5800
-290> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache zap hook 0x55b8e29b5800
-289> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: get_monmap_and_config
-288> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-287> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-286> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-285> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-284> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-283> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-282> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-281> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-280> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-279> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-278> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-277> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-276> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-275> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-274> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-273> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-272> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-271> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-270> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: init
-269> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding auth protocol: cephx
-268> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding auth protocol: cephx
-267> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding auth protocol: cephx
-266> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-265> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-264> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-263> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-262> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-261> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-260> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-259> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-258> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-257> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-256> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-255> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-254> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-253> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-252> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-251> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-c con 0x55b8e36f1b00 addr [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0]
-250> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-b con 0x55b8e36f1f80 addr [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0]
-249> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e36f2400 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-248> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-247> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-246> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-245> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-244> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:27.067132
-243> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e36f1f80 auth_method 0
-242> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-241> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): _init_auth method 2
-240> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-239> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e36f2400 auth_method 0
-238> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-237> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): _init_auth method 2
-236> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-235> 2021-04-21 09:48:27.061 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e36f1b00 auth_method 0
-234> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-233> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): _init_auth method 2
-232> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-231> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-230> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-229> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-228> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-227> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-226> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-225> 2021-04-21 09:48:27.065 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-224> 2021-04-21 09:48:27.065 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-223> 2021-04-21 09:48:27.065 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-222> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_done global_id 80038631 payload 290
-221> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient: _finish_hunting 0
-220> 2021-04-21 09:48:27.065 7f2e16085700 1 monclient: found mon.noname-b
-219> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient: _send_mon_message to mon.noname-b at v2:10.1.1.97:3300/0
-218> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: got monmap 17 from mon.noname-b (according to old e17)
-216> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-215> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: _finish_auth 0
-214> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: handle_config config(17 keys) v1
-213> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: handle_monmap mon_map magic: 0 v1
-212> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: got monmap 17 from mon.storage2 (according to old e17)
-211> 2021-04-21 09:48:27.065 7f2e0c872700 4 set_mon_vals no callback set
-210> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-209> 2021-04-21 09:48:27.065 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80038631
-208> 2021-04-21 09:48:27.065 7f2e2b3fc9c0 10 monclient: get_monmap_and_config success
-207> 2021-04-21 09:48:27.065 7f2e2b3fc9c0 10 monclient: shutdown
-206> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-205> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-204> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-203> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-202> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-201> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-200> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-199> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-198> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-197> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-196> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-195> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-194> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-193> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-192> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-191> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-190> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-189> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-188> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: starting msgr at
-187> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: starting objecter
-186> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command objecter_requests hook 0x55b8e286aed0 EEXIST
-185> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: setting wanted keys
-184> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: calling monclient init
-183> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: init
-182> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding auth protocol: cephx
-181> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding auth protocol: cephx
-180> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding auth protocol: cephx
-179> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-178> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-177> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-176> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-175> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-174> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-173> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-172> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-171> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-170> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-169> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-168> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-167> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-166> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-165> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-164> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e36f2400 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-163> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-a con 0x55b8e36f1f80 addr [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0]
-162> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-d con 0x55b8e36f1b00 addr [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0]
-161> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-160> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-159> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-158> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-157> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:27.073875
-156> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e36f1f80 auth_method 0
-155> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-154> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): _init_auth method 2
-153> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-152> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e36f2400 auth_method 0
-151> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-150> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): _init_auth method 2
-149> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-148> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e36f1b00 auth_method 0
-147> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-146> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): _init_auth method 2
-145> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-144> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-143> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-142> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-141> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-140> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-139> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-138> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-137> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-136> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-135> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_done global_id 80029125 payload 872
-134> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient: _finish_hunting 0
-133> 2021-04-21 09:48:27.069 7f2e16085700 1 monclient: found mon.noname-a
-132> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient: _send_mon_message to mon.noname-a at v2:10.1.1.95:3300/0
-131> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: handle_monmap mon_map magic: 0 v1
-130> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: got monmap 17 from mon.noname-a (according to old e17)
-129> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-128> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: _finish_auth 0
-127> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80029125
-126> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _renew_subs
-125> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage1 at v2:10.1.1.95:3300/0
-124> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: handle_config config(17 keys) v1
-123> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: handle_monmap mon_map magic: 0 v1
-122> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: got monmap 17 from mon.storage1 (according to old e17)
-121> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-120> 2021-04-21 09:48:27.073 7f2e0c071700 4 set_mon_vals no callback set
-119> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _renew_subs
-118> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage1 at v2:10.1.1.95:3300/0
-117> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 1 librados: init done
-116> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cr dump hook 0x55b8e28c07f8
-115> 2021-04-21 09:48:27.073 7f2e0a06d700 2 RGWDataChangesLog::ChangesRenewThread: start
-114> 2021-04-21 09:48:27.077 7f2e0d874700 4 mgrc handle_mgr_map Got map version 1741
-113> 2021-04-21 09:48:27.077 7f2e0d874700 4 mgrc handle_mgr_map Active mgr is now [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-112> 2021-04-21 09:48:27.077 7f2e0d874700 4 mgrc reconnect Starting new session with [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-111> 2021-04-21 09:48:27.077 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3848000 auth_method 0
-110> 2021-04-21 09:48:27.077 7f2e0906b700 2 object expiration: start
-109> 2021-04-21 09:48:27.077 7f2e0986c700 2 garbage collection: garbage collection: start
-108> 2021-04-21 09:48:27.077 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3849b00 auth_method 0
-107> 2021-04-21 09:48:27.077 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3849f80 auth_method 0
-106> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace show hook 0x55b8e28c2680
-105> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace history hook 0x55b8e28c2680
-104> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace active hook 0x55b8e28c2680
-103> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace active_short hook 0x55b8e28c2680
-102> 2021-04-21 09:48:27.081 7f2df5043700 5 lifecycle: schedule life cycle next start time: Thu Apr 22 04:00:00 2021
-101> 2021-04-21 09:48:27.081 7f2df4041700 10 ERROR: can't get key: ret=-2
-100> 2021-04-21 09:48:27.081 7f2df4041700 5 ERROR: sync_all_users() returned ret=-2
-99> 2021-04-21 09:48:27.081 7f2e16085700 10 monclient: get_auth_request con 0x55b8e384a400 auth_method 0
-98> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 0 starting handler: beast
-97> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 4 frontend listening on 0.0.0.0:7480
-96> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 4 frontend listening on [::]:7480
-95> 2021-04-21 09:48:27.085 7f2e2b3fc9c0 0 set uid:gid to 64045:64045 (ceph:ceph)
-94> 2021-04-21 09:48:27.085 7f2e2b3fc9c0 4 frontend spawning 512 threads
-93> 2021-04-21 09:48:27.089 7f2e0906b700 5 process_single_shard(): failed to acquire lock on obj_delete_at_hint.0000000000
-92> 2021-04-21 09:48:27.101 7f2e2b3fc9c0 1 mgrc service_daemon_register rgw.storage1 metadata {arch=x86_64,ceph_release=nautilus,ceph_version=ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable),ceph_version_short=14.2.20,cpu=Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz,distro=ubuntu,distro_description=Ubuntu 18.04.5 LTS,distro_version=18.04,frontend_config#0=beast port=7480,frontend_type#0=beast,hostname=storage1,kernel_description=#146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021,kernel_version=4.15.0-142-generic,mem_swap_kb=9804796,mem_total_kb=65916304,num_handles=1,os=Linux,pid=102195,zone_id=default,zone_name=default,zonegroup_id=default,zonegroup_name=default}
-91> 2021-04-21 09:48:27.101 7f2e2b3fc9c0 4 rgw realm watcher: No realm, disabling dynamic reconfiguration.
-90> 2021-04-21 09:48:27.101 7f2e0d874700 4 mgrc handle_mgr_configure stats_period=5
-89> 2021-04-21 09:48:27.101 7f2e0d874700 4 mgrc handle_mgr_configure updated stats threshold: 5
-88> 2021-04-21 09:48:27.109 7f2e17087700 10 monclient: get_auth_request con 0x55b8e384a880 auth_method 0
-87> 2021-04-21 09:48:27.129 7f2df3840700 0 RGWReshardLock::lock failed to acquire lock on reshard.0000000000 ret=-16
-86> 2021-04-21 09:48:27.129 7f2df3840700 5 process_single_logshard(): failed to acquire lock on reshard.0000000000, ret = -16
-85> 2021-04-21 09:48:27.149 7f2e16886700 10 monclient: get_auth_request con 0x55b8e384b180 auth_method 0
-84> 2021-04-21 09:48:27.197 7f2e16085700 10 monclient: get_auth_request con 0x55b8e384b600 auth_method 0
-83> 2021-04-21 09:48:27.197 7f2e17087700 10 monclient: get_auth_request con 0x55b8e384ad00 auth_method 0
-82> 2021-04-21 09:48:27.701 7f2e0986c700 10 garbage collection: RGWGC::process failed to acquire lock on gc.24
-81> 2021-04-21 09:48:28.729 7f2e0986c700 2 garbage collection: garbage collection: stop
-80> 2021-04-21 09:48:29.857 7f2e1087a700 10 monclient: tick
-79> 2021-04-21 09:48:30.069 7f2e0c872700 10 monclient: tick
-78> 2021-04-21 09:48:31.861 7f2e1207d700 10 monclient: _renew_subs
-77> 2021-04-21 09:48:31.861 7f2e1207d700 10 monclient: _send_mon_message to mon.storage2 at v2:10.1.1.97:3300/0
-76> 2021-04-21 09:48:35.305 7f2e0906b700 2 object expiration: stop
-75> 2021-04-21 09:48:39.857 7f2e1087a700 10 monclient: tick
-74> 2021-04-21 09:48:40.069 7f2e0c872700 10 monclient: tick
-73> 2021-04-21 09:48:49.073 7f2e0a06d700 2 RGWDataChangesLog::ChangesRenewThread: start
-72> 2021-04-21 09:48:49.857 7f2e1087a700 10 monclient: tick
-71> 2021-04-21 09:48:50.069 7f2e0c872700 10 monclient: tick
-70> 2021-04-21 09:48:59.858 7f2e1087a700 10 monclient: tick
-69> 2021-04-21 09:49:00.070 7f2e0c872700 10 monclient: tick
-68> 2021-04-21 09:49:01.962 7f2df203d700 1 ====== starting new request req=0x55b8e3a36880 =====
-67> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s initializing for trans_id = tx000000000000000000001-0060802d4d-4c54ae2-default
-66> 2021-04-21 09:49:01.962 7f2df203d700 10 rgw api priority: s3=8 s3website=7
-65> 2021-04-21 09:49:01.962 7f2df203d700 10 host=<.....>
-64> 2021-04-21 09:49:01.962 7f2df203d700 10 handler=22RGWHandler_REST_Obj_S3
-63> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s getting op 0
-62> 2021-04-21 09:49:01.962 7f2df203d700 10 req 1 0.000s s3:get_obj scheduling with dmclock client=2 cost=1
-61> 2021-04-21 09:49:01.962 7f2df203d700 10 op=21RGWGetObj_ObjStore_S3
-60> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj verifying requester
-59> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj normalizing buckets and tenants
-58> 2021-04-21 09:49:01.962 7f2df203d700 10 s->object=testfile s->bucket=test
-57> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj init permissions
-56> 2021-04-21 09:49:01.962 7f2df203d700 10 cache get: name=.rgw++test : miss
-55> 2021-04-21 09:49:01.962 7f2e16886700 10 monclient: get_auth_request con 0x55b8e384ba80 auth_method 0
-54> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++test info.flags=0x16
-53> 2021-04-21 09:49:01.966 7f2df203d700 10 adding .rgw++test to cache LRU end
-52> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-51> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++test : type miss (requested=0x13, cached=0x16)
-50> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++test info.flags=0x13
-49> 2021-04-21 09:49:01.966 7f2df203d700 10 moving .rgw++test to cache LRU end
-48> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-47> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++.bucket.meta.test:default.1021717.1 : miss
-46> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++.bucket.meta.test:default.1021717.1 info.flags=0x16
-45> 2021-04-21 09:49:01.966 7f2df203d700 10 adding .rgw++.bucket.meta.test:default.1021717.1 to cache LRU end
-44> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-43> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=user.rgw.acl bl.length()=149
-42> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++.bucket.meta.test:default.1021717.1 : type miss (requested=0x13, cached=0x16)
-41> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++.bucket.meta.test:default.1021717.1 info.flags=0x13
-40> 2021-04-21 09:49:01.966 7f2df203d700 10 moving .rgw++.bucket.meta.test:default.1021717.1 to cache LRU end
-39> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-38> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=user.rgw.acl bl.length()=149
-37> 2021-04-21 09:49:01.966 7f2df203d700 10 chain_cache_entry: cache_locator=.rgw++test
-36> 2021-04-21 09:49:01.966 7f2df203d700 10 chain_cache_entry: cache_locator=.rgw++.bucket.meta.test:default.1021717.1
-35> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.users.uid++anonymous : miss
-34> 2021-04-21 09:49:01.970 7f2e16085700 10 monclient: get_auth_request con 0x55b8e3a5e000 auth_method 0
-33> 2021-04-21 09:49:01.970 7f2df203d700 10 cache put: name=.users.uid++anonymous info.flags=0x0
-32> 2021-04-21 09:49:01.970 7f2df203d700 10 adding .users.uid++anonymous to cache LRU end
-31> 2021-04-21 09:49:01.970 7f2df203d700 2 req 1 0.008s s3:get_obj recalculating target
-30> 2021-04-21 09:49:01.970 7f2df203d700 2 req 1 0.008s s3:get_obj reading permissions
-29> 2021-04-21 09:49:01.970 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3a5e480 auth_method 0
-28> 2021-04-21 09:49:01.974 7f2df203d700 10 manifest: total_size = 12
-27> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj init op
-26> 2021-04-21 09:49:01.974 7f2df203d700 10 cache get: name=.users.uid++jaharkes : miss
-25> 2021-04-21 09:49:01.974 7f2df203d700 10 cache put: name=.users.uid++jaharkes info.flags=0x6
-24> 2021-04-21 09:49:01.974 7f2df203d700 10 adding .users.uid++jaharkes to cache LRU end
-23> 2021-04-21 09:49:01.974 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-22> 2021-04-21 09:49:01.974 7f2df203d700 10 cache get: name=.users.uid++jaharkes : type miss (requested=0x1, cached=0x6)
-21> 2021-04-21 09:49:01.974 7f2df203d700 10 cache put: name=.users.uid++jaharkes info.flags=0x1
-20> 2021-04-21 09:49:01.974 7f2df203d700 10 moving .users.uid++jaharkes to cache LRU end
-19> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op mask
-18> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op permissions
-17> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) mask=49
-16> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for uid=anonymous
-15> 2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for user not found
-14> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for group=1 mask=49
-13> 2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for group not found
-12> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0), owner=jaharkes, perm=0
-11> 2021-04-21 09:49:01.974 7f2df203d700 10 req 1 0.012s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=0, user_perm_mask=1, acl perm=0
-10> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) mask=49
-9> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for uid=anonymous
-8> 2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for user not found
-7> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for group=1 mask=49
-6> 2021-04-21 09:49:01.974 7f2df203d700 5 Found permission: 1
-5> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0), owner=jaharkes, perm=1
-4> 2021-04-21 09:49:01.974 7f2df203d700 10 req 1 0.012s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
-3> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op params
-2> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj pre-executing
-1> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj executing
0> 2021-04-21 09:49:01.982 7f2df203d700 -1 *** Caught signal (Aborted) **
in thread 7f2df203d700 thread_name:radosgw

ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable)
1: (()+0x12980) [0x7f2e1ffcc980]
2: (gsignal()+0xc7) [0x7f2e1f869fb7]
3: (abort()+0x141) [0x7f2e1f86b921]
4: (()+0x340ef9) [0x7f2e2185cef9]
5: (__cxxabiv1::__terminate(void (*)())+0x6) [0x7f2e21e94226]
6: (()+0x978291) [0x7f2e21e94291]
7: (()+0x9783e5) [0x7f2e21e943e5]
8: (virtual thunk to boost::exception_detail::clone_impl<boost::exception_detail::current_exception_std_exception_wrapper<std::bad_alloc> >::rethrow() const+0x6c) [0x55b8dfe236d8]
9: (boost::rethrow_exception(boost::exception_ptr const&)+0x12) [0x55b8dfef1e72]
10: (boost::coroutines::push_coroutine<void>::operator()()+0x69) [0x55b8dfef1f19]
11: (boost::asio::detail::executor_function<ceph::async::ForwardingHandler<ceph::async::CompletionHandler<boost::asio::detail::coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > >, std::tuple<boost::system::error_code, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > > > >, std::allocator<void> >::do_complete(boost::asio::detail::executor_function_base*, bool)+0x16a) [0x55b8dfefa00a]
12: (boost::asio::detail::executor_op<boost::asio::executor::function, std::allocator<void>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x5c) [0x55b8dff4a22c]
13: (boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>::operator()()+0x85) [0x55b8dff0a6a5]
14: (boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>, boost::asio::detail::recycling_allocator<void, boost::asio::detail::thread_info_base::default_tag>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x96) [0x55b8dff0aa06]
15: (boost::asio::detail::scheduler::run(boost::system::error_code&)+0x4f6) [0x55b8dfef6fe6]
16: (()+0x31c682) [0x55b8dfed6682]
17: (()+0x9f8e70) [0x7f2e21f14e70]
18: (()+0x76db) [0x7f2e1ffc16db]
19: (clone()+0x3f) [0x7f2e1f94c71f]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
10/10 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
-2/-2 (syslog threshold)
99/99 (stderr threshold)
max_recent 10000
max_new 1000
log_file
--- end dump of recent events ---
--- begin dump of recent events ---
-603> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command assert hook 0x55b8e286a5b0
-602> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command abort hook 0x55b8e286a5b0
-601> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perfcounters_dump hook 0x55b8e286a5b0
-600> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command 1 hook 0x55b8e286a5b0
-599> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf dump hook 0x55b8e286a5b0
-598> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perfcounters_schema hook 0x55b8e286a5b0
-597> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf histogram dump hook 0x55b8e286a5b0
-596> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command 2 hook 0x55b8e286a5b0
-595> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf schema hook 0x55b8e286a5b0
-594> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf histogram schema hook 0x55b8e286a5b0
-593> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command perf reset hook 0x55b8e286a5b0
-592> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config show hook 0x55b8e286a5b0
-591> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config help hook 0x55b8e286a5b0
-590> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config set hook 0x55b8e286a5b0
-589> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config unset hook 0x55b8e286a5b0
-588> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config get hook 0x55b8e286a5b0
-587> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config diff hook 0x55b8e286a5b0
-586> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command config diff get hook 0x55b8e286a5b0
-585> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command log flush hook 0x55b8e286a5b0
-584> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command log dump hook 0x55b8e286a5b0
-583> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command log reopen hook 0x55b8e286a5b0
-582> 2021-04-21 09:48:26.837 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command dump_mempools hook 0x55b8e351c068
-581> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 0 framework: beast
-580> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 0 framework conf key: port, val: 7480
-579> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: get_monmap_and_config
-578> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-577> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2021-04-21 09:48:26.849950
created 2021-04-21 09:48:26.849950
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-576> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding auth protocol: cephx
-575> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding auth protocol: cephx
-574> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding auth protocol: cephx
-573> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-572> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-571> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-570> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-569> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-568> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-567> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-566> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-565> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-564> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-563> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: crc
-562> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2956a40) adding con mode: secure
-561> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-560> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: init
-559> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding auth protocol: cephx
-558> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding auth protocol: cephx
-557> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding auth protocol: cephx
-556> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-555> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-554> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-553> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-552> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-551> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-550> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-549> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-548> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-547> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-546> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: crc
-545> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050fe838) adding con mode: secure
-544> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-543> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-542> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-541> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-a con 0x55b8e3600000 addr [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0]
-540> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e3600480 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-539> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-d con 0x55b8e3600900 addr [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0]
-538> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-537> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-536> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-535> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-534> 2021-04-21 09:48:26.845 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:26.851505
-533> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e3600000 auth_method 0
-532> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-531> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): _init_auth method 2
-530> 2021-04-21 09:48:26.845 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-529> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-528> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-527> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-526> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e3600480 auth_method 0
-525> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-524> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): _init_auth method 2
-523> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-522> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e3600900 auth_method 0
-521> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-520> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): _init_auth method 2
-519> 2021-04-21 09:48:26.849 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-518> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-517> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-516> 2021-04-21 09:48:26.849 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-515> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient(hunting): handle_auth_done global_id 80029120 payload 290
-514> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient: _finish_hunting 0
-513> 2021-04-21 09:48:26.849 7f2e16886700 1 monclient: found mon.noname-a
-512> 2021-04-21 09:48:26.849 7f2e16886700 10 monclient: _send_mon_message to mon.noname-a at v2:10.1.1.95:3300/0
-511> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: handle_monmap mon_map magic: 0 v1
-510> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: got monmap 17 from mon.noname-a (according to old e17)
-509> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-508> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: _finish_auth 0
-507> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: handle_config config(17 keys) v1
-506> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: handle_monmap mon_map magic: 0 v1
-505> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: got monmap 17 from mon.storage1 (according to old e17)
-504> 2021-04-21 09:48:26.849 7f2e14081700 4 set_mon_vals no callback set
-503> 2021-04-21 09:48:26.849 7f2e15884700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-502> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals device_failure_prediction_mode = local
-501> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_crush_update_on_start = false
-500> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_pg_autoscale_mode = on
-499> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_pg_num = 16
-498> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_pgp_num = 16
-497> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals osd_pool_default_size = 3
-496> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rbd_cache = true
-495> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rbd_cache_writethrough_until_flush = true
-494> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rbd_default_format = 2
-493> 2021-04-21 09:48:26.849 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80029120
-492> 2021-04-21 09:48:26.849 7f2e2b3fc9c0 10 monclient: get_monmap_and_config success
-491> 2021-04-21 09:48:26.849 7f2e14081700 10 set_mon_vals rgw_dns_name = <......>
-490> 2021-04-21 09:48:26.849 7f2e2b3fc9c0 10 monclient: shutdown
-489> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 0 deferred set uid:gid to 64045:64045 (ceph:ceph)
-488> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 0 ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable), process radosgw, pid 102195
-487> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) init /var/run/ceph/ceph-client.rgw.storage1.102195.94252564165832.asok
-486> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) bind_and_listen /var/run/ceph/ceph-client.rgw.storage1.102195.94252564165832.asok
-485> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command 0 hook 0x55b8e28691c8
-484> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command version hook 0x55b8e28691c8
-483> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command git_version hook 0x55b8e28691c8
-482> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command help hook 0x55b8e286a9a0
-481> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command get_command_descriptions hook 0x55b8e286a990
-480> 2021-04-21 09:48:26.853 7f2e14882700 5 asok(0x55b8e29b0000) entry start
-479> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: get_monmap_and_config
-478> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-477> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-476> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-475> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-474> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-473> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-472> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-471> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-470> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-469> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-468> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-467> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-466> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-465> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-464> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-463> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-462> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-461> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-460> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: init
-459> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding auth protocol: cephx
-458> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding auth protocol: cephx
-457> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding auth protocol: cephx
-456> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-455> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-454> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-453> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-452> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-451> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-450> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-449> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-448> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-447> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-446> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: crc
-445> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff1c8) adding con mode: secure
-444> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-443> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-442> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-441> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-c con 0x55b8e3600900 addr [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0]
-440> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-b con 0x55b8e3600480 addr [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0]
-439> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e3600000 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-438> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-437> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-436> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-435> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-434> 2021-04-21 09:48:26.853 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:26.859296
-433> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e3600480 auth_method 0
-432> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-431> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): _init_auth method 2
-430> 2021-04-21 09:48:26.853 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-429> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e3600900 auth_method 0
-428> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-427> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): _init_auth method 2
-426> 2021-04-21 09:48:26.853 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-425> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e3600000 auth_method 0
-424> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-423> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): _init_auth method 2
-422> 2021-04-21 09:48:26.853 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-421> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-420> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-419> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-418> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-417> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-416> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-415> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-414> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-413> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-412> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_done global_id 80058227 payload 290
-411> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient: _finish_hunting 0
-410> 2021-04-21 09:48:26.857 7f2e16886700 1 monclient: found mon.noname-c
-409> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient: _send_mon_message to mon.noname-c at v2:10.1.1.98:3300/0
-408> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: handle_monmap mon_map magic: 0 v1
-407> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: got monmap 17 from mon.noname-c (according to old e17)
-406> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-405> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: _finish_auth 0
-404> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80058227
-403> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: handle_config config(17 keys) v1
-402> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: get_monmap_and_config success
-401> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: shutdown
-400> 2021-04-21 09:48:26.857 7f2e1087a700 4 set_mon_vals no callback set
-399> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: handle_monmap mon_map magic: 0 v1
-398> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: got monmap 17 from mon.storage3 (according to old e17)
-397> 2021-04-21 09:48:26.857 7f2e1207d700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-396> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-395> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-394> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-393> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-392> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding auth protocol: cephx
-391> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-390> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-389> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-388> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-387> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-386> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-385> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-384> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-383> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-382> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-381> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: crc
-380> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2957340) adding con mode: secure
-379> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-378> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: starting msgr at
-377> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: starting objecter
-376> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command objecter_requests hook 0x55b8e286a330
-375> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: setting wanted keys
-374> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 1 librados: calling monclient init
-373> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: init
-372> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding auth protocol: cephx
-371> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding auth protocol: cephx
-370> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding auth protocol: cephx
-369> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-368> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-367> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-366> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-365> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-364> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-363> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-362> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-361> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-360> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-359> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: crc
-358> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e3615078) adding con mode: secure
-357> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-356> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-355> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-354> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-b con 0x55b8e3600900 addr [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0]
-353> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-c con 0x55b8e3600480 addr [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0]
-352> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e3600000 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-351> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-350> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-349> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-348> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-347> 2021-04-21 09:48:26.857 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:26.863315
-346> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e3600900 auth_method 0
-345> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-344> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): _init_auth method 2
-343> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-342> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e3600000 auth_method 0
-341> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-340> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): _init_auth method 2
-339> 2021-04-21 09:48:26.857 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-338> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e3600480 auth_method 0
-337> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-336> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): _init_auth method 2
-335> 2021-04-21 09:48:26.857 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-334> 2021-04-21 09:48:26.857 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-333> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-332> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-331> 2021-04-21 09:48:26.861 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-330> 2021-04-21 09:48:26.861 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-329> 2021-04-21 09:48:26.861 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-328> 2021-04-21 09:48:26.861 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-327> 2021-04-21 09:48:26.861 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-326> 2021-04-21 09:48:26.861 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-325> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient(hunting): handle_auth_done global_id 80038626 payload 872
-324> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient: _finish_hunting 0
-323> 2021-04-21 09:48:26.861 7f2e16886700 1 monclient: found mon.noname-b
-322> 2021-04-21 09:48:26.861 7f2e16886700 10 monclient: _send_mon_message to mon.noname-b at v2:10.1.1.97:3300/0
-321> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: handle_monmap mon_map magic: 0 v1
-320> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: got monmap 17 from mon.noname-b (according to old e17)
-319> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-318> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: _finish_auth 0
-317> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: handle_config config(17 keys) v1
-316> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: handle_monmap mon_map magic: 0 v1
-315> 2021-04-21 09:48:26.861 7f2e10079700 4 set_mon_vals no callback set
-314> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: got monmap 17 from mon.storage2 (according to old e17)
-313> 2021-04-21 09:48:26.861 7f2e1187c700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-312> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80038626
-311> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _renew_subs
-310> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage2 at v2:10.1.1.97:3300/0
-309> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _renew_subs
-308> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage2 at v2:10.1.1.97:3300/0
-307> 2021-04-21 09:48:26.861 7f2e2b3fc9c0 1 librados: init done
-306> 2021-04-21 09:48:26.865 7f2e1187c700 4 mgrc handle_mgr_map Got map version 1741
-305> 2021-04-21 09:48:26.865 7f2e1187c700 4 mgrc handle_mgr_map Active mgr is now [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-304> 2021-04-21 09:48:26.865 7f2e1187c700 4 mgrc reconnect Starting new session with [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-303> 2021-04-21 09:48:26.865 7f2e16085700 10 monclient: get_auth_request con 0x55b8e3602880 auth_method 0
-302> 2021-04-21 09:48:26.865 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3600d80 auth_method 0
-301> 2021-04-21 09:48:26.869 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3601200 auth_method 0
-300> 2021-04-21 09:48:26.869 7f2e2b3fc9c0 10 cannot find current period zonegroup using local zonegroup
-299> 2021-04-21 09:48:26.873 7f2e16085700 10 monclient: get_auth_request con 0x55b8e3601680 auth_method 0
-298> 2021-04-21 09:48:26.873 7f2e2b3fc9c0 10 Cannot find current period zone using local zone
-297> 2021-04-21 09:48:26.877 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3601f80 auth_method 0
-296> 2021-04-21 09:48:26.905 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3602400 auth_method 0
-295> 2021-04-21 09:48:26.965 7f2e16085700 10 monclient: get_auth_request con 0x55b8e36f1200 auth_method 0
-294> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 all 8 watchers are set, enabling cache
-293> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache list hook 0x55b8e29b5800
-292> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache inspect hook 0x55b8e29b5800
-291> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache erase hook 0x55b8e29b5800
-290> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cache zap hook 0x55b8e29b5800
-289> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: get_monmap_and_config
-288> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-287> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-286> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-285> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-284> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-283> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-282> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-281> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-280> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-279> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-278> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-277> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-276> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-275> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-274> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-273> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-272> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-271> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-270> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: init
-269> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding auth protocol: cephx
-268> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding auth protocol: cephx
-267> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding auth protocol: cephx
-266> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-265> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-264> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-263> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-262> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-261> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-260> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-259> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-258> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-257> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-256> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: crc
-255> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 5 AuthRegistry(0x7fff050ff648) adding con mode: secure
-254> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-253> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-252> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-251> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-c con 0x55b8e36f1b00 addr [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0]
-250> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-b con 0x55b8e36f1f80 addr [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0]
-249> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e36f2400 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-248> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-247> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-246> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-245> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-244> 2021-04-21 09:48:27.061 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:27.067132
-243> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e36f1f80 auth_method 0
-242> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-241> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): _init_auth method 2
-240> 2021-04-21 09:48:27.061 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-239> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e36f2400 auth_method 0
-238> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-237> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): _init_auth method 2
-236> 2021-04-21 09:48:27.061 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-235> 2021-04-21 09:48:27.061 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e36f1b00 auth_method 0
-234> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-233> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): _init_auth method 2
-232> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-231> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-230> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-229> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-228> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-227> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-226> 2021-04-21 09:48:27.065 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-225> 2021-04-21 09:48:27.065 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-224> 2021-04-21 09:48:27.065 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-223> 2021-04-21 09:48:27.065 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-222> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient(hunting): handle_auth_done global_id 80038631 payload 290
-221> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient: _finish_hunting 0
-220> 2021-04-21 09:48:27.065 7f2e16085700 1 monclient: found mon.noname-b
-219> 2021-04-21 09:48:27.065 7f2e16085700 10 monclient: _send_mon_message to mon.noname-b at v2:10.1.1.97:3300/0
-218> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: handle_monmap mon_map magic: 0 v1
-217> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: got monmap 17 from mon.noname-b (according to old e17)
-216> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-215> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: _finish_auth 0
-214> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: handle_config config(17 keys) v1
-213> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: handle_monmap mon_map magic: 0 v1
-212> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: got monmap 17 from mon.storage2 (according to old e17)
-211> 2021-04-21 09:48:27.065 7f2e0c872700 4 set_mon_vals no callback set
-210> 2021-04-21 09:48:27.065 7f2e0e075700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-209> 2021-04-21 09:48:27.065 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80038631
-208> 2021-04-21 09:48:27.065 7f2e2b3fc9c0 10 monclient: get_monmap_and_config success
-207> 2021-04-21 09:48:27.065 7f2e2b3fc9c0 10 monclient: shutdown
-206> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: build_initial_monmap
-205> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: monmap:
epoch 0
fsid 00000000-0000-0000-0000-000000000000
last_changed 0.000000
created 0.000000
min_mon_release 0 (unknown)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.noname-a
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.noname-b
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.noname-c
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.noname-d
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.noname-e

-204> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-203> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-202> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding auth protocol: cephx
-201> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-200> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-199> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-198> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-197> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-196> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-195> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-194> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-193> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-192> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-191> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: crc
-190> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e2958540) adding con mode: secure
-189> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-188> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: starting msgr at
-187> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: starting objecter
-186> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command objecter_requests hook 0x55b8e286aed0 EEXIST
-185> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: setting wanted keys
-184> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 1 librados: calling monclient init
-183> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: init
-182> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding auth protocol: cephx
-181> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding auth protocol: cephx
-180> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding auth protocol: cephx
-179> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-178> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-177> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-176> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-175> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-174> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-173> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-172> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-171> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-170> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-169> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: crc
-168> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 AuthRegistry(0x55b8e367e078) adding con mode: secure
-167> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-166> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 2 auth: KeyRing::load: loaded key file /var/lib/ceph/radosgw/ceph-rgw.storage1/keyring
-165> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient: _reopen_session rank -1
-164> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-e con 0x55b8e36f2400 addr [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0]
-163> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-a con 0x55b8e36f1f80 addr [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0]
-162> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): picked mon.noname-d con 0x55b8e36f1b00 addr [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0]
-161> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-160> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-159> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): start opening mon connection
-158> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): _renew_subs
-157> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 10 monclient(hunting): authenticate will time out at 2021-04-21 09:53:27.073875
-156> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): get_auth_request con 0x55b8e36f1f80 auth_method 0
-155> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-154> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): _init_auth method 2
-153> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): _init_auth creating new auth
-152> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): get_auth_request con 0x55b8e36f2400 auth_method 0
-151> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-150> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): _init_auth method 2
-149> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): _init_auth creating new auth
-148> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): get_auth_request con 0x55b8e36f1b00 auth_method 0
-147> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): get_auth_request method 2 preferred_modes [2,1]
-146> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): _init_auth method 2
-145> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): _init_auth creating new auth
-144> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload 9
-143> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-142> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-141> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload 9
-140> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-139> 2021-04-21 09:48:27.069 7f2e16886700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-138> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload 9
-137> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): handle_auth_reply_more payload_len 9
-136> 2021-04-21 09:48:27.069 7f2e17087700 10 monclient(hunting): handle_auth_reply_more responding with 36 bytes
-135> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient(hunting): handle_auth_done global_id 80029125 payload 872
-134> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient: _finish_hunting 0
-133> 2021-04-21 09:48:27.069 7f2e16085700 1 monclient: found mon.noname-a
-132> 2021-04-21 09:48:27.069 7f2e16085700 10 monclient: _send_mon_message to mon.noname-a at v2:10.1.1.95:3300/0
-131> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: handle_monmap mon_map magic: 0 v1
-130> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: got monmap 17 from mon.noname-a (according to old e17)
-129> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-128> 2021-04-21 09:48:27.069 7f2e0d874700 10 monclient: _finish_auth 0
-127> 2021-04-21 09:48:27.069 7f2e2b3fc9c0 5 monclient: authenticate success, global_id 80029125
-126> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _renew_subs
-125> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage1 at v2:10.1.1.95:3300/0
-124> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: handle_config config(17 keys) v1
-123> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: handle_monmap mon_map magic: 0 v1
-122> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: got monmap 17 from mon.storage1 (according to old e17)
-121> 2021-04-21 09:48:27.073 7f2e0d874700 10 monclient: dump:
epoch 17
fsid 33affea0-070b-4082-a07e-69825a474bf1
last_changed 2019-10-07 16:08:51.540056
created 2013-10-25 16:48:20.232017
min_mon_release 14 (nautilus)
0: [v2:10.1.1.95:3300/0,v1:10.1.1.95:6789/0] mon.storage1
1: [v2:10.1.1.97:3300/0,v1:10.1.1.97:6789/0] mon.storage2
2: [v2:10.1.1.98:3300/0,v1:10.1.1.98:6789/0] mon.storage3
3: [v2:10.1.1.99:3300/0,v1:10.1.1.99:6789/0] mon.storage4
4: [v2:10.1.1.100:3300/0,v1:10.1.1.100:6789/0] mon.storage5

-120> 2021-04-21 09:48:27.073 7f2e0c071700 4 set_mon_vals no callback set
-119> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _renew_subs
-118> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 10 monclient: _send_mon_message to mon.storage1 at v2:10.1.1.95:3300/0
-117> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 1 librados: init done
-116> 2021-04-21 09:48:27.073 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command cr dump hook 0x55b8e28c07f8
-115> 2021-04-21 09:48:27.073 7f2e0a06d700 2 RGWDataChangesLog::ChangesRenewThread: start
-114> 2021-04-21 09:48:27.077 7f2e0d874700 4 mgrc handle_mgr_map Got map version 1741
-113> 2021-04-21 09:48:27.077 7f2e0d874700 4 mgrc handle_mgr_map Active mgr is now [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-112> 2021-04-21 09:48:27.077 7f2e0d874700 4 mgrc reconnect Starting new session with [v2:10.1.1.95:6848/1559,v1:10.1.1.95:6849/1559]
-111> 2021-04-21 09:48:27.077 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3848000 auth_method 0
-110> 2021-04-21 09:48:27.077 7f2e0906b700 2 object expiration: start
-109> 2021-04-21 09:48:27.077 7f2e0986c700 2 garbage collection: garbage collection: start
-108> 2021-04-21 09:48:27.077 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3849b00 auth_method 0
-107> 2021-04-21 09:48:27.077 7f2e16886700 10 monclient: get_auth_request con 0x55b8e3849f80 auth_method 0
-106> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace show hook 0x55b8e28c2680
-105> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace history hook 0x55b8e28c2680
-104> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace active hook 0x55b8e28c2680
-103> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 5 asok(0x55b8e29b0000) register_command sync trace active_short hook 0x55b8e28c2680
-102> 2021-04-21 09:48:27.081 7f2df5043700 5 lifecycle: schedule life cycle next start time: Thu Apr 22 04:00:00 2021
-101> 2021-04-21 09:48:27.081 7f2df4041700 10 ERROR: can't get key: ret=-2
-100> 2021-04-21 09:48:27.081 7f2df4041700 5 ERROR: sync_all_users() returned ret=-2
-99> 2021-04-21 09:48:27.081 7f2e16085700 10 monclient: get_auth_request con 0x55b8e384a400 auth_method 0
-98> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 0 starting handler: beast
-97> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 4 frontend listening on 0.0.0.0:7480
-96> 2021-04-21 09:48:27.081 7f2e2b3fc9c0 4 frontend listening on [::]:7480
-95> 2021-04-21 09:48:27.085 7f2e2b3fc9c0 0 set uid:gid to 64045:64045 (ceph:ceph)
-94> 2021-04-21 09:48:27.085 7f2e2b3fc9c0 4 frontend spawning 512 threads
-93> 2021-04-21 09:48:27.089 7f2e0906b700 5 process_single_shard(): failed to acquire lock on obj_delete_at_hint.0000000000
-92> 2021-04-21 09:48:27.101 7f2e2b3fc9c0 1 mgrc service_daemon_register rgw.storage1 metadata {arch=x86_64,ceph_release=nautilus,ceph_version=ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable),ceph_version_short=14.2.20,cpu=Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz,distro=ubuntu,distro_description=Ubuntu 18.04.5 LTS,distro_version=18.04,frontend_config#0=beast port=7480,frontend_type#0=beast,hostname=storage1,kernel_description=#146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021,kernel_version=4.15.0-142-generic,mem_swap_kb=9804796,mem_total_kb=65916304,num_handles=1,os=Linux,pid=102195,zone_id=default,zone_name=default,zonegroup_id=default,zonegroup_name=default}
-91> 2021-04-21 09:48:27.101 7f2e2b3fc9c0 4 rgw realm watcher: No realm, disabling dynamic reconfiguration.
-90> 2021-04-21 09:48:27.101 7f2e0d874700 4 mgrc handle_mgr_configure stats_period=5
-89> 2021-04-21 09:48:27.101 7f2e0d874700 4 mgrc handle_mgr_configure updated stats threshold: 5
-88> 2021-04-21 09:48:27.109 7f2e17087700 10 monclient: get_auth_request con 0x55b8e384a880 auth_method 0
-87> 2021-04-21 09:48:27.129 7f2df3840700 0 RGWReshardLock::lock failed to acquire lock on reshard.0000000000 ret=-16
-86> 2021-04-21 09:48:27.129 7f2df3840700 5 process_single_logshard(): failed to acquire lock on reshard.0000000000, ret = -16
-85> 2021-04-21 09:48:27.149 7f2e16886700 10 monclient: get_auth_request con 0x55b8e384b180 auth_method 0
-84> 2021-04-21 09:48:27.197 7f2e16085700 10 monclient: get_auth_request con 0x55b8e384b600 auth_method 0
-83> 2021-04-21 09:48:27.197 7f2e17087700 10 monclient: get_auth_request con 0x55b8e384ad00 auth_method 0
-82> 2021-04-21 09:48:27.701 7f2e0986c700 10 garbage collection: RGWGC::process failed to acquire lock on gc.24
-81> 2021-04-21 09:48:28.729 7f2e0986c700 2 garbage collection: garbage collection: stop
-80> 2021-04-21 09:48:29.857 7f2e1087a700 10 monclient: tick
-79> 2021-04-21 09:48:30.069 7f2e0c872700 10 monclient: tick
-78> 2021-04-21 09:48:31.861 7f2e1207d700 10 monclient: _renew_subs
-77> 2021-04-21 09:48:31.861 7f2e1207d700 10 monclient: _send_mon_message to mon.storage2 at v2:10.1.1.97:3300/0
-76> 2021-04-21 09:48:35.305 7f2e0906b700 2 object expiration: stop
-75> 2021-04-21 09:48:39.857 7f2e1087a700 10 monclient: tick
-74> 2021-04-21 09:48:40.069 7f2e0c872700 10 monclient: tick
-73> 2021-04-21 09:48:49.073 7f2e0a06d700 2 RGWDataChangesLog::ChangesRenewThread: start
-72> 2021-04-21 09:48:49.857 7f2e1087a700 10 monclient: tick
-71> 2021-04-21 09:48:50.069 7f2e0c872700 10 monclient: tick
-70> 2021-04-21 09:48:59.858 7f2e1087a700 10 monclient: tick
-69> 2021-04-21 09:49:00.070 7f2e0c872700 10 monclient: tick
-68> 2021-04-21 09:49:01.962 7f2df203d700 1 ====== starting new request req=0x55b8e3a36880 =====
-67> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s initializing for trans_id = tx000000000000000000001-0060802d4d-4c54ae2-default
-66> 2021-04-21 09:49:01.962 7f2df203d700 10 rgw api priority: s3=8 s3website=7
-65> 2021-04-21 09:49:01.962 7f2df203d700 10 host=<.......>
-64> 2021-04-21 09:49:01.962 7f2df203d700 10 handler=22RGWHandler_REST_Obj_S3
-63> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s getting op 0
-62> 2021-04-21 09:49:01.962 7f2df203d700 10 req 1 0.000s s3:get_obj scheduling with dmclock client=2 cost=1
-61> 2021-04-21 09:49:01.962 7f2df203d700 10 op=21RGWGetObj_ObjStore_S3
-60> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj verifying requester
-59> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj normalizing buckets and tenants
-58> 2021-04-21 09:49:01.962 7f2df203d700 10 s->object=testfile s->bucket=test
-57> 2021-04-21 09:49:01.962 7f2df203d700 2 req 1 0.000s s3:get_obj init permissions
-56> 2021-04-21 09:49:01.962 7f2df203d700 10 cache get: name=.rgw++test : miss
-55> 2021-04-21 09:49:01.962 7f2e16886700 10 monclient: get_auth_request con 0x55b8e384ba80 auth_method 0
-54> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++test info.flags=0x16
-53> 2021-04-21 09:49:01.966 7f2df203d700 10 adding .rgw++test to cache LRU end
-52> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-51> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++test : type miss (requested=0x13, cached=0x16)
-50> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++test info.flags=0x13
-49> 2021-04-21 09:49:01.966 7f2df203d700 10 moving .rgw++test to cache LRU end
-48> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-47> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++.bucket.meta.test:default.1021717.1 : miss
-46> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++.bucket.meta.test:default.1021717.1 info.flags=0x16
-45> 2021-04-21 09:49:01.966 7f2df203d700 10 adding .rgw++.bucket.meta.test:default.1021717.1 to cache LRU end
-44> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-43> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=user.rgw.acl bl.length()=149
-42> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.rgw++.bucket.meta.test:default.1021717.1 : type miss (requested=0x13, cached=0x16)
-41> 2021-04-21 09:49:01.966 7f2df203d700 10 cache put: name=.rgw++.bucket.meta.test:default.1021717.1 info.flags=0x13
-40> 2021-04-21 09:49:01.966 7f2df203d700 10 moving .rgw++.bucket.meta.test:default.1021717.1 to cache LRU end
-39> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-38> 2021-04-21 09:49:01.966 7f2df203d700 10 updating xattr: name=user.rgw.acl bl.length()=149
-37> 2021-04-21 09:49:01.966 7f2df203d700 10 chain_cache_entry: cache_locator=.rgw++test
-36> 2021-04-21 09:49:01.966 7f2df203d700 10 chain_cache_entry: cache_locator=.rgw++.bucket.meta.test:default.1021717.1
-35> 2021-04-21 09:49:01.966 7f2df203d700 10 cache get: name=.users.uid++anonymous : miss
-34> 2021-04-21 09:49:01.970 7f2e16085700 10 monclient: get_auth_request con 0x55b8e3a5e000 auth_method 0
-33> 2021-04-21 09:49:01.970 7f2df203d700 10 cache put: name=.users.uid++anonymous info.flags=0x0
-32> 2021-04-21 09:49:01.970 7f2df203d700 10 adding .users.uid++anonymous to cache LRU end
-31> 2021-04-21 09:49:01.970 7f2df203d700 2 req 1 0.008s s3:get_obj recalculating target
-30> 2021-04-21 09:49:01.970 7f2df203d700 2 req 1 0.008s s3:get_obj reading permissions
-29> 2021-04-21 09:49:01.970 7f2e17087700 10 monclient: get_auth_request con 0x55b8e3a5e480 auth_method 0
-28> 2021-04-21 09:49:01.974 7f2df203d700 10 manifest: total_size = 12
-27> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj init op
-26> 2021-04-21 09:49:01.974 7f2df203d700 10 cache get: name=.users.uid++jaharkes : miss
-25> 2021-04-21 09:49:01.974 7f2df203d700 10 cache put: name=.users.uid++jaharkes info.flags=0x6
-24> 2021-04-21 09:49:01.974 7f2df203d700 10 adding .users.uid++jaharkes to cache LRU end
-23> 2021-04-21 09:49:01.974 7f2df203d700 10 updating xattr: name=ceph.objclass.version bl.length()=42
-22> 2021-04-21 09:49:01.974 7f2df203d700 10 cache get: name=.users.uid++jaharkes : type miss (requested=0x1, cached=0x6)
-21> 2021-04-21 09:49:01.974 7f2df203d700 10 cache put: name=.users.uid++jaharkes info.flags=0x1
-20> 2021-04-21 09:49:01.974 7f2df203d700 10 moving .users.uid++jaharkes to cache LRU end
-19> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op mask
-18> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op permissions
-17> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) mask=49
-16> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for uid=anonymous
-15> 2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for user not found
-14> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for group=1 mask=49
-13> 2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for group not found
-12> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0), owner=jaharkes, perm=0
-11> 2021-04-21 09:49:01.974 7f2df203d700 10 req 1 0.012s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=0, user_perm_mask=1, acl perm=0
-10> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj Searching permissions for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) mask=49
-9> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for uid=anonymous
-8> 2021-04-21 09:49:01.974 7f2df203d700 5 Permissions for user not found
-7> 2021-04-21 09:49:01.974 7f2df203d700 5 Searching permissions for group=1 mask=49
-6> 2021-04-21 09:49:01.974 7f2df203d700 5 Found permission: 1
-5> 2021-04-21 09:49:01.974 7f2df203d700 5 req 1 0.012s s3:get_obj -- Getting permissions done for identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0), owner=jaharkes, perm=1
-4> 2021-04-21 09:49:01.974 7f2df203d700 10 req 1 0.012s s3:get_obj identity=rgw::auth::SysReqApplier -> rgw::auth::LocalApplier(acct_user=anonymous, acct_name=, subuser=, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=1, user_perm_mask=15, acl perm=1
-3> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj verifying op params
-2> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj pre-executing
-1> 2021-04-21 09:49:01.974 7f2df203d700 2 req 1 0.012s s3:get_obj executing
0> 2021-04-21 09:49:01.982 7f2df203d700 -1 *** Caught signal (Aborted) **
in thread 7f2df203d700 thread_name:radosgw

ceph version 14.2.20 (36274af6eb7f2a5055f2d53ad448f2694e9046a0) nautilus (stable)
1: (()+0x12980) [0x7f2e1ffcc980]
2: (gsignal()+0xc7) [0x7f2e1f869fb7]
3: (abort()+0x141) [0x7f2e1f86b921]
4: (()+0x340ef9) [0x7f2e2185cef9]
5: (__cxxabiv1::__terminate(void (*)())+0x6) [0x7f2e21e94226]
6: (()+0x978291) [0x7f2e21e94291]
7: (()+0x9783e5) [0x7f2e21e943e5]
8: (virtual thunk to boost::exception_detail::clone_impl<boost::exception_detail::current_exception_std_exception_wrapper<std::bad_alloc> >::rethrow() const+0x6c) [0x55b8dfe236d8]
9: (boost::rethrow_exception(boost::exception_ptr const&)+0x12) [0x55b8dfef1e72]
10: (boost::coroutines::push_coroutine<void>::operator()()+0x69) [0x55b8dfef1f19]
11: (boost::asio::detail::executor_function<ceph::async::ForwardingHandler<ceph::async::CompletionHandler<boost::asio::detail::coro_handler<boost::asio::executor_binder<void (*)(), boost::asio::executor>, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > >, std::tuple<boost::system::error_code, std::shared_lock<ceph::async::SharedMutex<boost::asio::io_context::executor_type> > > > >, std::allocator<void> >::do_complete(boost::asio::detail::executor_function_base*, bool)+0x16a) [0x55b8dfefa00a]
12: (boost::asio::detail::executor_op<boost::asio::executor::function, std::allocator<void>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x5c) [0x55b8dff4a22c]
13: (boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>::operator()()+0x85) [0x55b8dff0a6a5]
14: (boost::asio::detail::executor_op<boost::asio::detail::strand_executor_service::invoker<boost::asio::io_context::executor_type const>, boost::asio::detail::recycling_allocator<void, boost::asio::detail::thread_info_base::default_tag>, boost::asio::detail::scheduler_operation>::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long)+0x96) [0x55b8dff0aa06]
15: (boost::asio::detail::scheduler::run(boost::system::error_code&)+0x4f6) [0x55b8dfef6fe6]
16: (()+0x31c682) [0x55b8dfed6682]
17: (()+0x9f8e70) [0x7f2e21f14e70]
18: (()+0x76db) [0x7f2e1ffc16db]
19: (clone()+0x3f) [0x7f2e1f94c71f]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
0/ 5 none
0/ 1 lockdep
0/ 1 context
1/ 1 crush
1/ 5 mds
1/ 5 mds_balancer
1/ 5 mds_locker
1/ 5 mds_log
1/ 5 mds_log_expire
1/ 5 mds_migrator
0/ 1 buffer
0/ 1 timer
0/ 1 filer
0/ 1 striper
0/ 1 objecter
0/ 5 rados
0/ 5 rbd
0/ 5 rbd_mirror
0/ 5 rbd_replay
0/ 5 journaler
0/ 5 objectcacher
0/ 5 client
1/ 5 osd
0/ 5 optracker
0/ 5 objclass
1/ 3 filestore
1/ 3 journal
0/ 0 ms
1/ 5 mon
0/10 monc
1/ 5 paxos
0/ 5 tp
1/ 5 auth
1/ 5 crypto
1/ 1 finisher
1/ 1 reserver
1/ 5 heartbeatmap
1/ 5 perfcounter
10/10 rgw
1/ 5 rgw_sync
1/10 civetweb
1/ 5 javaclient
1/ 5 asok
1/ 1 throttle
0/ 0 refs
1/ 5 xio
1/ 5 compressor
1/ 5 bluestore
1/ 5 bluefs
1/ 3 bdev
1/ 5 kstore
4/ 5 rocksdb
4/ 5 leveldb
4/ 5 memdb
1/ 5 kinetic
1/ 5 fuse
1/ 5 mgr
1/ 5 mgrc
1/ 5 dpdk
1/ 5 eventtrace
1/ 5 prioritycache
-2/-2 (syslog threshold)
99/99 (stderr threshold)
max_recent 10000
max_new 1000
log_file /var/lib/ceph/crash/2021-04-21_13:49:01.985146Z_2b7bf347-a374-4818-baed-fa66e0d6db8a/log
--- end dump of recent events ---
(2-2/2)