Feature #45091
closed
cephadm: CephX disabled: bad_method + failed to fetch mon config
Added by Ashley Merrick about 4 years ago.
Updated over 2 years ago.
Description
Upon trying to migrate an existing cluster to cephadm the new MDS via cephadm fail to start.
Cluster is running with the latest Octopus and with cephx disabled (set in the global config DB), error is showing from the cephadm logs, from my eyes seems its trying to connect with auth?:
Apr 13 06:26:15 sn-s01 systemd[1]: Started Ceph mds.cephfs.sn-s01.snkhfd for b1db6b36-0c4c-4bce-9cda-18834be0632d.
Apr 13 06:26:16 sn-s01 bash[3520809]: debug 2020-04-13T04:26:16.445+0000 7f380b908700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [1] but i only support [2]
Apr 13 06:26:16 sn-s01 bash[3520809]: debug 2020-04-13T04:26:16.445+0000 7f380b107700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [1] but i only support [2]
Apr 13 06:26:16 sn-s01 bash[3520809]: debug 2020-04-13T04:26:16.445+0000 7f380a906700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [1] but i only support [2]
Apr 13 06:26:16 sn-s01 bash[3520809]: failed to fetch mon config (--no-mon-config to skip
Workaround:¶
Enable Cephx authentication
- Project changed from Ceph to Orchestrator
- Subject changed from cephadm - mds start failing to cephadm: CephX disabled: bad_method + failed to fetch mon config
- Description updated (diff)
- Category changed from cephx to cephadm
- Priority changed from Normal to Urgent
- Priority changed from Urgent to High
It seems this isn't just a bug related to new MDS, I also hit the exact same erorr trying to add a new OSD/MON via cephadm also.
To update, I was able to restart all services and move to using cephx and the two MDS come online.
- Tracker changed from Bug to Feature
- Description updated (diff)
- Priority changed from High to Normal
/a/yuriw-2020-09-16_23:57:37-rados-wip-yuri8-testing-2020-09-16-2220-octopus-distro-basic-smithi/5441511/teuthology.log
monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2020-09-17T00:50:33.095 INFO:tasks.ceph.mgr.x.smithi136.stderr:2020-09-17T00:50:33.092+0000 7f6d77253700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2020-09-17T00:50:33.098 INFO:tasks.ceph.osd.0.smithi136.stderr:2020-09-17T00:50:33.096+0000 7f9c85103700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2020-09-17T00:50:33.130 INFO:teuthology.orchestra.run.smithi136.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused
2020-09-17T00:50:33.132 DEBUG:teuthology.orchestra.run:got remote process result: 22
2020-09-17T00:50:33.133 INFO:tasks.rebuild_mondb.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight']
2020-09-17T00:50:33.302 INFO:tasks.ceph.osd.0.smithi136.stderr:2020-09-17T00:50:33.300+0000 7f9c85904700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]
2020-09-17T00:50:33.466 INFO:tasks.ceph.osd.0.smithi136.stderr:failed to fetch mon config (--no-mon-config to skip)
...
exception getting command descriptions: [Errno 111] Connection refused
2020-09-17T00:55:15.939 DEBUG:teuthology.orchestra.run:got remote process result: 22
tasks.rebuild_mondb.ceph_manager:waiting on admin_socket for osd-0,
osd 0 log:
2020-09-17T00:49:32.184+0000 7fa16a8e7700 1 --2- v2:172.21.15.136:6806/12987 >> [v2:172.21.15.136:3301/0,v1:172.21.15.136:6790/0] conn(0x5591a00f8800 0x559│
│2020-09-17T00:49:32.184+0000 7fa1787ef700 10 NetHandler reconnect reconnect: Connection refused
nothing noteworthy
Deepika, this is something different:
handle_auth_bad_method server allowed_methods [2] but i only support [2]
Which is pretty much something like
handle_auth_bad_method server allowed_methods "cephx" but i only support "cephx"
Which looks super strange.
Mind creating a new tracker issue for this and ping a MON export?
sure, I saw things failing after auth command `adding auth for 'client.0': auth(key=AQAnsmJfMj0tCRAA/EDJ1+czLlBGbNdfSnllpw==) with caps({mds=allow,mgr=allow r,mon=allow rw,osd=allow rwx})` got confused, thanks for taking a look!
- Priority changed from Normal to Low
- Status changed from New to Closed
Also available in: Atom
PDF