Bug #19629: mgr: set_config from python module crashes mgr (assertion failure due to access denied) - mgr - Ceph

Actions

Copy link

Bug #19629

closed

mgr: set_config from python module crashes mgr (assertion failure due to access denied)

Added by Tim Serong about 7 years ago. Updated almost 7 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Tim Serong

Category:

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Tried `ceph tell mgr enable_auth false` (enable_auth being implemented by the rest module). The ceph CLI became unresponsive (never returned to the shell). Meanwhile, mgr crashed with:

    -8> 2017-04-14 15:34:45.938295 7f00c778a700  4 mgr[rest] handle_command: {
  "prefix": "enable_auth", 
  "val": "false" 
}
    -7> 2017-04-14 15:34:45.938336 7f00c778a700  1 lockdep using id 41
    -6> 2017-04-14 15:34:45.938369 7f00c778a700 10 monclient: _send_command 5 [{"prefix":"config-key put","key":"mgr.rest.enable_auth","val":"false"}]
    -5> 2017-04-14 15:34:45.938376 7f00c778a700 10 monclient: _send_mon_message to mon.b at 127.0.0.1:40447/0
    -4> 2017-04-14 15:34:45.938381 7f00c778a700  1 -- 127.0.0.1:0/3288703021 --> 127.0.0.1:40447/0 -- mon_command({"prefix":"config-key put","key":"mgr.rest.enable_auth","val":"false"} v 0) v1 -- 0x55ea04db41c0 con 0
    -3> 2017-04-14 15:34:45.938756 7f00caf91700  1 -- 127.0.0.1:0/3288703021 <== mon.1 127.0.0.1:40447/0 25 ==== mon_command_ack([{"prefix":"config-key put","key":"mgr.rest.enable_auth","val":"false"}]=-13 access denied v0) v1 ==== 117+0+0 (2523711155 0 0) 0x55ea04db41c0 con 0x55ea02d59800
    -2> 2017-04-14 15:34:45.938773 7f00caf91700 10 monclient: handle_mon_command_ack 5 [{"prefix":"config-key put","key":"mgr.rest.enable_auth","val":"false"}]
    -1> 2017-04-14 15:34:45.938778 7f00caf91700 10 monclient: _finish_command 5 = -13 access denied
     0> 2017-04-14 15:34:45.940929 7f00c778a700 -1 /home/tserong/src/github/SUSE/ceph/src/mgr/PyModules.cc: In function 'void PyModules::set_config(const string&, const string&, const string&)' thread 7f00c778a700 time 2017-04-14 15:34:45.938831
/home/tserong/src/github/SUSE/ceph/src/mgr/PyModules.cc: 546: FAILED assert(set_cmd.r == 0)

Why is the mon giving mgr "access denied" when it tries to set a config key?

We should consider getting rid of that assert too...

Actions

Copy link

Updated by Sage Weil almost 7 years ago

Priority changed from Normal to Urgent

Actions

Copy link

Updated by Tim Serong almost 7 years ago

The permission denied problem goes away if you give mgr full access to the mon. For example, in a vstart environment:

# bin/ceph auth caps mgr.x mon 'allow *' mds 'allow *' osd 'allow *'
updated caps for mgr.x
# bin/init-ceph restart mgr.x
# bin/ceph tell mgr enable_auth false
(...works fine, doesn't crash anymore...)

Originally it was "mon 'allow profile mgr'", now it's "mon 'allow *'". Is that too permissive?

Note that mgr must be restarted to pick up the auth caps change. Is that correct behaviour, or should it notice the changed caps by magic?

Actions

Copy link