Bug #20943
closedrbd: list of watchers not correspond to list of clients
0%
Description
It is possible to remove watcher from 'rbd status image' but
it's not preventing removed client from using of this rbd image.
prepare image:
$ rbd create image --size 100M
$ sudo rbd map image
$ sudo mkfs.ext4 /dev/rbd0
$ sudo rbd unmap image
172.21.12.201: map image on first client and enable ext4 multimount protection
$ sudo rbd map image
/dev/rbd0
$ sudo tune2fs /dev/rbd2 -O mmp
172.21.12.202: map image on second client and blacklist all other clients
$ sudo rbd map image
/dev/rbd0
$ rbd status image
Watchers:
watcher=172.21.12.201:0/4225036114 client.17881 cookie=18446462598732840961
watcher=172.21.12.202:0/790545486 client.18064 cookie=18446462598732840961
$ ceph osd blacklist add 172.21.12.201:0/4225036114 10
wait 10 seconds...
172.21.12.201: mount image on first client
$ sudo /dev/rbd0 mount /mnt
172.21.12.202: can't mount image on second client
$ rbd status image
Watchers:
watcher=172.21.12.202:0/790545486 client.18064 cookie=18446462598732840961
$ sudo /dev/rbd0 mount /mnt
mount: wrong fs type, bad option, bad superblock on /dev/rbd0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so.
$ dmesg
[ 2608.808818] EXT4-fs warning (device rbd0): ext4_multi_mount_protect:324: MMP interval 42 higher than expected, please wait.
[ 2630.461909] EXT4-fs warning (device rbd0): ext4_multi_mount_protect:337: Device is already active on another node.
[ 2630.461911] EXT4-fs warning (device rbd0): ext4_multi_mount_protect:337: MMP failure info: last update time: 1502183794, last update node: ceph-client-1, last update device: rbd0
Updated by Ilya Dryomov over 6 years ago
As of now that's the expected behavior -- the blacklist entry is expected to stay there and not go away (well, at least not until that mapping is teared down).
You shouldn't really need "ceph osd blacklist". That's a fairly low level command with some gotchas to be aware of. If you want an exclusive mapping, use "rbd map -o exclusive".