# rbd rm libvirt/test1 --debug-rbd=20 --debug-ms=1
2016-06-17 12:46:02.450577 7f7db4244100 -1 WARNING: the following dangerous and experimental features are enabled: *
2016-06-17 12:46:02.450622 7f7db4244100 -1 WARNING: the following dangerous and experimental features are enabled: *
2016-06-17 12:46:02.452120 7f7db4244100 -1 WARNING: the following dangerous and experimental features are enabled: *
2016-06-17 12:46:02.452339 7f7db4244100  1 -- :/0 messenger.start
2016-06-17 12:46:02.453061 7f7db4244100  1 -- :/666785443 --> 172.16.1.20:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- ?+0 0x560f4a40e130 con 0x560f4a40d320
2016-06-17 12:46:02.453690 7f7db4237700  1 -- 172.16.1.10:0/666785443 learned my addr 172.16.1.10:0/666785443
2016-06-17 12:46:02.454780 7f7d990e0700  1 -- 172.16.1.10:0/666785443 <== mon.1 172.16.1.20:6789/0 1 ==== mon_map magic: 0 v1 ==== 642+0+0 (1731561352 0 0) 0x7f7d84000c20 con 0x560f4a40d320
2016-06-17 12:46:02.454858 7f7d990e0700  1 -- 172.16.1.10:0/666785443 <== mon.1 172.16.1.20:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (1419757420 0 0) 0x7f7d84001110 con 0x560f4a40d320
2016-06-17 12:46:02.454980 7f7d990e0700  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x7f7d880019b0 con 0x560f4a40d320
2016-06-17 12:46:02.455938 7f7d990e0700  1 -- 172.16.1.10:0/666785443 <== mon.1 172.16.1.20:6789/0 3 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 206+0+0 (1544320143 0 0) 0x7f7d84001110 con 0x560f4a40d320
2016-06-17 12:46:02.456091 7f7d990e0700  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x7f7d88001dd0 con 0x560f4a40d320
2016-06-17 12:46:02.457050 7f7d990e0700  1 -- 172.16.1.10:0/666785443 <== mon.1 172.16.1.20:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 393+0+0 (56483488 0 0) 0x7f7d840008c0 con 0x560f4a40d320
2016-06-17 12:46:02.457175 7f7d990e0700  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x560f4a4120d0 con 0x560f4a40d320
2016-06-17 12:46:02.457321 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x560f4a40e130 con 0x560f4a40d320
2016-06-17 12:46:02.457840 7f7d990e0700  1 -- 172.16.1.10:0/666785443 <== mon.1 172.16.1.20:6789/0 5 ==== mon_map magic: 0 v1 ==== 642+0+0 (1731561352 0 0) 0x7f7d84001460 con 0x560f4a40d320
2016-06-17 12:46:02.457905 7f7d990e0700  1 -- 172.16.1.10:0/666785443 <== mon.1 172.16.1.20:6789/0 6 ==== osd_map(645..645 src has 1..645) v3 ==== 25017+0+0 (3930215128 0 0) 0x7f7d84007860 con 0x560f4a40d320
2016-06-17 12:46:02.458168 7f7db4244100 20 librbd: remove 0x7ffe2d1ae340 test1
2016-06-17 12:46:02.458274 7f7db4244100  5 librbd::AioImageRequestWQ: 0x560f4a4132c0 : ictx=0x560f4a412440
2016-06-17 12:46:02.458282 7f7db4244100 20 librbd::ImageState: open
2016-06-17 12:46:02.458284 7f7db4244100 10 librbd::ImageState: 0x560f4a411830 send_open_unlock
2016-06-17 12:46:02.458287 7f7db4244100 10 librbd::image::OpenRequest: 0x560f4a413a60 send_v2_detect_header
2016-06-17 12:46:02.458395 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6822/3637 -- osd_op(client.85449.0:1 4.55bd76d3 rbd_id.test1 [stat] snapc 0=[] ack+read+known_if_redirected e645) v7 -- ?+0 0x560f4a417240 con 0x560f4a415e50
2016-06-17 12:46:02.460141 7f7d90df7700  1 -- 172.16.1.10:0/666785443 <== osd.11 172.16.1.20:6822/3637 1 ==== osd_op_reply(1 rbd_id.test1 [stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v7 ==== 132+0+0 (2316701749 0 0) 0x7f7d74000a20 con 0x560f4a415e50
2016-06-17 12:46:02.460306 7f7d91efa700 10 librbd::image::OpenRequest: handle_v2_detect_header: r=-2
2016-06-17 12:46:02.460431 7f7d91efa700  1 -- 172.16.1.10:0/666785443 --> 172.16.1.21:6808/10860 -- osd_op(client.85449.0:2 4.2bc09b0b test1.rbd [stat] snapc 0=[] ack+read+known_if_redirected e645) v7 -- ?+0 0x7f7d78004070 con 0x7f7d78002c40
2016-06-17 12:46:02.462641 7f7d90bf5700  1 -- 172.16.1.10:0/666785443 <== osd.15 172.16.1.21:6808/10860 1 ==== osd_op_reply(2 test1.rbd [stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v7 ==== 129+0+0 (1021193307 0 0) 0x7f7d70000a20 con 0x7f7d78002c40
2016-06-17 12:46:02.462805 7f7d91efa700 10 librbd::image::OpenRequest: handle_v1_detect_header: r=-2
2016-06-17 12:46:02.462814 7f7d91efa700 10 librbd::image::OpenRequest: 0x560f4a413a60 send_close_image
2016-06-17 12:46:02.462817 7f7d91efa700 10 librbd::image::CloseRequest: 0x7f7d78004670 send_shut_down_aio_queue
2016-06-17 12:46:02.462821 7f7d91efa700  5 librbd::AioImageRequestWQ: shut_down: in_flight=0
2016-06-17 12:46:02.462848 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 handle_shut_down_aio_queue: r=0
2016-06-17 12:46:02.462859 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 send_flush
2016-06-17 12:46:02.462868 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 handle_flush: r=0
2016-06-17 12:46:02.462883 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 send_flush_readahead
2016-06-17 12:46:02.462889 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 handle_flush_readahead: r=0
2016-06-17 12:46:02.462892 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 send_shut_down_cache
2016-06-17 12:46:02.462895 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 handle_shut_down_cache: r=0
2016-06-17 12:46:02.462897 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 send_flush_op_work_queue
2016-06-17 12:46:02.462902 7f7d916f9700 10 librbd::image::CloseRequest: 0x7f7d78004670 handle_flush_op_work_queue: r=0
2016-06-17 12:46:02.462907 7f7d916f9700 10 librbd::image::OpenRequest: handle_close_image: r=0
2016-06-17 12:46:02.462916 7f7d916f9700 10 librbd::ImageState: 0x560f4a411830 handle_open: r=-2
2016-06-17 12:46:02.463003 7f7db4244100  2 librbd: error opening image: (2) No such file or directory
2016-06-17 12:46:02.463024 7f7db4244100  2 librbd: removing rbd image from directory...
2016-06-17 12:46:02.463133 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.22:6808/15894 -- osd_op(client.85449.0:3 4.30a98c1c rbd_directory [tmapup 0~0] snapc 0=[] ondisk+write+known_if_redirected e645) v7 -- ?+0 0x560f4a412440 con 0x560f4a413fd0
2016-06-17 12:46:02.465633 7f7d909f3700  1 -- 172.16.1.10:0/666785443 <== osd.28 172.16.1.22:6808/15894 1 ==== osd_op_reply(3 rbd_directory [tmapup 0~0] v0'0 uv32 ondisk = -2 ((2) No such file or directory)) v7 ==== 133+0+0 (1425705836 0 0) 0x7f7d68000a20 con 0x560f4a413fd0
2016-06-17 12:46:02.465770 7f7db4244100  5 librbd::Journal: remove: image=
2016-06-17 12:46:02.466580 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.22:6820/20336 -- osd_op(client.85449.0:4 4.5f5a7372 journal. [stat] snapc 0=[] ack+read+known_if_redirected e645) v7 -- ?+0 0x560f4a41d4f0 con 0x560f4a4186c0
2016-06-17 12:46:02.468557 7f7d636fd700  1 -- 172.16.1.10:0/666785443 <== osd.34 172.16.1.22:6820/20336 1 ==== osd_op_reply(4 journal. [stat] v0'0 uv0 ack = -2 ((2) No such file or directory)) v7 ==== 128+0+0 (3983481914 0 0) 0x7f7d58000a20 con 0x560f4a4186c0
2016-06-17 12:46:02.469295 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6808/29591 -- osd_op(client.85449.0:5 4.c5a1c848 rbd_object_map. [delete] snapc 0=[] ondisk+write+known_if_redirected e645) v7 -- ?+0 0x560f4a41d4f0 con 0x560f4a412cb0
2016-06-17 12:46:02.471903 7f7d634fb700  1 -- 172.16.1.10:0/666785443 <== osd.4 172.16.1.20:6808/29591 1 ==== osd_op_reply(5 rbd_object_map. [delete] v0'0 uv3 ondisk = -2 ((2) No such file or directory)) v7 ==== 135+0+0 (3751772400 0 0) 0x7f7d4c000a20 con 0x560f4a412cb0
2016-06-17 12:46:02.472044 7f7db4244100  2 librbd: removing id object...
2016-06-17 12:46:02.472080 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6822/3637 -- osd_op(client.85449.0:6 4.55bd76d3 rbd_id.test1 [delete] snapc 0=[] ondisk+write+known_if_redirected e645) v7 -- ?+0 0x560f4a41d4f0 con 0x560f4a415e50
2016-06-17 12:46:02.472838 7f7d90df7700  1 -- 172.16.1.10:0/666785443 <== osd.11 172.16.1.20:6822/3637 2 ==== osd_op_reply(6 rbd_id.test1 [delete] v0'0 uv37 ondisk = -2 ((2) No such file or directory)) v7 ==== 132+0+0 (2110787341 0 0) 0x7f7d74000a20 con 0x560f4a415e50
2016-06-17 12:46:02.473016 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.22:6808/15894 -- osd_op(client.85449.0:7 4.30a98c1c rbd_directory [call rbd.dir_get_id] snapc 0=[] ack+read+known_if_redirected e645) v7 -- ?+0 0x560f4a418040 con 0x560f4a413fd0
2016-06-17 12:46:02.474062 7f7d909f3700  1 -- 172.16.1.10:0/666785443 <== osd.28 172.16.1.22:6808/15894 2 ==== osd_op_reply(7 rbd_directory [call] v0'0 uv32 ondisk = 0) v7 ==== 133+0+16 (425628749 0 3476530269) 0x7f7d68000a20 con 0x560f4a413fd0
2016-06-17 12:46:02.474369 7f7db4244100  2 librbd: removing rbd image from directory...
2016-06-17 12:46:02.474414 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.22:6808/15894 -- osd_op(client.85449.0:8 4.30a98c1c rbd_directory [call rbd.dir_remove_image] snapc 0=[] ack+read+known_if_redirected e645) v7 -- ?+0 0x560f4a418040 con 0x560f4a413fd0
2016-06-17 12:46:02.477344 7f7d909f3700  1 -- 172.16.1.10:0/666785443 <== osd.28 172.16.1.22:6808/15894 3 ==== osd_op_reply(8 rbd_directory [call] v645'33 uv33 ack = 0) v7 ==== 133+0+0 (3612250829 0 0) 0x7f7d68001540 con 0x560f4a413fd0
2016-06-17 12:46:02.477461 7f7db4244100  2 librbd: removing image from rbd_mirroring object...
2016-06-17 12:46:02.477566 7f7db4244100  1 -- 172.16.1.10:0/666785443 --> 172.16.1.20:6816/17542 -- osd_op(client.85449.0:9 4.b74753e6 rbd_mirroring [call rbd.mirror_image_remove] snapc 0=[] ack+read+known_if_redirected e645) v7 -- ?+0 0x560f4a424b00 con 0x560f4a418250
2016-06-17 12:46:02.479705 7f7d632f9700  1 -- 172.16.1.10:0/666785443 <== osd.8 172.16.1.20:6816/17542 1 ==== osd_op_reply(9 rbd_mirroring [call] v0'0 uv1 ondisk = -2 ((2) No such file or directory)) v7 ==== 133+0+0 (3979857520 0 0) 0x7f7d44000a20 con 0x560f4a418250

Removing image2016-06-17 12:46:02.479847 7f7db4244100  2 librbd: done.
: 100% complete...done.
2016-06-17 12:46:02.480323 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x560f4a412cb0 -- 0x560f4a41d9a0
2016-06-17 12:46:02.480380 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x560f4a418250 -- 0x560f4a422670
2016-06-17 12:46:02.480413 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x560f4a415e50 -- 0x560f4a414b70
2016-06-17 12:46:02.480437 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x7f7d78002c40 -- 0x7f7d78001900
2016-06-17 12:46:02.480584 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x560f4a413fd0 -- 0x560f4a418860
2016-06-17 12:46:02.480672 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x560f4a4186c0 -- 0x560f4a41b060
2016-06-17 12:46:02.480748 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down 0x560f4a40d320 -- 0x560f4a40c040
2016-06-17 12:46:02.481033 7f7db4244100  1 -- 172.16.1.10:0/666785443 mark_down_all
2016-06-17 12:46:02.481381 7f7db4244100  1 -- 172.16.1.10:0/666785443 shutdown complete.