Actions
Bug #53358
closedmgr/cephadm: ssh errors too verbose and timeout too long when can't connect to host
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
When hosts are found to be offline a health warning will be raised with a corresponding error message. However, the current error messages are far too verbose and contain mostly non helpful information.
NOTE: seemingly an easy way to recreate this is to add a nonexistent host e.g. "ceph orch host add fake-host 2.2.2.2" will sit there for a few minutes then spit out one of these long messages
An excerpt from a "ceph health detail" call
[WRN] CEPHADM_HOST_CHECK_FAILED: 2 hosts fail cephadm check
host storage-01-08014 (10.221.1.56) failed check: Failed to connect to storage-01-08014 (10.221.1.56). Connection lost
Log: Opening SSH connection to 10.221.1.56, port 22
Opening SSH connection to 10.221.1.111, port 22
[conn=1687] Connected to SSH server at 10.221.1.56, port 22
[conn=1687] Local address: 10.221.1.188, port 42830
[conn=1687] Peer address: 10.221.1.56, port 22
[conn=1687] Sending version SSH-2.0-AsyncSSH_2.7.0
[conn=1688] Connected to SSH server at 10.221.1.111, port 22
[conn=1688] Local address: 10.221.1.188, port 45254
[conn=1688] Peer address: 10.221.1.111, port 22
[conn=1688] Sending version SSH-2.0-AsyncSSH_2.7.0
[conn=1687] Connection lost
[conn=88, chan=132] Set write buffer limits: low-water=16384, high-water=65536
[conn=88, chan=132] Requesting new SSH session
[conn=88, chan=132] Initial recv window 2097152, packet size 32768
[conn=88, pktid=1191] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=132, pktid=1192] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 84 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=84, chan=132] Set write buffer limits: low-water=16384, high-water=65536
[conn=84, chan=132] Requesting new SSH session
[conn=84, chan=132] Initial recv window 2097152, packet size 32768
[conn=84, pktid=1193] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=132, pktid=1194] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 84 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=80, chan=132] Set write buffer limits: low-water=16384, high-water=65536
[conn=80, chan=132] Requesting new SSH session
[conn=80, chan=132] Initial recv window 2097152, packet size 32768
[conn=80, pktid=1191] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=132, pktid=1192] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 84 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=1687] Aborting connection
host storage-02-08027 (10.221.1.111) failed check: Can't communicate with remote host `10.221.1.111`, possibly because python3 is not installed there. [Errno 104] Connection reset by peer
Log: Opening SSH connection to 10.221.1.111, port 22
[conn=1687] Connected to SSH server at 10.221.1.56, port 22
[conn=1687] Local address: 10.221.1.188, port 42830
[conn=1687] Peer address: 10.221.1.56, port 22
[conn=1687] Sending version SSH-2.0-AsyncSSH_2.7.0
[conn=1688] Connected to SSH server at 10.221.1.111, port 22
[conn=1688] Local address: 10.221.1.188, port 45254
[conn=1688] Peer address: 10.221.1.111, port 22
[conn=1688] Sending version SSH-2.0-AsyncSSH_2.7.0
[conn=1687] Connection lost
[conn=88, chan=132] Set write buffer limits: low-water=16384, high-water=65536
[conn=88, chan=132] Requesting new SSH session
[conn=88, chan=132] Initial recv window 2097152, packet size 32768
[conn=88, pktid=1191] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=132, pktid=1192] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 84 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=84, chan=132] Set write buffer limits: low-water=16384, high-water=65536
[conn=84, chan=132] Requesting new SSH session
[conn=84, chan=132] Initial recv window 2097152, packet size 32768
[conn=84, pktid=1193] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=132, pktid=1194] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 84 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=80, chan=132] Set write buffer limits: low-water=16384, high-water=65536
[conn=80, chan=132] Requesting new SSH session
[conn=80, chan=132] Initial recv window 2097152, packet size 32768
[conn=80, pktid=1191] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=132, pktid=1192] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 84 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=1687] Aborting connection
[conn=88, pktid=1329] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 84 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=88, chan=132] Initial send window 0, packet size 32768
[conn=84, pktid=1325] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 84 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=84, chan=132] Initial send window 0, packet size 32768
[conn=80, pktid=1328] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 84 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=80, chan=132] Initial send window 0, packet size 32768
[conn=88, chan=132] Command: sudo which python3
[conn=88, pktid=1193] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=132, pktid=1194] Sent MSG_CHANNEL_REQUEST (98), 36 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 12 73 75 64 6f 20 77 68 69 63 68 20 70 79 74 ..sudo which pyt
00000020: 68 6f 6e 33 hon3
[conn=84, chan=132] Command: sudo which python3
[conn=84, pktid=1195] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=132, pktid=1196] Sent MSG_CHANNEL_REQUEST (98), 36 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 12 73 75 64 6f 20 77 68 69 63 68 20 70 79 74 ..sudo which pyt
00000020: 68 6f 6e 33 hon3
[conn=80, chan=132] Command: sudo which python3
[conn=80, pktid=1193] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=132, pktid=1194] Sent MSG_CHANNEL_REQUEST (98), 36 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 12 73 75 64 6f 20 77 68 69 63 68 20 70 79 74 ..sudo which pyt
00000020: 68 6f 6e 33 hon3
[conn=88, chan=132, pktid=1330] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 84 00 20 00 00 ]..... ..
[conn=88, chan=132] Received window adjust of 2097152 bytes, new window 2097152
[conn=88, chan=132, pktid=1331] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 84 c....
[conn=88, chan=132, pktid=1332] Received MSG_CHANNEL_DATA (94), 26 bytes
00000000: 5e 00 00 00 84 00 00 00 11 2f 75 73 72 2f 62 69 ^......../usr/bi
00000010: 6e 2f 70 79 74 68 6f 6e 33 0a n/python3.
[conn=88, chan=132] Received 17 data bytes
[conn=88, chan=132, pktid=1333] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 84 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=88, chan=132] Received exit status 0
[conn=88, chan=132, pktid=1334] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 84 `....
[conn=88, chan=132] Received EOF
[conn=88, chan=132, pktid=1335] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 84 a....
[conn=88, chan=132] Received channel close
[conn=88, pktid=1195] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=132, pktid=1196] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=84, chan=132, pktid=1326] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 84 00 20 00 00 ]..... ..
[conn=84, chan=132] Received window adjust of 2097152 bytes, new window 2097152
[conn=84, chan=132, pktid=1327] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 84 c....
[conn=84, chan=132, pktid=1328] Received MSG_CHANNEL_DATA (94), 26 bytes
00000000: 5e 00 00 00 84 00 00 00 11 2f 75 73 72 2f 62 69 ^......../usr/bi
00000010: 6e 2f 70 79 74 68 6f 6e 33 0a n/python3.
[conn=84, chan=132] Received 17 data bytes
[conn=84, chan=132, pktid=1329] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 84 `....
[conn=84, chan=132] Received EOF
[conn=84, chan=132, pktid=1330] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 84 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=84, chan=132] Received exit status 0
[conn=84, chan=132, pktid=1331] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 84 a....
[conn=84, chan=132] Received channel close
[conn=84, pktid=1197] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=132, pktid=1198] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=80, chan=132, pktid=1329] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 84 00 20 00 00 ]..... ..
[conn=80, chan=132] Received window adjust of 2097152 bytes, new window 2097152
[conn=80, chan=132, pktid=1330] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 84 c....
[conn=80, chan=132, pktid=1331] Received MSG_CHANNEL_DATA (94), 26 bytes
00000000: 5e 00 00 00 84 00 00 00 11 2f 75 73 72 2f 62 69 ^......../usr/bi
00000010: 6e 2f 70 79 74 68 6f 6e 33 0a n/python3.
[conn=80, chan=132] Received 17 data bytes
[conn=80, chan=132, pktid=1332] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 84 `....
[conn=80, chan=132] Received EOF
[conn=80, chan=132, pktid=1333] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 84 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=80, chan=132] Received exit status 0
[conn=80, chan=132, pktid=1334] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 84 a....
[conn=80, chan=132] Received channel close
[conn=80, pktid=1195] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=132, pktid=1196] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=162, chan=130] Set write buffer limits: low-water=16384, high-water=65536
[conn=162, chan=130] Requesting new SSH session
[conn=162, chan=130] Initial recv window 2097152, packet size 32768
[conn=162, pktid=1179] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=162, chan=130, pktid=1180] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 82 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=164, chan=130] Set write buffer limits: low-water=16384, high-water=65536
[conn=164, chan=130] Requesting new SSH session
[conn=164, chan=130] Initial recv window 2097152, packet size 32768
[conn=164, pktid=1179] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=164, chan=130, pktid=1180] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 82 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=88, chan=132] Reading from channel started
[conn=84, chan=132] Reading from channel started
[conn=80, chan=132] Reading from channel started
[conn=88, chan=132] Channel closed
[conn=84, chan=132] Channel closed
[conn=80, chan=132] Channel closed
[conn=162, pktid=1310] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 82 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=162, chan=130] Initial send window 0, packet size 32768
[conn=164, pktid=1312] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 82 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=164, chan=130] Initial send window 0, packet size 32768
[conn=162, chan=130] Command: sudo which python3
[conn=162, pktid=1181] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=162, chan=130, pktid=1182] Sent MSG_CHANNEL_REQUEST (98), 36 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 12 73 75 64 6f 20 77 68 69 63 68 20 70 79 74 ..sudo which pyt
00000020: 68 6f 6e 33 hon3
[conn=164, chan=130] Command: sudo which python3
[conn=164, pktid=1181] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=164, chan=130, pktid=1182] Sent MSG_CHANNEL_REQUEST (98), 36 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 12 73 75 64 6f 20 77 68 69 63 68 20 70 79 74 ..sudo which pyt
00000020: 68 6f 6e 33 hon3
[conn=162, chan=130, pktid=1311] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 82 00 20 00 00 ]..... ..
[conn=162, chan=130] Received window adjust of 2097152 bytes, new window 2097152
[conn=162, chan=130, pktid=1312] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 82 c....
[conn=162, chan=130, pktid=1313] Received MSG_CHANNEL_DATA (94), 26 bytes
00000000: 5e 00 00 00 82 00 00 00 11 2f 75 73 72 2f 62 69 ^......../usr/bi
00000010: 6e 2f 70 79 74 68 6f 6e 33 0a n/python3.
[conn=162, chan=130] Received 17 data bytes
[conn=162, chan=130, pktid=1314] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 82 `....
[conn=162, chan=130] Received EOF
[conn=162, chan=130, pktid=1315] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 82 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=162, chan=130] Received exit status 0
[conn=162, chan=130, pktid=1316] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 82 a....
[conn=162, chan=130] Received channel close
[conn=162, pktid=1183] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=162, chan=130, pktid=1184] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=164, chan=130, pktid=1313] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 82 00 20 00 00 ]..... ..
[conn=164, chan=130] Received window adjust of 2097152 bytes, new window 2097152
[conn=164, chan=130, pktid=1314] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 82 c....
[conn=164, chan=130, pktid=1315] Received MSG_CHANNEL_DATA (94), 26 bytes
00000000: 5e 00 00 00 82 00 00 00 11 2f 75 73 72 2f 62 69 ^......../usr/bi
00000010: 6e 2f 70 79 74 68 6f 6e 33 0a n/python3.
[conn=164, chan=130] Received 17 data bytes
[conn=164, chan=130, pktid=1316] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 82 `....
[conn=164, chan=130] Received EOF
[conn=164, chan=130, pktid=1317] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 82 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=164, chan=130] Received exit status 0
[conn=164, chan=130, pktid=1318] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 82 a....
[conn=164, chan=130] Received channel close
[conn=164, pktid=1183] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=164, chan=130, pktid=1184] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=80, chan=133] Set write buffer limits: low-water=16384, high-water=65536
[conn=80, chan=133] Requesting new SSH session
[conn=80, chan=133] Initial recv window 2097152, packet size 32768
[conn=80, pktid=1197] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=133, pktid=1198] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 85 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=88, chan=133] Set write buffer limits: low-water=16384, high-water=65536
[conn=88, chan=133] Requesting new SSH session
[conn=88, chan=133] Initial recv window 2097152, packet size 32768
[conn=88, pktid=1197] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=133, pktid=1198] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 85 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=84, chan=133] Set write buffer limits: low-water=16384, high-water=65536
[conn=84, chan=133] Requesting new SSH session
[conn=84, chan=133] Initial recv window 2097152, packet size 32768
[conn=84, pktid=1199] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=133, pktid=1200] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 85 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=162, chan=130] Reading from channel started
[conn=162, chan=130] Channel closed
[conn=164, chan=130] Reading from channel started
[conn=80, pktid=1335] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 85 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=80, chan=133] Initial send window 0, packet size 32768
[conn=88, pktid=1336] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 85 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=88, chan=133] Initial send window 0, packet size 32768
[conn=84, pktid=1332] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 85 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=84, chan=133] Initial send window 0, packet size 32768
[conn=164, chan=130] Channel closed
[conn=80, chan=133] Command: sudo /usr/bin/python3 /var/lib/ceph/f148c330-47c9-11ec-9f19-1dfe2cdc6a6d/cephadm.4fd955d0dbf0cdd56104823ef3e950293acaef76608980c02d37021d4c51ee67 check-host
[conn=80, pktid=1199] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=133, pktid=1200] Sent MSG_CHANNEL_REQUEST (98), 174 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 9c 73 75 64 6f 20 2f 75 73 72 2f 62 69 6e 2f ..sudo /usr/bin/
00000020: 70 79 74 68 6f 6e 33 20 2f 76 61 72 2f 6c 69 62 python3 /var/lib
00000030: 2f 63 65 70 68 2f 66 31 34 38 63 33 33 30 2d 34 /ceph/f148c330-4
00000040: 37 63 39 2d 31 31 65 63 2d 39 66 31 39 2d 31 64 7c9-11ec-9f19-1d
00000050: 66 65 32 63 64 63 36 61 36 64 2f 63 65 70 68 61 fe2cdc6a6d/cepha
00000060: 64 6d 2e 34 66 64 39 35 35 64 30 64 62 66 30 63 dm.4fd955d0dbf0c
00000070: 64 64 35 36 31 30 34 38 32 33 65 66 33 65 39 35 dd56104823ef3e95
00000080: 30 32 39 33 61 63 61 65 66 37 36 36 30 38 39 38 0293acaef7660898
00000090: 30 63 30 32 64 33 37 30 32 31 64 34 63 35 31 65 0c02d37021d4c51e
000000a0: 65 36 37 20 63 68 65 63 6b 2d 68 6f 73 74 e67 check-host
[conn=88, chan=133] Command: sudo /usr/bin/python3 /var/lib/ceph/f148c330-47c9-11ec-9f19-1dfe2cdc6a6d/cephadm.4fd955d0dbf0cdd56104823ef3e950293acaef76608980c02d37021d4c51ee67 check-host
[conn=88, pktid=1199] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=133, pktid=1200] Sent MSG_CHANNEL_REQUEST (98), 174 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 9c 73 75 64 6f 20 2f 75 73 72 2f 62 69 6e 2f ..sudo /usr/bin/
00000020: 70 79 74 68 6f 6e 33 20 2f 76 61 72 2f 6c 69 62 python3 /var/lib
00000030: 2f 63 65 70 68 2f 66 31 34 38 63 33 33 30 2d 34 /ceph/f148c330-4
00000040: 37 63 39 2d 31 31 65 63 2d 39 66 31 39 2d 31 64 7c9-11ec-9f19-1d
00000050: 66 65 32 63 64 63 36 61 36 64 2f 63 65 70 68 61 fe2cdc6a6d/cepha
00000060: 64 6d 2e 34 66 64 39 35 35 64 30 64 62 66 30 63 dm.4fd955d0dbf0c
00000070: 64 64 35 36 31 30 34 38 32 33 65 66 33 65 39 35 dd56104823ef3e95
00000080: 30 32 39 33 61 63 61 65 66 37 36 36 30 38 39 38 0293acaef7660898
00000090: 30 63 30 32 64 33 37 30 32 31 64 34 63 35 31 65 0c02d37021d4c51e
000000a0: 65 36 37 20 63 68 65 63 6b 2d 68 6f 73 74 e67 check-host
[conn=84, chan=133] Command: sudo /usr/bin/python3 /var/lib/ceph/f148c330-47c9-11ec-9f19-1dfe2cdc6a6d/cephadm.4fd955d0dbf0cdd56104823ef3e950293acaef76608980c02d37021d4c51ee67 check-host
[conn=84, pktid=1201] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=133, pktid=1202] Sent MSG_CHANNEL_REQUEST (98), 174 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 9c 73 75 64 6f 20 2f 75 73 72 2f 62 69 6e 2f ..sudo /usr/bin/
00000020: 70 79 74 68 6f 6e 33 20 2f 76 61 72 2f 6c 69 62 python3 /var/lib
00000030: 2f 63 65 70 68 2f 66 31 34 38 63 33 33 30 2d 34 /ceph/f148c330-4
00000040: 37 63 39 2d 31 31 65 63 2d 39 66 31 39 2d 31 64 7c9-11ec-9f19-1d
00000050: 66 65 32 63 64 63 36 61 36 64 2f 63 65 70 68 61 fe2cdc6a6d/cepha
00000060: 64 6d 2e 34 66 64 39 35 35 64 30 64 62 66 30 63 dm.4fd955d0dbf0c
00000070: 64 64 35 36 31 30 34 38 32 33 65 66 33 65 39 35 dd56104823ef3e95
00000080: 30 32 39 33 61 63 61 65 66 37 36 36 30 38 39 38 0293acaef7660898
00000090: 30 63 30 32 64 33 37 30 32 31 64 34 63 35 31 65 0c02d37021d4c51e
000000a0: 65 36 37 20 63 68 65 63 6b 2d 68 6f 73 74 e67 check-host
[conn=80, chan=133, pktid=1336] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 85 00 20 00 00 ]..... ..
[conn=80, chan=133] Received window adjust of 2097152 bytes, new window 2097152
[conn=80, chan=133, pktid=1337] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 85 c....
[conn=88, chan=133, pktid=1337] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 85 00 20 00 00 ]..... ..
[conn=88, chan=133] Received window adjust of 2097152 bytes, new window 2097152
[conn=88, chan=133, pktid=1338] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 85 c....
[conn=84, chan=133, pktid=1333] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 85 00 20 00 00 ]..... ..
[conn=84, chan=133] Received window adjust of 2097152 bytes, new window 2097152
[conn=84, chan=133, pktid=1334] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 85 c....
[conn=162, chan=131] Set write buffer limits: low-water=16384, high-water=65536
[conn=162, chan=131] Requesting new SSH session
[conn=162, chan=131] Initial recv window 2097152, packet size 32768
[conn=162, pktid=1185] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=162, chan=131, pktid=1186] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 83 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=80, chan=133] Reading from channel started
[conn=88, chan=133] Reading from channel started
[conn=164, chan=131] Set write buffer limits: low-water=16384, high-water=65536
[conn=164, chan=131] Requesting new SSH session
[conn=164, chan=131] Initial recv window 2097152, packet size 32768
[conn=164, pktid=1185] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=164, chan=131, pktid=1186] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 83 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=84, chan=133] Reading from channel started
[conn=162, pktid=1317] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 83 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=162, chan=131] Initial send window 0, packet size 32768
[conn=162, chan=131] Command: sudo /usr/bin/python3 /var/lib/ceph/f148c330-47c9-11ec-9f19-1dfe2cdc6a6d/cephadm.4fd955d0dbf0cdd56104823ef3e950293acaef76608980c02d37021d4c51ee67 check-host
[conn=162, pktid=1187] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=162, chan=131, pktid=1188] Sent MSG_CHANNEL_REQUEST (98), 174 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 9c 73 75 64 6f 20 2f 75 73 72 2f 62 69 6e 2f ..sudo /usr/bin/
00000020: 70 79 74 68 6f 6e 33 20 2f 76 61 72 2f 6c 69 62 python3 /var/lib
00000030: 2f 63 65 70 68 2f 66 31 34 38 63 33 33 30 2d 34 /ceph/f148c330-4
00000040: 37 63 39 2d 31 31 65 63 2d 39 66 31 39 2d 31 64 7c9-11ec-9f19-1d
00000050: 66 65 32 63 64 63 36 61 36 64 2f 63 65 70 68 61 fe2cdc6a6d/cepha
00000060: 64 6d 2e 34 66 64 39 35 35 64 30 64 62 66 30 63 dm.4fd955d0dbf0c
00000070: 64 64 35 36 31 30 34 38 32 33 65 66 33 65 39 35 dd56104823ef3e95
00000080: 30 32 39 33 61 63 61 65 66 37 36 36 30 38 39 38 0293acaef7660898
00000090: 30 63 30 32 64 33 37 30 32 31 64 34 63 35 31 65 0c02d37021d4c51e
000000a0: 65 36 37 20 63 68 65 63 6b 2d 68 6f 73 74 e67 check-host
[conn=164, pktid=1319] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 83 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=164, chan=131] Initial send window 0, packet size 32768
[conn=164, chan=131] Command: sudo /usr/bin/python3 /var/lib/ceph/f148c330-47c9-11ec-9f19-1dfe2cdc6a6d/cephadm.4fd955d0dbf0cdd56104823ef3e950293acaef76608980c02d37021d4c51ee67 check-host
[conn=164, pktid=1187] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=164, chan=131, pktid=1188] Sent MSG_CHANNEL_REQUEST (98), 174 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 9c 73 75 64 6f 20 2f 75 73 72 2f 62 69 6e 2f ..sudo /usr/bin/
00000020: 70 79 74 68 6f 6e 33 20 2f 76 61 72 2f 6c 69 62 python3 /var/lib
00000030: 2f 63 65 70 68 2f 66 31 34 38 63 33 33 30 2d 34 /ceph/f148c330-4
00000040: 37 63 39 2d 31 31 65 63 2d 39 66 31 39 2d 31 64 7c9-11ec-9f19-1d
00000050: 66 65 32 63 64 63 36 61 36 64 2f 63 65 70 68 61 fe2cdc6a6d/cepha
00000060: 64 6d 2e 34 66 64 39 35 35 64 30 64 62 66 30 63 dm.4fd955d0dbf0c
00000070: 64 64 35 36 31 30 34 38 32 33 65 66 33 65 39 35 dd56104823ef3e95
00000080: 30 32 39 33 61 63 61 65 66 37 36 36 30 38 39 38 0293acaef7660898
00000090: 30 63 30 32 64 33 37 30 32 31 64 34 63 35 31 65 0c02d37021d4c51e
000000a0: 65 36 37 20 63 68 65 63 6b 2d 68 6f 73 74 e67 check-host
[conn=162, chan=131, pktid=1318] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 83 00 20 00 00 ]..... ..
[conn=162, chan=131] Received window adjust of 2097152 bytes, new window 2097152
[conn=162, chan=131, pktid=1319] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 83 c....
[conn=164, chan=131, pktid=1320] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 83 00 20 00 00 ]..... ..
[conn=164, chan=131] Received window adjust of 2097152 bytes, new window 2097152
[conn=164, chan=131, pktid=1321] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 83 c....
[conn=162, chan=131] Reading from channel started
[conn=164, chan=131] Reading from channel started
[conn=80, chan=133, pktid=1338] Received MSG_CHANNEL_EXTENDED_DATA (95), 49 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 24 64 6f 63 _...........$doc
00000010: 6b 65 72 20 28 2f 75 73 72 2f 62 69 6e 2f 64 6f ker (/usr/bin/do
00000020: 63 6b 65 72 29 20 69 73 20 70 72 65 73 65 6e 74 cker) is present
00000030: 0a .
[conn=80, chan=133] Received 36 data bytes from STDERR
[conn=80, chan=133, pktid=1339] Received MSG_CHANNEL_EXTENDED_DATA (95), 54 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 29 73 79 73 _...........)sys
00000010: 74 65 6d 63 74 6c 20 69 73 20 70 72 65 73 65 6e temctl is presen
00000020: 74 0a 6c 76 63 72 65 61 74 65 20 69 73 20 70 72 t.lvcreate is pr
00000030: 65 73 65 6e 74 0a esent.
[conn=80, chan=133] Received 41 data bytes from STDERR
[conn=80, chan=133, pktid=1340] Received MSG_CHANNEL_EXTENDED_DATA (95), 56 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 2b 55 6e 69 _...........+Uni
00000010: 74 20 63 68 72 6f 6e 79 2e 73 65 72 76 69 63 65 t chrony.service
00000020: 20 69 73 20 65 6e 61 62 6c 65 64 20 61 6e 64 20 is enabled and
00000030: 72 75 6e 6e 69 6e 67 0a running.
[conn=80, chan=133] Received 43 data bytes from STDERR
[conn=80, chan=133, pktid=1341] Received MSG_CHANNEL_EXTENDED_DATA (95), 27 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 0e 48 6f 73 _............Hos
00000010: 74 20 6c 6f 6f 6b 73 20 4f 4b 0a t looks OK.
[conn=80, chan=133] Received 14 data bytes from STDERR
[conn=80, chan=133, pktid=1342] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 85 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=80, chan=133] Received exit status 0
[conn=80, chan=133, pktid=1343] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 85 `....
[conn=80, chan=133] Received EOF
[conn=80, chan=133, pktid=1344] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 85 a....
[conn=80, chan=133] Received channel close
[conn=80, pktid=1201] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=80, chan=133, pktid=1202] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=88, chan=133, pktid=1339] Received MSG_CHANNEL_EXTENDED_DATA (95), 70 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 39 64 6f 63 _...........9doc
00000010: 6b 65 72 20 28 2f 75 73 72 2f 62 69 6e 2f 64 6f ker (/usr/bin/do
00000020: 63 6b 65 72 29 20 69 73 20 70 72 65 73 65 6e 74 cker) is present
00000030: 0a 73 79 73 74 65 6d 63 74 6c 20 69 73 20 70 72 .systemctl is pr
00000040: 65 73 65 6e 74 0a esent.
[conn=88, chan=133] Received 57 data bytes from STDERR
[conn=88, chan=133, pktid=1340] Received MSG_CHANNEL_EXTENDED_DATA (95), 33 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 14 6c 76 63 _............lvc
00000010: 72 65 61 74 65 20 69 73 20 70 72 65 73 65 6e 74 reate is present
00000020: 0a .
[conn=88, chan=133] Received 20 data bytes from STDERR
[conn=88, chan=133, pktid=1341] Received MSG_CHANNEL_EXTENDED_DATA (95), 56 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 2b 55 6e 69 _...........+Uni
00000010: 74 20 63 68 72 6f 6e 79 2e 73 65 72 76 69 63 65 t chrony.service
00000020: 20 69 73 20 65 6e 61 62 6c 65 64 20 61 6e 64 20 is enabled and
00000030: 72 75 6e 6e 69 6e 67 0a running.
[conn=88, chan=133] Received 43 data bytes from STDERR
[conn=88, chan=133, pktid=1342] Received MSG_CHANNEL_EXTENDED_DATA (95), 27 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 0e 48 6f 73 _............Hos
00000010: 74 20 6c 6f 6f 6b 73 20 4f 4b 0a t looks OK.
[conn=88, chan=133] Received 14 data bytes from STDERR
[conn=88, chan=133, pktid=1343] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 85 `....
[conn=88, chan=133] Received EOF
[conn=88, chan=133, pktid=1344] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 85 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=88, chan=133] Received exit status 0
[conn=88, chan=133, pktid=1345] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 85 a....
[conn=88, chan=133] Received channel close
[conn=88, pktid=1201] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=88, chan=133, pktid=1202] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=84, chan=133, pktid=1335] Received MSG_CHANNEL_EXTENDED_DATA (95), 70 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 39 64 6f 63 _...........9doc
00000010: 6b 65 72 20 28 2f 75 73 72 2f 62 69 6e 2f 64 6f ker (/usr/bin/do
00000020: 63 6b 65 72 29 20 69 73 20 70 72 65 73 65 6e 74 cker) is present
00000030: 0a 73 79 73 74 65 6d 63 74 6c 20 69 73 20 70 72 .systemctl is pr
00000040: 65 73 65 6e 74 0a esent.
[conn=84, chan=133] Received 57 data bytes from STDERR
[conn=84, chan=133, pktid=1336] Received MSG_CHANNEL_EXTENDED_DATA (95), 33 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 14 6c 76 63 _............lvc
00000010: 72 65 61 74 65 20 69 73 20 70 72 65 73 65 6e 74 reate is present
00000020: 0a .
[conn=84, chan=133] Received 20 data bytes from STDERR
[conn=84, chan=133, pktid=1337] Received MSG_CHANNEL_EXTENDED_DATA (95), 56 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 2b 55 6e 69 _...........+Uni
00000010: 74 20 63 68 72 6f 6e 79 2e 73 65 72 76 69 63 65 t chrony.service
00000020: 20 69 73 20 65 6e 61 62 6c 65 64 20 61 6e 64 20 is enabled and
00000030: 72 75 6e 6e 69 6e 67 0a running.
[conn=84, chan=133] Received 43 data bytes from STDERR
[conn=84, chan=133, pktid=1338] Received MSG_CHANNEL_EXTENDED_DATA (95), 27 bytes
00000000: 5f 00 00 00 85 00 00 00 01 00 00 00 0e 48 6f 73 _............Hos
00000010: 74 20 6c 6f 6f 6b 73 20 4f 4b 0a t looks OK.
[conn=84, chan=133] Received 14 data bytes from STDERR
[conn=84, chan=133, pktid=1339] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 85 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=84, chan=133] Received exit status 0
[conn=84, chan=133, pktid=1340] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 85 `....
[conn=84, chan=133] Received EOF
[conn=84, chan=133, pktid=1341] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 85 a....
[conn=84, chan=133] Received channel close
[conn=84, pktid=1203] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=84, chan=133, pktid=1204] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=162, chan=131, pktid=1320] Received MSG_CHANNEL_EXTENDED_DATA (95), 70 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 39 64 6f 63 _...........9doc
00000010: 6b 65 72 20 28 2f 75 73 72 2f 62 69 6e 2f 64 6f ker (/usr/bin/do
00000020: 63 6b 65 72 29 20 69 73 20 70 72 65 73 65 6e 74 cker) is present
00000030: 0a 73 79 73 74 65 6d 63 74 6c 20 69 73 20 70 72 .systemctl is pr
00000040: 65 73 65 6e 74 0a esent.
[conn=162, chan=131] Received 57 data bytes from STDERR
[conn=162, chan=131, pktid=1321] Received MSG_CHANNEL_EXTENDED_DATA (95), 33 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 14 6c 76 63 _............lvc
00000010: 72 65 61 74 65 20 69 73 20 70 72 65 73 65 6e 74 reate is present
00000020: 0a .
[conn=162, chan=131] Received 20 data bytes from STDERR
[conn=162, chan=131, pktid=1322] Received MSG_CHANNEL_EXTENDED_DATA (95), 56 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 2b 55 6e 69 _...........+Uni
00000010: 74 20 63 68 72 6f 6e 79 2e 73 65 72 76 69 63 65 t chrony.service
00000020: 20 69 73 20 65 6e 61 62 6c 65 64 20 61 6e 64 20 is enabled and
00000030: 72 75 6e 6e 69 6e 67 0a running.
[conn=162, chan=131] Received 43 data bytes from STDERR
[conn=162, chan=131, pktid=1323] Received MSG_CHANNEL_EXTENDED_DATA (95), 27 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 0e 48 6f 73 _............Hos
00000010: 74 20 6c 6f 6f 6b 73 20 4f 4b 0a t looks OK.
[conn=162, chan=131] Received 14 data bytes from STDERR
[conn=162, chan=131, pktid=1324] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 83 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=162, chan=131] Received exit status 0
[conn=162, chan=131, pktid=1325] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 83 `....
[conn=162, chan=131] Received EOF
[conn=162, chan=131, pktid=1326] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 83 a....
[conn=162, chan=131] Received channel close
[conn=162, pktid=1189] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=162, chan=131, pktid=1190] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=164, chan=131, pktid=1322] Received MSG_CHANNEL_EXTENDED_DATA (95), 49 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 24 64 6f 63 _...........$doc
00000010: 6b 65 72 20 28 2f 75 73 72 2f 62 69 6e 2f 64 6f ker (/usr/bin/do
00000020: 63 6b 65 72 29 20 69 73 20 70 72 65 73 65 6e 74 cker) is present
00000030: 0a .
[conn=164, chan=131] Received 36 data bytes from STDERR
[conn=164, chan=131, pktid=1323] Received MSG_CHANNEL_EXTENDED_DATA (95), 54 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 29 73 79 73 _...........)sys
00000010: 74 65 6d 63 74 6c 20 69 73 20 70 72 65 73 65 6e temctl is presen
00000020: 74 0a 6c 76 63 72 65 61 74 65 20 69 73 20 70 72 t.lvcreate is pr
00000030: 65 73 65 6e 74 0a esent.
[conn=164, chan=131] Received 41 data bytes from STDERR
[conn=164, chan=131, pktid=1324] Received MSG_CHANNEL_EXTENDED_DATA (95), 56 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 2b 55 6e 69 _...........+Uni
00000010: 74 20 63 68 72 6f 6e 79 2e 73 65 72 76 69 63 65 t chrony.service
00000020: 20 69 73 20 65 6e 61 62 6c 65 64 20 61 6e 64 20 is enabled and
00000030: 72 75 6e 6e 69 6e 67 0a running.
[conn=164, chan=131] Received 43 data bytes from STDERR
[conn=164, chan=131, pktid=1325] Received MSG_CHANNEL_EXTENDED_DATA (95), 27 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 0e 48 6f 73 _............Hos
00000010: 74 20 6c 6f 6f 6b 73 20 4f 4b 0a t looks OK.
[conn=164, chan=131] Received 14 data bytes from STDERR
[conn=164, chan=131, pktid=1326] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 83 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=164, chan=131] Received exit status 0
[conn=164, chan=131, pktid=1327] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 83 `....
[conn=164, chan=131] Received EOF
[conn=164, chan=131, pktid=1328] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 83 a....
[conn=164, chan=131] Received channel close
[conn=164, pktid=1189] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=164, chan=131, pktid=1190] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=80, chan=133] Channel closed
[conn=88, chan=133] Channel closed
[conn=84, chan=133] Channel closed
[conn=162, chan=131] Channel closed
[conn=164, chan=131] Channel closed
[conn=158, chan=130] Set write buffer limits: low-water=16384, high-water=65536
[conn=158, chan=130] Requesting new SSH session
[conn=158, chan=130] Initial recv window 2097152, packet size 32768
[conn=158, pktid=1179] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=158, chan=130, pktid=1180] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 82 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=158, pktid=1313] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 82 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=158, chan=130] Initial send window 0, packet size 32768
[conn=158, chan=130] Command: sudo which python3
[conn=158, pktid=1181] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=158, chan=130, pktid=1182] Sent MSG_CHANNEL_REQUEST (98), 36 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 12 73 75 64 6f 20 77 68 69 63 68 20 70 79 74 ..sudo which pyt
00000020: 68 6f 6e 33 hon3
[conn=158, chan=130, pktid=1314] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 82 00 20 00 00 ]..... ..
[conn=158, chan=130] Received window adjust of 2097152 bytes, new window 2097152
[conn=158, chan=130, pktid=1315] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 82 c....
[conn=158, chan=130, pktid=1316] Received MSG_CHANNEL_DATA (94), 26 bytes
00000000: 5e 00 00 00 82 00 00 00 11 2f 75 73 72 2f 62 69 ^......../usr/bi
00000010: 6e 2f 70 79 74 68 6f 6e 33 0a n/python3.
[conn=158, chan=130] Received 17 data bytes
[conn=158, chan=130, pktid=1317] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 82 `....
[conn=158, chan=130] Received EOF
[conn=158, chan=130, pktid=1318] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 82 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=158, chan=130] Received exit status 0
[conn=158, chan=130, pktid=1319] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 82 a....
[conn=158, chan=130] Received channel close
[conn=158, pktid=1183] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=158, chan=130, pktid=1184] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=158, chan=130] Reading from channel started
[conn=158, chan=130] Channel closed
[conn=158, chan=131] Set write buffer limits: low-water=16384, high-water=65536
[conn=158, chan=131] Requesting new SSH session
[conn=158, chan=131] Initial recv window 2097152, packet size 32768
[conn=158, pktid=1185] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=158, chan=131, pktid=1186] Sent MSG_CHANNEL_OPEN (90), 24 bytes
00000000: 5a 00 00 00 07 73 65 73 73 69 6f 6e 00 00 00 83 Z....session....
00000010: 00 20 00 00 00 00 80 00 . ......
[conn=158, pktid=1320] Received MSG_CHANNEL_OPEN_CONFIRMATION (91), 17 bytes
00000000: 5b 00 00 00 83 00 00 00 00 00 00 00 00 00 00 80 [...............
00000010: 00 .
[conn=158, chan=131] Initial send window 0, packet size 32768
[conn=158, chan=131] Command: sudo /usr/bin/python3 /var/lib/ceph/f148c330-47c9-11ec-9f19-1dfe2cdc6a6d/cephadm.4fd955d0dbf0cdd56104823ef3e950293acaef76608980c02d37021d4c51ee67 check-host
[conn=158, pktid=1187] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=158, chan=131, pktid=1188] Sent MSG_CHANNEL_REQUEST (98), 174 bytes
00000000: 62 00 00 00 00 00 00 00 04 65 78 65 63 01 00 00 b........exec...
00000010: 00 9c 73 75 64 6f 20 2f 75 73 72 2f 62 69 6e 2f ..sudo /usr/bin/
00000020: 70 79 74 68 6f 6e 33 20 2f 76 61 72 2f 6c 69 62 python3 /var/lib
00000030: 2f 63 65 70 68 2f 66 31 34 38 63 33 33 30 2d 34 /ceph/f148c330-4
00000040: 37 63 39 2d 31 31 65 63 2d 39 66 31 39 2d 31 64 7c9-11ec-9f19-1d
00000050: 66 65 32 63 64 63 36 61 36 64 2f 63 65 70 68 61 fe2cdc6a6d/cepha
00000060: 64 6d 2e 34 66 64 39 35 35 64 30 64 62 66 30 63 dm.4fd955d0dbf0c
00000070: 64 64 35 36 31 30 34 38 32 33 65 66 33 65 39 35 dd56104823ef3e95
00000080: 30 32 39 33 61 63 61 65 66 37 36 36 30 38 39 38 0293acaef7660898
00000090: 30 63 30 32 64 33 37 30 32 31 64 34 63 35 31 65 0c02d37021d4c51e
000000a0: 65 36 37 20 63 68 65 63 6b 2d 68 6f 73 74 e67 check-host
[conn=158, chan=131, pktid=1321] Received MSG_CHANNEL_WINDOW_ADJUST (93), 9 bytes
00000000: 5d 00 00 00 83 00 20 00 00 ]..... ..
[conn=158, chan=131] Received window adjust of 2097152 bytes, new window 2097152
[conn=158, chan=131, pktid=1322] Received MSG_CHANNEL_SUCCESS (99), 5 bytes
00000000: 63 00 00 00 83 c....
[conn=158, chan=131] Reading from channel started
[conn=158, chan=131, pktid=1323] Received MSG_CHANNEL_EXTENDED_DATA (95), 70 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 39 64 6f 63 _...........9doc
00000010: 6b 65 72 20 28 2f 75 73 72 2f 62 69 6e 2f 64 6f ker (/usr/bin/do
00000020: 63 6b 65 72 29 20 69 73 20 70 72 65 73 65 6e 74 cker) is present
00000030: 0a 73 79 73 74 65 6d 63 74 6c 20 69 73 20 70 72 .systemctl is pr
00000040: 65 73 65 6e 74 0a esent.
[conn=158, chan=131] Received 57 data bytes from STDERR
[conn=158, chan=131, pktid=1324] Received MSG_CHANNEL_EXTENDED_DATA (95), 33 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 14 6c 76 63 _............lvc
00000010: 72 65 61 74 65 20 69 73 20 70 72 65 73 65 6e 74 reate is present
00000020: 0a .
[conn=158, chan=131] Received 20 data bytes from STDERR
[conn=158, chan=131, pktid=1325] Received MSG_CHANNEL_EXTENDED_DATA (95), 56 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 2b 55 6e 69 _...........+Uni
00000010: 74 20 63 68 72 6f 6e 79 2e 73 65 72 76 69 63 65 t chrony.service
00000020: 20 69 73 20 65 6e 61 62 6c 65 64 20 61 6e 64 20 is enabled and
00000030: 72 75 6e 6e 69 6e 67 0a running.
[conn=158, chan=131] Received 43 data bytes from STDERR
[conn=158, chan=131, pktid=1326] Received MSG_CHANNEL_EXTENDED_DATA (95), 27 bytes
00000000: 5f 00 00 00 83 00 00 00 01 00 00 00 0e 48 6f 73 _............Hos
00000010: 74 20 6c 6f 6f 6b 73 20 4f 4b 0a t looks OK.
[conn=158, chan=131] Received 14 data bytes from STDERR
[conn=158, chan=131, pktid=1327] Received MSG_CHANNEL_EOF (96), 5 bytes
00000000: 60 00 00 00 83 `....
[conn=158, chan=131] Received EOF
[conn=158, chan=131, pktid=1328] Received MSG_CHANNEL_REQUEST (98), 25 bytes
00000000: 62 00 00 00 83 00 00 00 0b 65 78 69 74 2d 73 74 b........exit-st
00000010: 61 74 75 73 00 00 00 00 00 atus.....
[conn=158, chan=131] Received exit status 0
[conn=158, chan=131, pktid=1329] Received MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 83 a....
[conn=158, chan=131] Received channel close
[conn=158, pktid=1189] Sent MSG_IGNORE (2), 5 bytes
00000000: 02 00 00 00 00 .....
[conn=158, chan=131, pktid=1190] Sent MSG_CHANNEL_CLOSE (97), 5 bytes
00000000: 61 00 00 00 00 a....
[conn=158, chan=131] Channel closed
[conn=1688] Connection failure: [Errno 104] Connection reset by peer
[conn=1688] Aborting connection
The error message is long enough it makes it difficult to use the ceph health detail command at all.
Similar messages will also appear within the mgr logs which can also make them harder to read and can make the logs quite large over time.
It seems to be the message that includes "possibly because python3 is not installed there" that is the worst offender.
Updated by Sebastian Wagner over 2 years ago
- Status changed from New to Resolved
Actions