Bug #17027
Updated by Yuri Weinstein over 7 years ago
teuthology-nuke failes to nuke nodes command comman line: <pre> for owner in $(teuthology-lock --summary | grep -i scheduled | awk '{ print $4 }' | sort -u); do echo $owner; echo "========"; teuthology-nuke --stale --owner $owner --unlock; echo "======== "; done </pre> <pre> 2016-08-15 15:05:43,100.100 ERROR:teuthology.nuke:Could not nuke the following targets: targets: mira027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw1XIeSo/m6bc6Ie5RFuV36EzP71fDosKlxkYPERNrJw5qtwysmPBBXicat8bnEwG93urM/4iD4zPla2j86GEPg4xMHbNY9xdlITZdBvsEz5XIUzKx0Y3aHZoxBD6kROwvEqT9SWS5V45T1I1SVkjrBax0ccZouT9uFW/JIME4rYJ3JNfVLQXCo6AQ0B+k0F6mV7gWXxlKnprLmjfEWj4vZQoS6mj5o2zGpmRTmIKH2aqJTtXtbPVgywzQX9Ls379yMetDI/ABqmhNaRvoUdJYW8ha7JMSRy7BJ/XB1mM7XIJebccM/7hihxGofln7OJ/OImtt602rtffNnLgKH/1t mira075.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3cHQzOUD8E8DsgSQHut4srF8aA+ba+J0Y/mkWmXnjPZqp/4vwxqyS+LE/V7tbNko+vQqVOhpq7ykI8KWcwyfA5cpfUy9cfIsPNJlgIW5W1qq4L3oS32Z91gx9vO9AocoPnPXg119VcOdIfE9CpGP3iFYmYH+N8GAAPqN212zs6SSs9gKMlX6b32y7R4iKOD0zX8PHudV8BxhFpwgJL5Cwn/KAovXYwJQLDrkcj+DjDFfhJAh0mIF2ngPEKezsPOi1g2EvpGwON0tcdRq2PGMJ61dvPILT7Xc9YfPehfJch8r4xAaJz6u9KzQEkR2PHwsJAbQzFGNgGcAS+hknEbRb mira077.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDM02+v5Rx15O2xxLyabm6MI0h0Ud75CrN/7O6a1og09L1yWzLy+ae1kSksHwokPYWzwmM+eai+B66KPnK1VwO5DUMogDs5/UwWUCH7u4YRnTtKdNVTuYBDfHVRBolAaQIPGbVlOPUrDhAwec27dLyxyNk4wfz2liAMJwHruhyJd8RhzalsYrRH8Z+jRq3xG/4HyepRRCTqAnkaEqAE6kBRwIIIebXXRJtQwcoVe7a5qUa2NC+54FkmuWnq/THqhVogbmKn6EDKC3y/PqLDZ+1De3Znx6xC466XIz8jVysRHvpafYYqAxfDbfYF/V/iaguqV3Oe4JFlxsxy7GdDM7g7 mira080.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDS/QBiL/olJ48f2jPj2f+tWmylm55rj42TxsU5k35FxIniDuiSP8mT3eNJoSKODxVNuYaFochSrCRu1V7N3uYiCxRlJfI3OiL+btWCmaZ75CJlCS7CGJLYZRK7Z6An0doZK8QSGHo7YXHppkv/tRDs2JtlOE+Z2K59VwRrL0TJZ6ENfafaWSuEcOlwbPlXMgwIoMWW3nvxw70IeZ6/VuWYHA3PuzVorm6hs1YRbiGNH0kixzPYmks/r7jnIVtgXDPLFBSse6PpaIkC/MHAscweO7nfFgkx0XM6cqsaf5eb5uRPysrMcLpKx5DIPQwkmcxA4QgnBt+16QVpl2bXuNen mira086.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMLAqZF/xtRYiKLOsl7PuJkkNdkUoNstui1CC76K9W0K17e5D+97Z1v7sfAavKH+IOqR9ntetseC6oYDM50+sfuuRjdTGfuNaxuyfA801Ni7wQfi61cgKf98zUr3V55Kr+QdpJs/dOgShq+GjOAX4bQMZu649ya4OkK65PIjicAtZj3YxFyj6IhpMTOGoDmqGp4nd+VOV5JWCsND/JeYDcgdcA/e3ONKgDX3AkaVU3jTzkmiStIeN9pIlc2qq32l/43CFWVUpVaDWtbhSD5SpdG5QuNnW2DoAIEjYboktc2fEmCXKpKDjsXbYWwqNhTH1TFvVJ2VIsZVafWPlQHhxx mira105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCZthumZGtMsYYb9/GWrURIBU7uSZRirXIfvdKrguvjA25j1EOovfs3JUam6B/0PlSarELQcA/qzDTyTuyz3zboEfcQwBlKMycmIbKFBbUDlJJQibXuA468mgE5pPW4jt38JASwXNbQajtEhpQ0k44sZ2Vtf2AyM2NluLvUzhGOymzyEX499zR7N5edu+SBf71tHCNoAOuLlKvPkULigX2RsWeKWqLsF1e1dtoM3zdppxCnnoB1hc9ocDpWCzfKCZMO6juV0rJNmfG59Iy3e7IcsAnuIkq02j3aQOT3vwooVhuVU31dBqaFTTkWcc5k2almkmKhxkRJOZGs51JXsZPN =========================================== 2016-08-15 15:02:24,835.835 ERROR:teuthology.nuke:Could not nuke {u'mira080.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDS/QBiL/olJ48f2jPj2f+tWmylm55rj42TxsU5k35FxIniDuiSP8mT3eNJoSKODxVNuYaFochSrCRu1V7N3uYiCxRlJfI3OiL+btWCmaZ75CJlCS7CGJLYZRK7Z6An0doZK8QSGHo7YXHppkv/tRDs2JtlOE+Z2K59VwRrL0TJZ6ENfafaWSuEcOlwbPlXMgwIoMWW3nvxw70IeZ6/VuWYHA3PuzVorm6hs1YRbiGNH0kixzPYmks/r7jnIVtgXDPLFBSse6PpaIkC/MHAscweO7nfFgkx0XM6cqsaf5eb5uRPysrMcLpKx5DIPQwkmcxA4QgnBt+16QVpl2bXuNen'} Traceback (most recent call last): File "/home/yuriw/teuthology/teuthology/nuke.py", line 636, in nuke_one nuke_helper(ctx, should_unlock) File "/home/yuriw/teuthology/teuthology/nuke.py", line 688, in nuke_helper connect(ctx, None) File "/home/yuriw/teuthology/teuthology/task/internal.py", line 343, in connect rem.connect() File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 63, in connect self.ssh = connection.connect(**args) File "/home/yuriw/teuthology/teuthology/orchestra/connection.py", line 112, in connect ssh.connect(**connect_args) File "/home/yuriw/teuthology/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 324, in connect raise NoValidConnectionsError(errors) NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.7.126 2016-08-15 15:02:24,841.841 ERROR:teuthology.nuke:Could not nuke {u'mira105.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCZthumZGtMsYYb9/GWrURIBU7uSZRirXIfvdKrguvjA25j1EOovfs3JUam6B/0PlSarELQcA/qzDTyTuyz3zboEfcQwBlKMycmIbKFBbUDlJJQibXuA468mgE5pPW4jt38JASwXNbQajtEhpQ0k44sZ2Vtf2AyM2NluLvUzhGOymzyEX499zR7N5edu+SBf71tHCNoAOuLlKvPkULigX2RsWeKWqLsF1e1dtoM3zdppxCnnoB1hc9ocDpWCzfKCZMO6juV0rJNmfG59Iy3e7IcsAnuIkq02j3aQOT3vwooVhuVU31dBqaFTTkWcc5k2almkmKhxkRJOZGs51JXsZPN'} Traceback (most recent call last): File "/home/yuriw/teuthology/teuthology/nuke.py", line 636, in nuke_one nuke_helper(ctx, should_unlock) File "/home/yuriw/teuthology/teuthology/nuke.py", line 688, in nuke_helper connect(ctx, None) File "/home/yuriw/teuthology/teuthology/task/internal.py", line 343, in connect rem.connect() File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 63, in connect self.ssh = connection.connect(**args) File "/home/yuriw/teuthology/teuthology/orchestra/connection.py", line 112, in connect ssh.connect(**connect_args) File "/home/yuriw/teuthology/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 324, in connect raise NoValidConnectionsError(errors) NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.8.110 2016-08-15 15:02:24,841.841 ERROR:teuthology.nuke:Could not nuke {u'mira027.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw1XIeSo/m6bc6Ie5RFuV36EzP71fDosKlxkYPERNrJw5qtwysmPBBXicat8bnEwG93urM/4iD4zPla2j86GEPg4xMHbNY9xdlITZdBvsEz5XIUzKx0Y3aHZoxBD6kROwvEqT9SWS5V45T1I1SVkjrBax0ccZouT9uFW/JIME4rYJ3JNfVLQXCo6AQ0B+k0F6mV7gWXxlKnprLmjfEWj4vZQoS6mj5o2zGpmRTmIKH2aqJTtXtbPVgywzQX9Ls379yMetDI/ABqmhNaRvoUdJYW8ha7JMSRy7BJ/XB1mM7XIJebccM/7hihxGofln7OJ/OImtt602rtffNnLgKH/1t'} Traceback (most recent call last): File "/home/yuriw/teuthology/teuthology/nuke.py", line 636, in nuke_one nuke_helper(ctx, should_unlock) File "/home/yuriw/teuthology/teuthology/nuke.py", line 688, in nuke_helper connect(ctx, None) File "/home/yuriw/teuthology/teuthology/task/internal.py", line 343, in connect rem.connect() File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 63, in connect self.ssh = connection.connect(**args) File "/home/yuriw/teuthology/teuthology/orchestra/connection.py", line 112, in connect ssh.connect(**connect_args) File "/home/yuriw/teuthology/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 324, in connect raise NoValidConnectionsError(errors) NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.4.124 2016-08-15 15:02:24,841.841 ERROR:teuthology.nuke:Could not nuke {u'mira075.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3cHQzOUD8E8DsgSQHut4srF8aA+ba+J0Y/mkWmXnjPZqp/4vwxqyS+LE/V7tbNko+vQqVOhpq7ykI8KWcwyfA5cpfUy9cfIsPNJlgIW5W1qq4L3oS32Z91gx9vO9AocoPnPXg119VcOdIfE9CpGP3iFYmYH+N8GAAPqN212zs6SSs9gKMlX6b32y7R4iKOD0zX8PHudV8BxhFpwgJL5Cwn/KAovXYwJQLDrkcj+DjDFfhJAh0mIF2ngPEKezsPOi1g2EvpGwON0tcdRq2PGMJ61dvPILT7Xc9YfPehfJch8r4xAaJz6u9KzQEkR2PHwsJAbQzFGNgGcAS+hknEbRb'} Traceback (most recent call last): File "/home/yuriw/teuthology/teuthology/nuke.py", line 636, in nuke_one nuke_helper(ctx, should_unlock) File "/home/yuriw/teuthology/teuthology/nuke.py", line 688, in nuke_helper connect(ctx, None) File "/home/yuriw/teuthology/teuthology/task/internal.py", line 343, in connect rem.connect() File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 63, in connect self.ssh = connection.connect(**args) File "/home/yuriw/teuthology/teuthology/orchestra/connection.py", line 112, in connect ssh.connect(**connect_args) File "/home/yuriw/teuthology/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 324, in connect raise NoValidConnectionsError(errors) NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.7.136 2016-08-15 15:02:24,841.841 ERROR:teuthology.nuke:Could not nuke {u'mira086.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMLAqZF/xtRYiKLOsl7PuJkkNdkUoNstui1CC76K9W0K17e5D+97Z1v7sfAavKH+IOqR9ntetseC6oYDM50+sfuuRjdTGfuNaxuyfA801Ni7wQfi61cgKf98zUr3V55Kr+QdpJs/dOgShq+GjOAX4bQMZu649ya4OkK65PIjicAtZj3YxFyj6IhpMTOGoDmqGp4nd+VOV5JWCsND/JeYDcgdcA/e3ONKgDX3AkaVU3jTzkmiStIeN9pIlc2qq32l/43CFWVUpVaDWtbhSD5SpdG5QuNnW2DoAIEjYboktc2fEmCXKpKDjsXbYWwqNhTH1TFvVJ2VIsZVafWPlQHhxx'} Traceback (most recent call last): File "/home/yuriw/teuthology/teuthology/nuke.py", line 636, in nuke_one nuke_helper(ctx, should_unlock) File "/home/yuriw/teuthology/teuthology/nuke.py", line 688, in nuke_helper connect(ctx, None) File "/home/yuriw/teuthology/teuthology/task/internal.py", line 343, in connect rem.connect() File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 63, in connect self.ssh = connection.connect(**args) File "/home/yuriw/teuthology/teuthology/orchestra/connection.py", line 112, in connect ssh.connect(**connect_args) File "/home/yuriw/teuthology/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 324, in connect raise NoValidConnectionsError(errors) NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.7.112 2016-08-15 15:02:24,841.841 ERROR:teuthology.nuke:Could not nuke {u'mira077.front.sepia.ceph.com': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDM02+v5Rx15O2xxLyabm6MI0h0Ud75CrN/7O6a1og09L1yWzLy+ae1kSksHwokPYWzwmM+eai+B66KPnK1VwO5DUMogDs5/UwWUCH7u4YRnTtKdNVTuYBDfHVRBolAaQIPGbVlOPUrDhAwec27dLyxyNk4wfz2liAMJwHruhyJd8RhzalsYrRH8Z+jRq3xG/4HyepRRCTqAnkaEqAE6kBRwIIIebXXRJtQwcoVe7a5qUa2NC+54FkmuWnq/THqhVogbmKn6EDKC3y/PqLDZ+1De3Znx6xC466XIz8jVysRHvpafYYqAxfDbfYF/V/iaguqV3Oe4JFlxsxy7GdDM7g7'} Traceback (most recent call last): File "/home/yuriw/teuthology/teuthology/nuke.py", line 636, in nuke_one nuke_helper(ctx, should_unlock) File "/home/yuriw/teuthology/teuthology/nuke.py", line 688, in nuke_helper connect(ctx, None) File "/home/yuriw/teuthology/teuthology/task/internal.py", line 343, in connect rem.connect() File "/home/yuriw/teuthology/teuthology/orchestra/remote.py", line 63, in connect self.ssh = connection.connect(**args) File "/home/yuriw/teuthology/teuthology/orchestra/connection.py", line 112, in connect ssh.connect(**connect_args) File "/home/yuriw/teuthology/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 324, in connect raise NoValidConnectionsError(errors) NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 172.21.7.132 </pre>