Actions
Bug #50071
closedunable to reboot gibba nodes using "teuthology-kill -r"
Status:
Closed
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
i have
2021-03-31 07:34:33,105.105 INFO:teuthology.kill:2021-03-31 07:34:33,105.105 INFO:teuthology.nuke.actions:console ready on gibba022.ipmi.sepia.ceph.com
2021-03-31 07:34:33,105.105 INFO:teuthology.kill:2021-03-31 07:34:33,105.105 INFO:teuthology.task.internal:Opening connections...
2021-03-31 07:34:33,131.131 INFO:teuthology.kill:2021-03-31 07:34:33,131.131 ERROR:teuthology.nuke:Could not nuke {'gibba022.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+UU9UBtNVR9zt4CdIHWwW18PMHTavfAicFR4ekd60T17RY0+wKkPTkFft+2+QRCJoYbOHAylvhiAZ44r8gzPHHcOSf1WkaJRNe/HAvbzj7gQ3E4bzUBE3a32PxgMamO7IudtGFjPGUgyJQgflGp/GCf/fYR/iXHiK4xbNXEUlTyDRwW351bOh11g/CimBlLkezeZVnoKrbkrMJTvg3eYZybFMDlkRhT769MhyeC3/IplRnmnxn232dCrJgyNFRA80OY/xs3xr1cGgbB7zUaBV7HJMgTJwTrPZetl0qh+8sMuWkJOaseelESYMKwazUi97T6/UWEyg8ST3ckExWr3R'}
2021-03-31 07:34:33,132.132 INFO:teuthology.kill:Traceback (most recent call last):
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: File "/home/kchai/teuthology/teuthology/nuke/__init__.py", line 292, in nuke_one
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: nuke_helper(ctx, should_unlock, keep_logs, should_reboot)
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: File "/home/kchai/teuthology/teuthology/nuke/__init__.py", line 335, in nuke_helper
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: connect(ctx, None)
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: File "/home/kchai/teuthology/teuthology/task/internal/__init__.py", line 178, in connect
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: rem.connect()
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: File "/home/kchai/teuthology/teuthology/orchestra/remote.py", line 77, in connect
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: self.ssh = connection.connect(**args)
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: File "/home/kchai/teuthology/teuthology/orchestra/connection.py", line 105, in connect
2021-03-31 07:34:33,132.132 INFO:teuthology.kill: ssh.connect(**connect_args)
2021-03-31 07:34:33,133.133 INFO:teuthology.kill: File "/home/kchai/teuthology/virtualenv/lib/python3.6/site-packages/paramiko/client.py", line 423, in connect
2021-03-31 07:34:33,133.133 INFO:teuthology.kill: raise BadHostKeyException(hostname, server_key, our_key)
2021-03-31 07:34:33,133.133 INFO:teuthology.kill:paramiko.ssh_exception.BadHostKeyException: Host key for server 'gibba022.front.sepia.ceph.com' does not match: got 'AAAAB3NzaC1yc2EAAAADAQABAAABgQC5z3r0mshOrhMWwLsaEzsiJSXZrpuznkDsVl62pFzTHeW+2I3vP9BCagSgHP/jTHazbwFQvc0tCEGaLfJBCXncvFki42aXo6cevlGewTY3ovNtkr/j5PrXtQZ1S4PjVPG06ec7FfBRGoju+7yJT92dlKmppbWkZmCvJbdOyNX7aHlIyWYW1h3HTG4eelX4bTR0ZC9aBQvUBXrzek0JIXeUu3D7Wq+uK4eUoYvATzx2JfpMqBwFRpVTmIb8RFdam1cnlo94Z6l1Xx1CYuNPx3EZ6HJRq17Dzq3szlSPZJkUW/D2oSplFtmrP+Tf1xEnLpOUiIjqlrSINVcJCzsKCuG8PoI1FaThoGTOW1g8ARy4TVtTy5FrNPO7z+vMkp3oZXY53Bic5BWoLnrQCStsCcS5TRAErfwCazfsbcZG/apX4Zq7fSL5dUjcGxnJY8zGH1m4Y94ePJYjszVhXWNe7b2sNdhOdwyI0D4qOmxchbxrUnGK99iKIynS2rG0J+Arl8k=', expected 'AAAAB3NzaC1yc2EAAAADAQABAAABAQC+UU9UBtNVR9zt4CdIHWwW18PMHTavfAicFR4ekd60T17RY0+wKkPTkFft+2+QRCJoYbOHAylvhiAZ44r8gzPHHcOSf1WkaJRNe/HAvbzj7gQ3E4bzUBE3a32PxgMamO7IudtGFjPGUgyJQgflGp/GCf/fYR/iXHiK4xbNXEUlTyDRwW351bOh11g/CimBlLkezeZVnoKrbkrMJTvg3eYZybFMDlkRhT769MhyeC3/IplRnmnxn232dCrJgyNFRA80OY/xs3xr1cGgbB7zUaBV7HJMgTJwTrPZetl0qh+8sMuWkJOaseelESYMKwazUi97T6/UWEyg8ST3ckExWr3R'
2021-03-31 07:37:54,724.724 INFO:teuthology.kill:2021-03-31 07:37:54,723.723 INFO:teuthology.misc:trying to connect to ubuntu@gibba011.front.sepia.ceph.com 2021-03-31 07:37:54,724.724 INFO:teuthology.kill:2021-03-31 07:37:54,724.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-31 07:37:54,914.914 INFO:teuthology.kill:2021-03-31 07:37:54,914.914 ERROR:teuthology.orchestra.connection:Error authenticating with gibba026.front.sepia.ceph.com: Authentication failed. 2021-03-31 07:37:55,021.021 INFO:teuthology.kill:2021-03-31 07:37:55,020.020 ERROR:teuthology.orchestra.connection:Error authenticating with gibba020.front.sepia.ceph.com: Authentication failed. 2021-03-31 07:37:55,044.044 INFO:teuthology.kill:2021-03-31 07:37:55,043.043 INFO:teuthology.misc:trying to connect to ubuntu@gibba016.front.sepia.ceph.com 2021-03-31 07:37:55,044.044 INFO:teuthology.kill:2021-03-31 07:37:55,044.044 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-31 07:37:55,236.236 INFO:teuthology.kill:2021-03-31 07:37:55,236.236 INFO:teuthology.misc:trying to connect to ubuntu@gibba034.front.sepia.ceph.com 2021-03-31 07:37:55,237.237 INFO:teuthology.kill:2021-03-31 07:37:55,236.236 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-31 07:37:55,408.408 INFO:teuthology.kill:2021-03-31 07:37:55,408.408 INFO:teuthology.misc:trying to connect to ubuntu@gibba032.front.sepia.ceph.com 2021-03-31 07:37:55,408.408 INFO:teuthology.kill:2021-03-31 07:37:55,408.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-31 07:37:55,460.460 INFO:teuthology.kill:2021-03-31 07:37:55,460.460 INFO:teuthology.misc:trying to connect to ubuntu@gibba015.front.sepia.ceph.com 2021-03-31 07:37:55,461.461 INFO:teuthology.kill:2021-03-31 07:37:55,460.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-31 07:37:56,292.292 INFO:teuthology.kill:2021-03-31 07:37:56,291.291 INFO:teuthology.misc:trying to connect to ubuntu@gibba003.front.sepia.ceph.com 2021-03-31 07:37:56,292.292 INFO:teuthology.kill:2021-03-31 07:37:56,292.292 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-31 07:37:56,429.429 INFO:teuthology.kill:2021-03-31 07:37:56,429.429 ERROR:teuthology.orchestra.connection:Error authenticating with gibba036.front.sepia.ceph.com: Authentication timeout. 2021-03-31 07:37:56,646.646 INFO:teuthology.kill:2021-03-31 07:37:56,645.645 ERROR:teuthology.orchestra.connection:Error authenticating with gibba026.front.sepia.ceph.com: Authentication failed. 2021-03-31 07:37:57,256.256 INFO:teuthology.kill:2021-03-31 07:37:57,256.256 ERROR:teuthology.orchestra.connection:Error authenticating with gibba032.front.sepia.ceph.com: Authentication failed.
Updated by Kefu Chai about 3 years ago
- Priority changed from Normal to High
raise the priority to high, as it seems that some test nodes cannot be freed.
Updated by Laura Flores 11 months ago
- Status changed from New to Closed
There's not enough info here, and we don't use gibba for teuthology testing anymore.
If the issue comes up again, please file a new tracker.
Actions