Actions
Bug #27053
openqa: thrashosds: "[ERR] : 2.0 has 1 objects unfound and apparently lost"
Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
kcephfs, rados
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
This is for 12.2.8
Run: http://pulpito.ceph.com/yuriw-2018-08-21_16:17:40-rados-luminous-distro-basic-smithi/
Job: 2932441
Logs: http://qa-proxy.ceph.com/teuthology/yuriw-2018-08-21_16:17:40-rados-luminous-distro-basic-smithi/2932441/teuthology.log
2018-08-21T22:49:38.349 INFO:tasks.ceph.osd.3.smithi196.stderr:2018-08-21 22:49:38.351396 7f34b3ccd700 -1 received signal: Hangup from PID: 34513 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 UID: 0 2018-08-21T22:49:38.361 INFO:teuthology.orchestra.run.smithi196:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops' 2018-08-21T22:49:38.455 INFO:teuthology.orchestra.run.smithi143.stderr:2018-08-21 22:49:38.456908 7f0bb1fa1700 -1 WARNING: all dangerous and experimental features are enabled. 2018-08-21T22:49:38.467 INFO:tasks.ceph.osd.4.smithi196.stderr:2018-08-21 22:49:38.469281 7f59672ad700 -1 received signal: Hangup from PID: 12522 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 UID: 0 2018-08-21T22:49:38.472 INFO:teuthology.orchestra.run.smithi143.stderr:2018-08-21 22:49:38.474731 7f0bb1fa1700 -1 WARNING: all dangerous and experimental features are enabled. 2018-08-21T22:49:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:osd.0: osd_enable_op_tracker = 'true' 2018-08-21T22:49:38.510 INFO:teuthology.orchestra.run.smithi143.stderr:osd.1: osd_enable_op_tracker = 'true' 2018-08-21T22:49:38.521 INFO:teuthology.orchestra.run.smithi143.stderr:osd.2: osd_enable_op_tracker = 'true' 2018-08-21T22:49:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:osd.3: osd_enable_op_tracker = 'true' 2018-08-21T22:49:38.551 INFO:tasks.ceph.osd.3.smithi196.stderr:2018-08-21 22:49:38.553689 7f34b3ccd700 -1 received signal: Hangup from PID: 34513 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 UID: 0 2018-08-21T22:49:38.553 INFO:teuthology.orchestra.run.smithi143.stderr:osd.4: osd_enable_op_tracker = 'true' 2018-08-21T22:49:38.556 INFO:teuthology.orchestra.run.smithi143:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight' 2018-08-21T22:49:38.563 INFO:tasks.radosbench.radosbench.0.smithi196.stdout:22266 16 648074 648058 29.1013 0 - 0.0123366 2018-08-21T22:49:38.566 INFO:teuthology.orchestra.run.smithi143.stderr:osd.5: osd_enable_op_tracker = 'true' 2018-08-21T22:49:38.653 INFO:tasks.ceph.osd.4.smithi196.stderr:2018-08-21 22:49:38.654849 7f59672ad700 -1 received signal: Hangup from PID: 12522 task name: /usr/bin/python /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 UID: 0 2018-08-21T22:49:38.682 INFO:teuthology.orchestra.run.smithi143:Running: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops' 2018-08-21T22:49:38.686 ERROR:teuthology.run_tasks:Manager failed: radosbench Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 159, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__ self.gen.next() File "/home/teuthworker/src/github.com_ceph_ceph_luminous/qa/tasks/radosbench.py", line 132, in task run.wait(radosbench.itervalues(), timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 441, in wait check_time() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 132, in __call__ raise MaxWhileTries(error_msg) MaxWhileTries: reached maximum tries (3650) after waiting for 21900 seconds 2018-08-21T22:49:38.687 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2018-08-21T22:49:38.927 INFO:tasks.thrashosds:joining thrashosds 2018-08-21T22:49:38.928 INFO:tasks.ceph.osd.3.smithi196.stderr:2018-08-21 22:49:38.890808 7f34c1dea700 -1 log_channel(cluster) log [ERR] : 2.0 has 1 objects unfound and apparently lost
Actions