Actions
Bug #54517
closedscrub/osd-scrub-snaps.sh: TEST FAILED WITH 1 ERRORS
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
/a/teuthology-archive/yuriw-2022-03-04_21:56:41-rados-wip-yuri4-testing-2022-03-03-1448-distro-default-smithi/6721751
2022-03-05T05:48:04.554 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:705: TEST_scrub_snaps: ERRORS=1
2022-03-05T05:48:04.554 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:709: TEST_scrub_snaps: '[' 1 '!=' 0 ']'
2022-03-05T05:48:04.555 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:711: TEST_scrub_snaps: echo 'TEST FAILED WITH 1 ERRORS'
2022-03-05T05:48:04.555 INFO:tasks.workunit.client.0.smithi111.stdout:TEST FAILED WITH 1 ERRORS
2022-03-05T05:48:04.558 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:712: TEST_scrub_snaps: return 1
2022-03-05T05:48:04.558 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh:39: run: return 1
2022-03-05T05:48:04.558 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: main: code=1
2022-03-05T05:48:04.558 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: main: teardown td/osd-scrub-snaps 1
2022-03-05T05:48:04.559 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-scrub-snaps
2022-03-05T05:48:04.559 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=1
2022-03-05T05:48:04.559 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-scrub-snaps KILL
2022-03-05T05:48:04.560 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace
2022-03-05T05:48:04.560 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true
2022-03-05T05:48:04.560 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true
2022-03-05T05:48:04.560 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true
2022-03-05T05:48:04.560 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace
2022-03-05T05:48:04.566 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0
2022-03-05T05:48:04.567 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname
2022-03-05T05:48:04.568 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']'
2022-03-05T05:48:04.569 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T .
2022-03-05T05:48:04.571 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']'
2022-03-05T05:48:04.571 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no
2022-03-05T05:48:04.572 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern
2022-03-05T05:48:04.573 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core
2022-03-05T05:48:04.573 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']'
2022-03-05T05:48:04.574 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$'
2022-03-05T05:48:04.574 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core
2022-03-05T05:48:04.575 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump
2022-03-05T05:48:04.578 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 1 = 1 ']'
2022-03-05T05:48:04.579 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: '[' -n '' ']'
2022-03-05T05:48:04.579 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:194: teardown: mkdir -p /home/ubuntu/cephtest/archive/log
2022-03-05T05:48:04.580 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:195: teardown: mv td/osd-scrub-snaps/mgr.x.log td/osd-scrub-snaps/mon.a.log td/osd-scrub-snaps/osd.0.log /home/ubuntu/cephtest/archive/log
2022-03-05T05:48:04.582 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-scrub-snaps
2022-03-05T05:48:04.591 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir
2022-03-05T05:48:04.592 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']'
2022-03-05T05:48:04.592 INFO:tasks.workunit.client.0.smithi111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.494700
2022-03-05T05:48:04.592 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.494700
2022-03-05T05:48:04.593 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']'
2022-03-05T05:48:04.594 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0
2022-03-05T05:48:04.594 INFO:tasks.workunit.client.0.smithi111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: main: return 1
2022-03-05T05:48:04.595 DEBUG:teuthology.orchestra.run:got remote process result: 1
2022-03-05T05:48:04.596 INFO:tasks.workunit:Stopping ['scrub'] on client.0...
2022-03-05T05:48:04.596 DEBUG:teuthology.orchestra.run.smithi111:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0
2022-03-05T05:48:04.877 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/run_tasks.py", line 91, in run_tasks
manager = run_one_task(taskname, ctx=ctx, config=config)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/run_tasks.py", line 70, in run_one_task
return task(**kwargs)
File "/home/teuthworker/src/github.com_ceph_ceph-c_c8f79f870e0d6a996c92d420e6256d312bac1c7c/qa/tasks/workunit.py", line 148, in task
cleanup=cleanup)
File "/home/teuthworker/src/github.com_ceph_ceph-c_c8f79f870e0d6a996c92d420e6256d312bac1c7c/qa/tasks/workunit.py", line 298, in _spawn_on_all_clients
timeout=timeout)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/parallel.py", line 84, in __exit__
for result in self:
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/parallel.py", line 98, in __next__
resurrect_traceback(result)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/parallel.py", line 30, in resurrect_traceback
raise exc.exc_info[1]
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/parallel.py", line 23, in capture_traceback
return func(*args, **kwargs)
File "/home/teuthworker/src/github.com_ceph_ceph-c_c8f79f870e0d6a996c92d420e6256d312bac1c7c/qa/tasks/workunit.py", line 427, in _run_tests
label="workunit test {workunit}".format(workunit=workunit)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/orchestra/remote.py", line 509, in run
r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/orchestra/run.py", line 455, in run
r.wait()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/orchestra/run.py", line 161, in wait
self._raise_for_status()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_eeaeee38264fad8a01fd1ce912d8908676ed64bc/teuthology/orchestra/run.py", line 183, in _raise_for_status
node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed (workunit test scrub/osd-scrub-snaps.sh) on smithi111 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c8f79f870e0d6a996c92d420e6256d312bac1c7c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-scrub-snaps.sh'
Updated by Laura Flores about 2 years ago
- Assignee changed from David Zafman to Ronen Friedman
Updated by Laura Flores about 2 years ago
- Related to Bug #54458: osd-scrub-snaps.sh: TEST_scrub_snaps failed due to malformed log message added
Updated by Laura Flores about 2 years ago
Ronen this looks a lot like https://tracker.ceph.com/issues/54458, just with a slightly different output. Can you check if it's the same? And if so mark this one as a duplicate (or let me know).
This Tracker:
...found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: 1,2,3,4,5,6,7 ...repaired'
Resolved issue 54458:
...found snap mapper error on pg 1.0 oid 1:461f8b5e:::obj16:7 snaps missing in mapper, should be: 1,2,3,4,5,6,7 was r -2...repaired
Updated by Laura Flores about 2 years ago
- Related to deleted (Bug #54458: osd-scrub-snaps.sh: TEST_scrub_snaps failed due to malformed log message)
Updated by Laura Flores about 2 years ago
- Has duplicate Bug #54458: osd-scrub-snaps.sh: TEST_scrub_snaps failed due to malformed log message added
Actions