Project

General

Profile

Actions

Bug #12544

open

CEPH_QA_SUITE/AARCH64: Command failed (workunit test rbd/kernel.sh)

Added by Yazen Ghannam over 8 years ago. Updated over 7 years ago.

Status:
New
Priority:
Low
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
aarch64 arm64
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
krbd
Crash signature (v1):
Crash signature (v2):

Description

The following test fails on AArch64:
krbd:rbd-nomount/{clusters/fixed-3.yaml conf.yaml fs/btrfs.yaml install/ceph.yaml msgr-failures/many.yaml tasks/rbd_kernel.yaml}

Sample Output:

2015-07-29T17:40:00.137 INFO:teuthology.orchestra.run.msteuth17:Running (workunit test rbd/kernel.sh): 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=workunit-fixes-for-aarch64 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/rbd/kernel.sh'
2015-07-29T17:40:00.155 INFO:tasks.workunit.client.0.msteuth17.stderr:+ CEPH_SECRET_FILE=
2015-07-29T17:40:00.156 INFO:tasks.workunit.client.0.msteuth17.stderr:+ CEPH_ID=0
2015-07-29T17:40:00.156 INFO:tasks.workunit.client.0.msteuth17.stderr:+ SECRET_ARGS=
2015-07-29T17:40:00.157 INFO:tasks.workunit.client.0.msteuth17.stderr:+ '[' '!' -z ']'
2015-07-29T17:40:00.158 INFO:tasks.workunit.client.0.msteuth17.stderr:+ TMP_FILES='/tmp/img1 /tmp/img1.small /tmp/img1.snap1 /tmp/img1.export /tmp/img1.trunc'
2015-07-29T17:40:00.159 INFO:tasks.workunit.client.0.msteuth17.stderr:+ '[' -d /sys/bus/rbd ']'
2015-07-29T17:40:00.160 INFO:tasks.workunit.client.0.msteuth17.stderr:+ clean_up
2015-07-29T17:40:00.161 INFO:tasks.workunit.client.0.msteuth17.stderr:+ udevadm settle
2015-07-29T17:40:00.164 INFO:tasks.workunit.client.0.msteuth17.stderr:+ '[' -e /dev/rbd/rbd/testimg1@snap1 ']'
2015-07-29T17:40:00.165 INFO:tasks.workunit.client.0.msteuth17.stderr:+ true
2015-07-29T17:40:00.166 INFO:tasks.workunit.client.0.msteuth17.stderr:+ '[' -e /dev/rbd/rbd/testimg1 ']'
2015-07-29T17:40:00.167 INFO:tasks.workunit.client.0.msteuth17.stderr:+ udevadm settle
2015-07-29T17:40:00.168 INFO:tasks.workunit.client.0.msteuth17.stderr:+ sudo chown root /sys/bus/rbd/add /sys/bus/rbd/remove
2015-07-29T17:40:00.170 INFO:tasks.workunit.client.0.msteuth17.stderr:+ rbd ls
2015-07-29T17:40:00.171 INFO:tasks.workunit.client.0.msteuth17.stderr:+ grep testimg1
2015-07-29T17:40:00.240 INFO:tasks.workunit.client.0.msteuth17.stderr:+ true
2015-07-29T17:40:00.241 INFO:tasks.workunit.client.0.msteuth17.stderr:+ sudo rm -f /tmp/img1 /tmp/img1.small /tmp/img1.snap1 /tmp/img1.export /tmp/img1.trunc
2015-07-29T17:40:00.250 INFO:tasks.workunit.client.0.msteuth17.stderr:+ trap clean_up INT TERM EXIT
2015-07-29T17:40:00.251 INFO:tasks.workunit.client.0.msteuth17.stderr:+ sudo chown ubuntu /sys/bus/rbd/add /sys/bus/rbd/remove
2015-07-29T17:40:00.260 INFO:tasks.workunit.client.0.msteuth17.stderr:+ dd if=/bin/sh of=/tmp/img1 bs=1k count=1 seek=10
2015-07-29T17:40:00.265 INFO:tasks.workunit.client.0.msteuth17.stderr:1+0 records in
2015-07-29T17:40:00.266 INFO:tasks.workunit.client.0.msteuth17.stderr:1+0 records out
2015-07-29T17:40:00.267 INFO:tasks.workunit.client.0.msteuth17.stderr:1024 bytes (1.0 kB) copied, 0.000243803 s, 4.2 MB/s
2015-07-29T17:40:00.268 INFO:tasks.workunit.client.0.msteuth17.stderr:+ dd if=/bin/dd of=/tmp/img1 bs=1k count=10 seek=100
2015-07-29T17:40:00.269 INFO:tasks.workunit.client.0.msteuth17.stderr:10+0 records in
2015-07-29T17:40:00.270 INFO:tasks.workunit.client.0.msteuth17.stderr:10+0 records out
2015-07-29T17:40:00.271 INFO:tasks.workunit.client.0.msteuth17.stderr:10240 bytes (10 kB) copied, 0.000283943 s, 36.1 MB/s
2015-07-29T17:40:00.272 INFO:tasks.workunit.client.0.msteuth17.stderr:+ dd if=/bin/rm of=/tmp/img1 bs=1k count=100 seek=1000
2015-07-29T17:40:00.272 INFO:tasks.workunit.client.0.msteuth17.stderr:46+1 records in
2015-07-29T17:40:00.273 INFO:tasks.workunit.client.0.msteuth17.stderr:46+1 records out
2015-07-29T17:40:00.274 INFO:tasks.workunit.client.0.msteuth17.stderr:47736 bytes (48 kB) copied, 0.000476726 s, 100 MB/s
2015-07-29T17:40:00.275 INFO:tasks.workunit.client.0.msteuth17.stderr:+ dd if=/bin/ls of=/tmp/img1 bs=1k seek=10000
2015-07-29T17:40:00.280 INFO:tasks.workunit.client.0.msteuth17.stderr:95+1 records in
2015-07-29T17:40:00.281 INFO:tasks.workunit.client.0.msteuth17.stderr:95+1 records out
2015-07-29T17:40:00.282 INFO:tasks.workunit.client.0.msteuth17.stderr:97640 bytes (98 kB) copied, 0.000740268 s, 132 MB/s
2015-07-29T17:40:00.283 INFO:tasks.workunit.client.0.msteuth17.stderr:+ dd if=/bin/ln of=/tmp/img1 bs=1k seek=100000
2015-07-29T17:40:00.284 INFO:tasks.workunit.client.0.msteuth17.stderr:46+1 records in
2015-07-29T17:40:00.285 INFO:tasks.workunit.client.0.msteuth17.stderr:46+1 records out
2015-07-29T17:40:00.286 INFO:tasks.workunit.client.0.msteuth17.stderr:47752 bytes (48 kB) copied, 0.000483085 s, 98.8 MB/s
2015-07-29T17:40:00.286 INFO:tasks.workunit.client.0.msteuth17.stderr:+ dd if=/dev/zero of=/tmp/img1 count=0 seek=150000
2015-07-29T17:40:00.287 INFO:tasks.workunit.client.0.msteuth17.stderr:0+0 records in
2015-07-29T17:40:00.288 INFO:tasks.workunit.client.0.msteuth17.stderr:0+0 records out
2015-07-29T17:40:00.289 INFO:tasks.workunit.client.0.msteuth17.stderr:0 bytes (0 B) copied, 0.000210842 s, 0.0 kB/s
2015-07-29T17:40:00.290 INFO:tasks.workunit.client.0.msteuth17.stderr:+ rbd import /tmp/img1 testimg1
2015-07-29T17:40:00.536 INFO:tasks.workunit.client.0.msteuth17.stderr:
Importing image: 5% complete...
Importing image: 10% complete...
Importing image: 16% complete...
Importing image: 21% complete...
Importing image: 27% complete...
Importing image: 32% complete...
Importing image: 38% complete...
Importing image: 43% complete...
Importing image: 49% complete...
Importing image: 54% complete...
Importing image: 60% complete...
Importing image: 65% complete...
Importing image: 70% complete...
Importing image: 76% complete...
Importing image: 81% complete...
Importing image: 87% complete...
Importing image: 92% complete...
Importing image: 98% complete...
Importing image: 100% complete...done.
2015-07-29T17:40:00.708 INFO:tasks.workunit.client.0.msteuth17.stderr:2015-07-29 21:40:00.707004 7f8e0fe110  0 -- 10.100.0.68:0/1027204 >> 10.100.0.61:6800/17283 pipe(0x7f84007090 sd=11 :40962 s=2 pgs=7 cs=1 l=1 c=0x7f84006190).injecting socket failure
2015-07-29T17:40:00.713 INFO:tasks.workunit.client.0.msteuth17.stderr:+ rbd map testimg1 --user 0
2015-07-29T17:40:00.773 INFO:tasks.workunit.client.0.msteuth17.stderr:rbd: sysfs write failed
2015-07-29T17:40:00.774 INFO:tasks.workunit.client.0.msteuth17.stderr:rbd: map failed: (13) Permission denied
2015-07-29T17:40:00.777 INFO:tasks.workunit.client.0.msteuth17.stderr:+ clean_up
2015-07-29T17:40:00.778 INFO:tasks.workunit.client.0.msteuth17.stderr:+ udevadm settle
2015-07-29T17:40:00.779 INFO:tasks.workunit.client.0.msteuth17.stderr:+ '[' -e /dev/rbd/rbd/testimg1@snap1 ']'
2015-07-29T17:40:00.780 INFO:tasks.workunit.client.0.msteuth17.stderr:+ true
2015-07-29T17:40:00.781 INFO:tasks.workunit.client.0.msteuth17.stderr:+ '[' -e /dev/rbd/rbd/testimg1 ']'
2015-07-29T17:40:00.782 INFO:tasks.workunit.client.0.msteuth17.stderr:+ udevadm settle
2015-07-29T17:40:00.789 INFO:tasks.workunit.client.0.msteuth17.stderr:+ sudo chown root /sys/bus/rbd/add /sys/bus/rbd/remove
2015-07-29T17:40:00.800 INFO:tasks.workunit.client.0.msteuth17.stderr:+ rbd ls
2015-07-29T17:40:00.801 INFO:tasks.workunit.client.0.msteuth17.stderr:+ grep testimg1
2015-07-29T17:40:00.869 INFO:tasks.workunit.client.0.msteuth17.stderr:+ rbd rm testimg1
2015-07-29T17:40:00.959 INFO:tasks.workunit.client.0.msteuth17.stderr:
Removing image: 5% complete...
Removing image: 10% complete...
Removing image: 15% complete...
Removing image: 21% complete...
Removing image: 26% complete...
Removing image: 31% complete...
Removing image: 36% complete...
Removing image: 42% complete...
Removing image: 47% complete...
Removing image: 52% complete...
Removing image: 57% complete...
Removing image: 63% complete...2015-07-29 21:40:00.953285 7f8e5ff110  0 -- 10.100.0.68:0/1027264 >> 10.100.0.61:6804/17284 pipe(0x27eb3aa0 sd=10 :45011 s=2 pgs=3 cs=1 l=1 c=0x27eb7d50).injecting socket failure
2015-07-29T17:40:01.020 INFO:tasks.workunit.client.0.msteuth17.stderr:
Removing image: 68% complete...
Removing image: 73% complete...
Removing image: 78% complete...
Removing image: 84% complete...
Removing image: 89% complete...
Removing image: 94% complete...
Removing image: 100% complete...done.
2015-07-29T17:40:01.024 INFO:tasks.workunit.client.0.msteuth17.stderr:+ sudo rm -f /tmp/img1 /tmp/img1.small /tmp/img1.snap1 /tmp/img1.export /tmp/img1.trunc
2015-07-29T17:40:01.051 INFO:tasks.workunit:Stopping ['rbd/kernel.sh'] on client.0...
2015-07-29T17:40:01.052 INFO:teuthology.orchestra.run.msteuth17:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/workunit.client.0'
2015-07-29T17:40:01.135 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/ceph-qa-suite_no_valgrind/tasks/workunit.py", line 361, in _run_tests
    label="workunit test {workunit}".format(workunit=workunit)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 137, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed (workunit test rbd/kernel.sh) on msteuth17 with status 13: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=workunit-fixes-for-aarch64 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/rbd/kernel.sh'
2015-07-29T17:40:01.137 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 53, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 41, in run_one_task
    return fn(**kwargs)
  File "/home/teuthworker/src/ceph-qa-suite_no_valgrind/tasks/workunit.py", line 115, in task
    config.get('subdir'), timeout=timeout)
  File "/home/teuthworker/src/ceph-qa-suite_no_valgrind/tasks/workunit.py", line 252, in _spawn_on_all_clients
    timeout=timeout)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/ceph-qa-suite_no_valgrind/tasks/workunit.py", line 361, in _run_tests
    label="workunit test {workunit}".format(workunit=workunit)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 137, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait
    label=self.label)
CommandFailedError: Command failed (workunit test rbd/kernel.sh) on msteuth17 with status 13: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=workunit-fixes-for-aarch64 TESTDIR="/home/ubuntu/cephtest" CEPH_ID="0" PATH=$PATH:/usr/sbin adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/workunit.client.0/rbd/kernel.sh'


Files

Actions #1

Updated by Loïc Dachary about 8 years ago

  • Project changed from Ceph to CephFS
  • Tags changed from aarch64 to aarch64 arm64
Actions #2

Updated by Greg Farnum about 8 years ago

  • Project changed from CephFS to rbd
Actions #3

Updated by Jason Dillaman over 7 years ago

  • Project changed from rbd to Linux kernel client
  • Priority changed from Normal to Low
Actions

Also available in: Atom PDF