Bug #5367
closed
multiclient tests: kernel mount gets EPERM
Added by Sage Weil almost 11 years ago.
Updated almost 11 years ago.
Description
2013-06-15T11:53:25.609 INFO:teuthology.orchestra.run.out:access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
2013-06-15T11:53:25.609 INFO:teuthology.orchestra.run.out:------ --------- ---------- --------- -------- -------- -------- -------- ----
2013-06-15T11:53:25.611 INFO:teuthology.orchestra.run.out:ior ERROR: open64() failed, errno 13, Permission denied (aiori-POSIX.c:158)
2013-06-15T11:53:25.611 INFO:teuthology.orchestra.run.out:ior ERROR: open64() failed, errno 13, Permission denied (aiori-POSIX.c:158)
2013-06-15T11:53:25.611 INFO:teuthology.orchestra.run.out:ior ERROR: open64() failed, errno 13, Permission denied (aiori-POSIX.c:158)
ubuntu@teuthology:/a/sage-2013-06-15_09:42:09-foo-next-testing-basic/36504$ cat orig.config.yaml
kernel:
kdb: true
sha1: f28d6bf9ef4d3f8dd2e669fd21be827663f215e2
machine_type: plana
nuke-on-error: true
overrides:
ceph:
conf:
mon:
debug mon: 20
debug ms: 20
debug paxos: 20
osd:
osd op thread timeout: 60
fs: btrfs
log-whitelist:
- slow request
sha1: 9253cd8174ca7690414923259b01c0a484323f19
install:
ceph:
sha1: 9253cd8174ca7690414923259b01c0a484323f19
s3tests:
branch: next
workunit:
sha1: 9253cd8174ca7690414923259b01c0a484323f19
roles:
- - mon.a
- mon.b
- mon.c
- mds.a
- osd.0
- osd.1
- - client.2
- - client.1
- - client.0
tasks:
- chef: null
- clock.check: null
- install: null
- ceph: null
- kclient: null
- pexec:
clients:
- cd $TESTDIR
- wget http://ceph.com/qa/ior.tbz2
- tar xvfj ior.tbz2
- cd ior
- ./configure
- make
- make install DESTDIR=$TESTDIR/binary/
- cd $TESTDIR/
- rm ior.tbz2
- rm -r ior
- ln -s $TESTDIR/mnt.* $TESTDIR/gmnt
- ssh_keys: null
- mpi:
exec: $TESTDIR/binary/usr/local/bin/ior -e -w -r -W -b 10m -a POSIX -o $TESTDIR/gmnt/ior.testfile
- pexec:
all:
- rm -f $TESTDIR/gmnt/ior.testfile
- rm -f $TESTDIR/gmnt
kclient and MDS never return -EACCES. was ior executed with root privilege?
saw similar error on ceph-fuse:
2013-07-17T12:35:23.001 INFO:teuthology.orchestra.run.out:access bw(MiB/s) block(KiB) xfer(KiB) open(s) wr/rd(s) close(s) total(s) iter
2013-07-17T12:35:23.001 INFO:teuthology.orchestra.run.out:------ --------- ---------- --------- -------- -------- -------- -------- ----
2013-07-17T12:35:23.340 INFO:teuthology.orchestra.run.out:ior ERROR: open64() failed, errno 13, Permission denied (aiori-POSIX.c:158)
2013-07-17T12:35:23.340 INFO:teuthology.orchestra.run.err:application called MPI_Abort(MPI_COMM_WORLD, -1) - process 2
2013-07-17T12:35:23.843 INFO:teuthology.orchestra.run.err:Fatal error in PMPI_Reduce: Other MPI error, error stack:
2013-07-17T12:35:23.843 INFO:teuthology.orchestra.run.err:PMPI_Reduce(1270)...............: MPI_Reduce(sbuf=0x7fff84fe1ffc, rbuf=0x7fff84fe2008, count=1, MPI_INT, MPI_SUM, root=0, comm=0x84000000) failed
2013-07-17T12:35:23.843 INFO:teuthology.orchestra.run.err:MPIR_Reduce_impl(1087)..........:
2013-07-17T12:35:23.843 INFO:teuthology.orchestra.run.err:MPIR_Reduce_intra(895)..........:
2013-07-17T12:35:23.843 INFO:teuthology.orchestra.run.err:MPIR_Reduce_binomial(144).......:
2013-07-17T12:35:23.843 INFO:teuthology.orchestra.run.err:MPIDI_CH3U_Recvq_FDU_or_AEP(380): Communication error with rank 2
job was
ubuntu@teuthology:/a/teuthology-2013-07-17_01:01:03-fs-next-testing-basic/70462$ cat orig.config.yaml
kernel:
kdb: true
sha1: 77c8bf2f972a9d6ff446c49a41678bf931bbee44
machine_type: plana
nuke-on-error: true
overrides:
admin_socket:
branch: next
ceph:
conf:
mon:
debug mon: 20
debug ms: 20
debug paxos: 20
osd:
osd op thread timeout: 60
fs: btrfs
log-whitelist:
- slow request
sha1: 884fa2fcb6d707b23317bab1da909586ddc27608
ceph-deploy:
conf:
client:
debug monc: 20
debug ms: 1
debug objecter: 20
debug rados: 20
log file: /var/log/ceph/ceph-..log
mon:
debug mon: 20
debug ms: 20
debug paxos: 20
install:
ceph:
sha1: 884fa2fcb6d707b23317bab1da909586ddc27608
s3tests:
branch: next
workunit:
sha1: 884fa2fcb6d707b23317bab1da909586ddc27608
roles:
- - mon.a
- mon.b
- mon.c
- mds.a
- osd.0
- osd.1
- osd.2
- - client.2
- - client.1
- - client.0
tasks:
- chef: null
- clock.check: null
- install: null
- ceph: null
- ceph-fuse: null
- pexec:
clients:
- cd $TESTDIR
- wget http://ceph.com/qa/ior.tbz2
- tar xvfj ior.tbz2
- cd ior
- ./configure
- make
- make install DESTDIR=$TESTDIR/binary/
- cd $TESTDIR/
- rm ior.tbz2
- rm -r ior
- ln -s $TESTDIR/mnt.* $TESTDIR/gmnt
- ssh_keys: null
- mpi:
exec: $TESTDIR/binary/usr/local/bin/ior -e -w -r -W -b 10m -a POSIX -o $TESTDIR/gmnt/ior.testfile
- pexec:
all:
- rm -f $TESTDIR/gmnt/ior.testfile
- rm -f $TESTDIR/gmnt
- rm -rf $TESTDIR/binary
Zheng Yan wrote:
kclient and MDS never return -EACCES. was ior executed with root privilege?
not root...
can't reproduce locally. Is it possible your clients have different 'username -> uid' mapping?
ah, a few do! the ones this test failed on don't, but maybe they were reimaged since then.
- Status changed from New to Resolved
We can reopen if we see this again, but the tests are now checking (and chef is setting) the uid for the ubuntu (test) user.
Also available in: Atom
PDF