Bug #11255
nfs: mount failures on ceph-backed NFS share
0%
Description
http://pulpito.ceph.com/teuthology-2015-03-25_23:10:02-knfs-master-testing-basic-multi/822373/
2015-03-26T20:49:50.194 INFO:teuthology.run_tasks:Running task knfsd... 2015-03-26T20:49:50.194 INFO:teuthology.task.knfsd:Exporting nfs server... 2015-03-26T20:49:50.194 DEBUG:teuthology.task.knfsd:Client client.0 config is {'options': ['rw', 'no_root_squash', 'async']} 2015-03-26T20:49:50.194 INFO:teuthology.task.knfsd:Exporting knfsd client.0 at ubuntu@plana77.front.sepia.ceph.com *:/home/ubuntu/cephtest/mnt.0 (rw,no_root_squash,async)... 2015-03-26T20:49:50.195 INFO:teuthology.orchestra.run.plana77:Running: 'sudo chmod 777 /home/ubuntu/cephtest/mnt.0' 2015-03-26T20:49:50.298 INFO:teuthology.task.knfsd:remote run ['sudo', 'exportfs', '-o', 'fsid=1230,rw,no_root_squash,async', '10.214.132.15:/home/ubuntu/cephtest/mnt.0', '10.214.132.18:/home/ubuntu/cephtest/mnt.0', '10.214.132.14:/home/ubuntu/cephtest/mnt.0', '10.214.133.39:/home/ubuntu/cephtest/mnt.0'] 2015-03-26T20:49:50.298 INFO:teuthology.orchestra.run.plana77:Running: 'sudo exportfs -o fsid=1230,rw,no_root_squash,async 10.214.132.15:/home/ubuntu/cephtest/mnt.0 10.214.132.18:/home/ubuntu/cephtest/mnt.0 10.214.132.14:/home/ubuntu/cephtest/mnt.0 10.214.133.39:/home/ubuntu/cephtest/mnt.0' 2015-03-26T20:49:50.448 INFO:teuthology.run_tasks:Running task nfs... 2015-03-26T20:49:50.448 INFO:teuthology.task.nfs:Mounting nfs clients... 2015-03-26T20:49:50.448 DEBUG:teuthology.task.nfs:Client client.1 config is {'options': ['rw', 'hard', 'intr', 'nfsvers=4'], 'server': 'client.0'} 2015-03-26T20:49:50.449 INFO:teuthology.task.nfs:Mounting client.1 from client.0 2015-03-26T20:49:50.449 DEBUG:teuthology.task.nfs:mount -o rw,hard,intr,nfsvers=4 plana77.front.sepia.ceph.com:/home/ubuntu/cephtest/mnt.0 /home/ubuntu/cephtest/mnt.1 2015-03-26T20:49:50.449 INFO:teuthology.orchestra.run.plana64:Running: 'mkdir -- /home/ubuntu/cephtest/mnt.1' 2015-03-26T20:49:50.541 INFO:teuthology.orchestra.run.plana64:Running: 'sudo mount -o rw,hard,intr,nfsvers=4 plana77.front.sepia.ceph.com:/home/ubuntu/cephtest/mnt.0 /home/ubuntu/cephtest/mnt.1' 2015-03-26T20:49:51.360 INFO:teuthology.orchestra.run.plana64.stderr:mount.nfs: access denied by server while mounting plana77.front.sepia.ceph.com:/home/ubuntu/cephtest/mnt.0 2015-03-26T20:49:51.363 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/teuthology_master/teuthology/run_tasks.py", line 55, in run_tasks manager.__enter__() File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/home/teuthworker/src/teuthology_master/teuthology/task/nfs.py", line 106, in task mnt File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 137, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run r.wait() File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 114, in wait label=self.label) CommandFailedError: Command failed on plana64 with status 32: 'sudo mount -o rw,hard,intr,nfsvers=4 plana77.front.sepia.ceph.com:/home/ubuntu/cephtest/mnt.0 /home/ubuntu/cephtest/mnt.1'
I couldn't find anything interesting in the logs, though. :(
History
#1 Updated by Greg Farnum almost 9 years ago
#3 Updated by Greg Farnum almost 9 years ago
Looking through the man pages, I see
Exporting to IP networks or DNS and NIS domains does not enable clients from these groups to access NFS immediately. Rather, these sorts of exports are hints to rpc.mountd(8) to grant any mount requests from these clients.
I'm not quite sure if that means we maybe need to be doing some kind of blocking until the mountd is happy or something?
#4 Updated by Greg Farnum almost 9 years ago
- Subject changed from nfs: access denied to nfs: mount failures on ceph-backed NFS share
http://pulpito.ceph.com/teuthology-2015-04-29_23:10:02-knfs-next-testing-basic-multi/869428/
I also realized recently that (d'oh) error number 32 here is not actually Access Denied but just what the mount command returns to indicate "mount failure". Which again makes me think that the clients and servers are using different settings of some description...
#5 Updated by John Spray over 8 years ago
- Regression set to No
#6 Updated by Zheng Yan over 8 years ago
maybe https://github.com/ceph/teuthology/pull/342 can fix this
#7 Updated by Greg Farnum over 8 years ago
Did you mean to reference https://github.com/ceph/teuthology/pull/554, Zheng?
#8 Updated by Greg Farnum over 8 years ago
- Status changed from New to Fix Under Review
#10 Updated by John Spray almost 8 years ago
- Status changed from Fix Under Review to Resolved
The linked PR merged and this is not appearing currently in master.