Ceph » CephFS

i maybe found the problem.
before you mkcephfs,you should ensure the dir(/var/lib/ceph/osd/ceph-0) empty.
once i wrote a script:
#mount /dev/sd6 /var/lib/ceph/osd/ceph-2
#mount /dev/sd6 /var/lib/ceph/osd/ceph-3

you can see that i mount the one same device to two paths.

then everytime i execute the order, the path /var/lib/ceph/osd/ceph-3 will not be empty.

got the error:

2014-08-22 17:17:02.869346 7f99e9898780 -1 provided osd id 3 != superblock's -1
2014-08-22 17:17:02.870036 7f99e9898780 -1 ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-3: (22) Invalid argument
what we need to do is:

1. mount /dev/sda6 /var/lib/ceph/osd/ceph-2
2. mount /dev/sda7 /var/lib/ceph/osd/ceph-3

Joe Buck wrote:

While running a teuthology test, mkcephfs failed with this error:
INFO:teuthology.task.ceph:Running mkfs on osd nodes...
INFO:teuthology.task.ceph:{}
INFO:teuthology.task.ceph:0
INFO:teuthology.orchestra.run.err:2013-03-25 11:57:49.200303 7f3ce27cc780 1 filestore(/var/lib/ceph/osd/ceph-0) limited size xattrs - filestore_xattr_use_omap enabled
INFO:teuthology.orchestra.run.err:2013-03-25 11:57:49.967481 7f3ce27cc780 -1 provided osd id 0 != superblock's -1
INFO:teuthology.orchestra.run.err:2013-03-25 11:57:49.968781 7f3ce27cc780 -1 ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0: (22) Invalid argument
ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
File "/mnt/ssd/git/teuthology/teuthology/contextutil.py", line 25, in nested
vars.append(enter())
File "/usr/lib/python2.7/contextlib.py", line 17, in enter
return self.gen.next()
File "/mnt/ssd/git/teuthology/teuthology/task/ceph.py", line 607, in cluster
'--monmap', '{tdir}/monmap'.format(tdir=testdir),
File "/mnt/ssd/git/teuthology/teuthology/orchestra/remote.py", line 42, in run
r = self._runner(client=self.ssh, **kwargs)
File "/mnt/ssd/git/teuthology/teuthology/orchestra/run.py", line 266, in run
r.exitstatus = check_status(r.exitstatus)
File "/mnt/ssd/git/teuthology/teuthology/orchestra/run.py", line 262, in _check_status
raise CommandFailedError(command=r.command, exitstatus=status, node=host)
CommandFailedError: Command failed on 10.214.133.20 with status 1: 'MALLOC_CHECK=3 /tmp/cephtest/buck@bender-2013-03-25_11-56-36/en

I'll upload the YAML file I was using and the entire output from the run in a minute.