Bug #17772
closed
journal on ramdisk fails ceph-osd --mkfs
Added by Jayashree Ajay Candadai over 7 years ago.
Updated almost 7 years ago.
Description
Journal on ramdisk fails ceph-osd --mkfs - Using a file outside of the ramdisk works, it has to do with the ramdisk which is not handled properly by ceph-osd --mkfs. Enclosed are the log and commands to reproduce this issue. This happens both with Kraken and Jewel release.
1) OSD_Error.log: -- This is before applying the patch in main.py trying to use tmpfs as journal space (http://tracker.ceph.com/issues/17662)
2) OSD_afterPatch.log -- After applying the patch
3) OSD_StatusCommands_AfterPatch.txt - shows the journal space is not created.
4) NewCommands.txt -- Is output after running
ceph-disk zap /dev/sdc /dev/sdd /dev/sde /dev/sdf
ceph-disk list
rm -f /tmp/ramdisk/journal-sdc.journal
ceph-disk prepare /dev/sdc /tmp/ramdisk/journal-sdc.journal
sleep 10
ceph-disk list
5) NewError_OSD.log after running
ceph-disk zap /dev/sdc /dev/sdd /dev/sde /dev/sdf
ceph-disk list
rm -f /tmp/ramdisk/journal-sdc.journal
ceph-disk --verbose prepare /dev/sdc /tmp/ramdisk/journal-sdc.journal
sleep 10
ceph-disk list
ceph-disk --verbose activate /dev/sdc1
6) Einval.log has the verbose output which shows that journal on ramdisk fails ceph-osd --mkfs.
ceph-disk zap /dev/sdc /dev/sdd /dev/sde /dev/sdf
ceph-disk list
rm -f /tmp/foo
ceph-disk --verbose prepare /dev/sdc /tmp/foo
sleep 10
ceph-disk list
ceph-disk --verbose activate /dev/sdc1
Files
- Priority changed from Immediate to Normal
- Severity changed from 1 - critical to 3 - minor
I may be missing something, but can't you just put a filesystem on the ramdisk and create a file?
Samuel Just wrote:
I may be missing something, but can't you just put a filesystem on the ramdisk and create a file?
One option is to have additional disk and use that as the journal space for all the OSD's which is working by the way. But, with it we don't get very good read performance. So, what we are trying to do is to use the ramdisk as tmpfs file system and use that space as the journal space. Please follow the other bug which I had created earlier for more information
- tempfs as CEPH journal
mkdir /tmp/ramdisk
chmod 777 /tmp/ramdisk
- Create a mount point and reserve small amount of RAM as filesystem.
sudo mount -t tmpfs -o size=4G tmpfs /tmp/ramdisk/
Suppose If I create the journal files
- Create the journal files for each OSD disk
cd /tmp/ramdisk
touch journal-sdc.journal
touch journal-sdd.journal
touch journal-sde.journal
sudo chmod 664 journal-sd*
With this I should be able to use the journal space
prepare and activate present in ceph::osd class. '/dev/sdb' is the file system - mount point.
ceph::osd {
'/dev/sdc':
journal => '/tmp/ramdisk/journal-sdc.journal';
'/dev/sdd':
journal => '/tmp/ramdisk/journal-sdd.journal';
'/dev/sde':
journal => '/tmp/ramdisk/journal-sde.journal';
}
Which will give faster performance.
Jayashree Ajay Candadai wrote:
Samuel Just wrote:
I may be missing something, but can't you just put a filesystem on the ramdisk and create a file?
One option is to have additional disk and use that as the journal space for all the OSD's which is working by the way. But, with it we don't get very good read performance. So, what we are trying to do is to use the ramdisk as tmpfs file system and use that space as the journal space. Please follow the other bug which I had created earlier for more information
~~~
- tempfs as CEPH journal
mkdir /tmp/ramdisk
chmod 777 /tmp/ramdisk
- Create a mount point and reserve small amount of RAM as filesystem.
sudo mount -t tmpfs -o size=4G tmpfs /tmp/ramdisk/
- Create the journal files for each OSD disk
cd /tmp/ramdisk
touch journal-sdc.journal
touch journal-sdd.journal
touch journal-sde.journal
sudo chmod 664 journal-sd*
~~~
With this I should be able to use the journal space
prepare and activate present in ceph::osd class. '/dev/sdb' is the file system - mount point.
ceph::osd {
'/dev/sdc':
journal => '/tmp/ramdisk/journal-sdc.journal';
'/dev/sdd':
journal => '/tmp/ramdisk/journal-sdd.journal';
'/dev/sde':
journal => '/tmp/ramdisk/journal-sde.journal';
}
Which will give faster performance.
- Status changed from New to Closed
I think this is a feature. People who want to test for speed purposes can hack around it. :)
Also available in: Atom
PDF