Bug #12674
closedSemi-reproducible crash of ceph-fuse
0%
Description
we are running ceph version 0.94.2 with a cephfs mounted using ceph-fuse on
Ubuntu 14.04 LTS. I think we have found a bug that lets us semi-reprodicibly
crash the ceph-fuse process.
On the file system we have many files that contain non-ASCII characters in
various encodings (ISO8859, UTF-8) we are currently normalizing by renaming
them. The crash seems to be triggered by those rename operations. However,
it is not as simple as just creating a file with some specific name and then
renaming it.
A log file of the crash has been attached.
Files
Updated by Zheng Yan over 8 years ago
the log seems like memory allocation failure. how many memory does your machine have?
Updated by Jörg Henne over 8 years ago
Zheng Yan wrote:
the log seems like memory allocation failure. how many memory does your machine have?
The VM has 4GB of memory with usually more than .5GB free. It has an additional 4GB if swap which has never been touched during the VM's uptime.
Updated by Jörg Henne over 8 years ago
- We are no longer 100% certain that the crashes relate to renames. The actual reason might also be other accesses.
- The system in question just mounts cephfs via fuse and re-exports it via SMABA. Almost all accesses to the file system therefore originate from smbd.
Updated by Jörg Henne over 8 years ago
- File ceph-client.admin.log.gz ceph-client.admin.log.gz added
Status changed from New to Need More Info
What kind of info are you looking for? As I've just hit another situation where I can reproducibly crash ceph-fuse, I've attached a new log file of the crash. The crash happens upon reading several files from a directory tree on cephfs. I've updated to 0.94.3 in the meantime.
Updated by Zheng Yan over 8 years ago
- Status changed from Need More Info to 12
RP https://github.com/ceph/ceph/pull/4753 may fix this issue. could you try compiling ceph-fuse from the newest ceph source (https://github.com/ceph/ceph.git master)
Updated by Jörg Henne over 8 years ago
Please also note that this crash can be triggered with a simple operation of the style
find . -name \*.txt | xargs cat > /dev/null
The files are nested in directories 2-3 levels deep and the names are extremely simple this time. So this problem seems to be unrelated to the original observation about fancy file names and renaming.
Updated by Zheng Yan over 8 years ago
It's likely been fixed by pull request https://github.com/ceph/ceph/pull/4753 (it's large change, we haven't back-ported it) . please try compiling ceph-fuse from the newest ceph source (https://github.com/ceph/ceph.git master)
Updated by Jörg Henne over 8 years ago
I can confirm that the issue is gone with the latest ceph source from Git. Sorry for taking so long to build and test it. Thanks a ton!