Bug #11615
openMemory Spike and page fault in slab allocator
0%
Description
Attached is a backtrace and a memory graph from a kernel panic related to multiple simultaneous rsyncs of files into a cephfs volume.
Server info: Ceph 0.94.1 on CentOS 7.1
Client info: Gentoo, kernel 3.19.5
Files
Updated by Ilya Dryomov almost 9 years ago
Sorry I didn't see this earlier.
The panic is not in ceph_kvmalloc() but in the guts of slab memory allocator. It's hard to make conclusions from the backtrace alone, the best I can tell is it died from a page fault in mm/slub.c
244 static inline void *get_freepointer(struct kmem_cache *s, void *object) 245 { 246 return *(void **)(object + s->offset); <<-- 247 }
on a invalid *object - 0x6232ae8. This value came from
2416 preempt_enable(); 2417 2418 object = c->freelist; <<-- 2419 page = c->page; 2420 if (unlikely(!object || !node_match(page, node))) {
which suggests a busted struct kmem_cache_cpu. How exactly it got busted is an open question and it probably has to do with the sudden memory usage spike shown on your graph. More questions then answers for sure, if you can reproduce this on a latest kernel we can take a deeper look - a lot of changes went into cephfs caps-related bits since 3.19.5.
Updated by Ilya Dryomov almost 9 years ago
- Subject changed from Memory Spike and Panic in ceph_kvmalloc to Memory Spike and page fault in slab allocator
- Status changed from New to Need More Info