Bug #47661
closedCannot allocate memory appears when using io_uring osd
0%
Description
Hello everyone,
When I use ubuntu 20.04, try to use bluestore, and experiment with nvme io_uring, the problem of Cannot allocate memory appears.
Kernel version:
root@test:~# uname -a
Linux test 5.4.0-48-generic #52-Ubuntu SMP Thu Sep 10 10:58:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
ceph version 15.2.5
My steps are as follows:
1. clone ceph, Modify CMakeLists.txt
include(CMakeDependentOption)
CMAKE_DEPENDENT_OPTION(WITH_LIBURING "Build with liburing library support" ON
"WITH_BLUESTORE;HAVE_LIBAIO" OFF)
set(HAVE_LIBURING ${WITH_LIBURING})
2.Execute ./make-debs.sh
3.Use ceph-ansible 5.0 branch to deploy an osd ceph
4.Stop the ceph process, modify /etc/ceph.conf, bluestore_ioring = true
5.Restart the ceph service, and osd Cannot allocate memory appears.
Is my step wrong?
osd log:
2020-09-27T07:23:37.355+0000 7fb8fb328dc0 0 set uid:gid to 64045:64045 (ceph:ceph)
2020-09-27T07:23:37.355+0000 7fb8fb328dc0 0 ceph version 15.2.5-1-g4b2918df4b (4b2918df4b5f89cae6d6377ff9bdc4d95f87b9e9) octopus (stable), process ceph-osd, pid 731734
2020-09-27T07:23:37.355+0000 7fb8fb328dc0 0 pidfile_write: ignore empty --pid-file
2020-09-27T07:23:37.355+0000 7fb8fb328dc0 1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2020-09-27T07:23:37.387+0000 7fb8fb328dc0 1 bdev(0x563828664000 /var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2020-09-27T07:23:37.387+0000 7fb8fb328dc0 -1 bdev(0x563828664000 /var/lib/ceph/osd/ceph-0/block) _aio_start io_setup(2) failed: (12) Cannot allocate memory
2020-09-27T07:23:37.387+0000 7fb8fb328dc0 0 starting osd.0 osd_data /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal
2020-09-27T07:23:37.399+0000 7fb8fb328dc0 0 load: jerasure load: lrc load: isa
2020-09-27T07:23:37.399+0000 7fb8fb328dc0 1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 1 bdev(0x563828664000 /var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 -1 bdev(0x563828664000 /var/lib/ceph/osd/ceph-0/block) _aio_start io_setup(2) failed: (12) Cannot allocate memory
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 0 osd.0:0.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 0 osd.0:1.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 0 osd.0:2.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 0 osd.0:3.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 0 osd.0:4.OSDShard using op scheduler ClassedOpQueueScheduler(queue=WeightedPriorityQueue, cutoff=196)
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 1 bluestore(/var/lib/ceph/osd/ceph-0) _mount path /var/lib/ceph/osd/ceph-0
2020-09-27T07:23:37.431+0000 7fb8fb328dc0 1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2020-09-27T07:23:37.459+0000 7fb8fb328dc0 1 bdev(0x563828664000 /var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2020-09-27T07:23:37.459+0000 7fb8fb328dc0 -1 bdev(0x563828664000 /var/lib/ceph/osd/ceph-0/block) _aio_start io_setup(2) failed: (12) Cannot allocate memory
2020-09-27T07:23:37.459+0000 7fb8fb328dc0 -1 osd.0 0 OSD:init: unable to mount object store
2020-09-27T07:23:37.459+0000 7fb8fb328dc0 -1 ** ERROR: osd init failed: (12) Cannot allocate memory
Files