Bug #36094
Updated by Patrick Donnelly over 5 years ago
Hi I have met mds error "FAILED assert(omap_num_objs <= MAX_OBJECTS)" in my product env. Ceph version mimic 13.2.1 client CentOS7.5 mds log: <pre> 2018-09-20 08:53:17.610 7f8571755700 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.1/rpm/el7/BUILD/ceph-13.2.1/src/mds/OpenFileTable.cc: In function 'void OpenFileTable::commit(MDSInternalContextBase*, uint64_t, int)' thread 7f8571755700 time 2018-09-20 08:53:17.609555 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/13.2.1/rpm/el7/BUILD/ceph-13.2.1/src/mds/OpenFileTable.cc: 467: FAILED assert(omap_num_objs <= MAX_OBJECTS) ceph version 13.2.1 (5533ecdc0fda920179d7ad84e0aa65a127b20d77) mimic (stable) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xff) [0x7f857d828e1f] 2: (()+0x284fe7) [0x7f857d828fe7] 3: (OpenFileTable::commit(MDSInternalContextBase*, unsigned long, int)+0x1b13) [0x55e77b081533] 4: (MDLog::trim(int)+0x5a6) [0x55e77b06ede6] 5: (MDSRankDispatcher::tick()+0x23b) [0x55e77ae0cfcb] 6: (FunctionContext::finish(int)+0x2c) [0x55e77adfd70c] 7: (Context::complete(int)+0x9) [0x55e77adfb189] 8: (SafeTimer::timer_thread()+0x18b) [0x7f857d82573b] 9: (SafeTimerThread::entry()+0xd) [0x7f857d826cfd] 10: (()+0x7e25) [0x7f857b453e25] 11: (clone()+0x6d) [0x7f857a533bad] NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --- begin dump of recent events --- -10000> 2018-09-20 08:53:17.238 7f85741e3700 1 -- 10.10.216.20:6800/4206406544 <== client.2772295 10.10.216.2:0/258380382 1423254100 ==== client_request(client.2772295:14342717695 rmxattr #0x100008666eb security.ima 2018-09-20 08:53:17.238752 caller_uid=0, caller_gid=0{}) v2 ==== 178+0+0 (3162234629 0 0) 0x55f1c9fc5b00 con 0x55e7931db800 -9999> 2018-09-20 08:53:17.238 7f85741e3700 4 mds.0.server handle_client_request client_request(client.2772295:14342717695 rmxattr #0x100008666eb security.ima 2018-09-20 08:53:17.238752 caller_uid=0, caller_gid=0{}) v2 -9998> 2018-09-20 08:53:17.238 7f856cf4c700 5 mds.0.log _submit_thread 2610936708022~994 : EOpen [metablob 0x10000866586, 1 dirs], 1 open files -9997> 2018-09-20 08:53:17.238 7f85741e3700 1 -- 10.10.216.20:6800/4206406544 --> 10.10.216.2:0/258380382 -- client_reply(???:14342717695 = -61 (61) No data available safe) v1 -- 0x55ef49ea4b00 con 0 -9996> 2018-09-20 08:53:17.238 7f85761e7700 5 -- 10.10.216.20:6800/4206406544 >> 10.10.216.2:0/258380382 conn(0x55e7931db800 :6800 s=STATE_OPEN_MESSAGE_READ_FOOTER_AND_DISPATCH pgs=14770 cs=1 l=0). rx client.2772295 seq 1423254101 0x55f1967b4900 client_request(client.2772295:14342717696 rmxattr #0x100008666ec security.ima 2018-09-20 08:53:17.238752 caller_uid=0, caller_gid=0{}) v2 -9995> 2018-09-20 08:53:17.238 7f85741e3700 1 -- 10.10.216.20:6800/4206406544 <== client.2772295 10.10.216.2:0/258380382 1423254101 ==== client_request(client.2772295:14342717696 rmxattr #0x100008666ec security.ima 2018-09-20 08:53:17.238752 caller_uid=0, caller_gid=0{}) v2 ==== 178+0+0 (2702756268 0 0) 0x55f1967b4900 con 0x55e7931db800 -9994> 2018-09-20 08:53:17.238 7f85741e3700 4 mds.0.server handle_client_request client_request(client.2772295:14342717696 rmxattr #0x100008666ec security.ima 2018-09-20 08:53:17.238752 caller_uid=0, caller_gid=0{}) v2 -9993> 2018-09-20 08:53:17.238 7f856cf4c700 5 mds.0.log _submit_thread 2610936709036~993 : EOpen [metablob 0x10000866586, 1 dirs], 1 open files -9992> 2018-09-20 08:53:17.238 7f85741e3700 1 -- 10.10.216.20:6800/4206406544 --> 10.10.216.2:0/258380382 -- client_reply(???:14342717696 = -61 (61) No data available safe) v1 -- 0x55ea6c5a6840 con 0 </pre>