Project

General

Profile

Actions
Copy link

Bug #51023

closed

mds: tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0xf3)

Added by Xiubo Li almost 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
High
Assignee:
Xiubo Li
Category:
Correctness/Safety
Target version:
Ceph - v17.0.0
% Done:

0%

Source:
Development
Tags:
Backport:
pacific,octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
fs
Component(FS):
MDS
Labels (FS):
crash
Pull request ID:
41667
Crash signature (v1):
Crash signature (v2):

Description

From https://pulpito.ceph.com/yuriw-2021-05-27_19:31:33-kcephfs-wip-yuri3-testing-2021-05-27-0818-nautilus-distro-basic-smithi/6138429/:

2021-05-27T23:14:13.379 INFO:tasks.ceph.mds.c.smithi104.stderr:tcmalloc: large alloc 94317937795072 bytes == (nil) @  0x7f75fe0ea35f 0x7f75fe109a56 0x55c817ca10f9 0x55c817ca127b 0x55c817ca1340 0x55c817ca141e 0x55c817e90230 0x55c817e904cf 0x55c817d7af8d 0x55c817f33ea4 0x55c817f340ff 0x55c817f34310 0x7f75fe7b151f 0x7f75fc5dbea5 0x7f75fb2889fd
2021-05-27T23:14:13.379 INFO:tasks.ceph.mds.c.smithi104.stderr:terminate called after throwing an instance of 'std::bad_alloc'
2021-05-27T23:14:13.379 INFO:tasks.ceph.mds.c.smithi104.stderr:  what():  std::bad_alloc
2021-05-27T23:14:13.380 INFO:tasks.ceph.mds.c.smithi104.stderr:*** Caught signal (Aborted) **
2021-05-27T23:14:13.380 INFO:tasks.ceph.mds.c.smithi104.stderr: in thread 7f75eec84700 thread_name:fn_anonymous
2021-05-27T23:14:13.381 INFO:tasks.ceph.mds.c.smithi104.stderr: ceph version 14.2.21-281-gb7a4754d941 (b7a4754d941f016678ca17d548fafb10e7444735) nautilus (stable)
2021-05-27T23:14:13.381 INFO:tasks.ceph.mds.c.smithi104.stderr: 1: (()+0xf630) [0x7f75fc5e3630]
2021-05-27T23:14:13.381 INFO:tasks.ceph.mds.c.smithi104.stderr: 2: (gsignal()+0x37) [0x7f75fb1c03d7]
2021-05-27T23:14:13.382 INFO:tasks.ceph.mds.c.smithi104.stderr: 3: (abort()+0x148) [0x7f75fb1c1ac8]
2021-05-27T23:14:13.382 INFO:tasks.ceph.mds.c.smithi104.stderr: 4: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f75fbad0a95]
2021-05-27T23:14:13.382 INFO:tasks.ceph.mds.c.smithi104.stderr: 5: (()+0x5ea06) [0x7f75fbacea06]
2021-05-27T23:14:13.382 INFO:tasks.ceph.mds.c.smithi104.stderr: 6: (()+0x5ea33) [0x7f75fbacea33]
2021-05-27T23:14:13.383 INFO:tasks.ceph.mds.c.smithi104.stderr: 7: (()+0x5ec53) [0x7f75fbacec53]
2021-05-27T23:14:13.383 INFO:tasks.ceph.mds.c.smithi104.stderr: 8: (()+0x1772b) [0x7f75fe0eb72b]
2021-05-27T23:14:13.383 INFO:tasks.ceph.mds.c.smithi104.stderr: 9: (tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0xf3) [0x7f75fe109a03]
2021-05-27T23:14:13.383 INFO:tasks.ceph.mds.c.smithi104.stderr: 10: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x55c817ca10f9]
2021-05-27T23:14:13.383 INFO:tasks.ceph.mds.c.smithi104.stderr: 11: (std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long)+0x1b) [0x55c817ca127b]
2021-05-27T23:14:13.384 INFO:tasks.ceph.mds.c.smithi104.stderr: 12: (std::string::reserve(unsigned long)+0x40) [0x55c817ca1340]
2021-05-27T23:14:13.384 INFO:tasks.ceph.mds.c.smithi104.stderr: 13: (std::string::append(char const*, unsigned long)+0x5e) [0x55c817ca141e]
2021-05-27T23:14:13.384 INFO:tasks.ceph.mds.c.smithi104.stderr: 14: (CDentry::make_path(filepath&, bool) const+0x90) [0x55c817e90230]
2021-05-27T23:14:13.384 INFO:tasks.ceph.mds.c.smithi104.stderr: 15: (operator<<(std::ostream&, CDentry const&)+0x6f) [0x55c817e904cf]
2021-05-27T23:14:13.385 INFO:tasks.ceph.mds.c.smithi104.stderr: 16: (MDCache::_create_system_file_finish(boost::intrusive_ptr<MutationImpl>&, CDentry*, unsigned long, MDSContext*)+0xbd) [0x55c817d7af8d]
2021-05-27T23:14:13.385 INFO:tasks.ceph.mds.c.smithi104.stderr: 17: (MDSContext::complete(int)+0x74) [0x55c817f33ea4]
2021-05-27T23:14:13.385 INFO:tasks.ceph.mds.c.smithi104.stderr: 18: (MDSIOContextBase::complete(int)+0x16f) [0x55c817f340ff]
2021-05-27T23:14:13.385 INFO:tasks.ceph.mds.c.smithi104.stderr: 19: (MDSLogContextBase::complete(int)+0x40) [0x55c817f34310]
2021-05-27T23:14:13.385 INFO:tasks.ceph.mds.c.smithi104.stderr: 20: (Finisher::finisher_thread_entry()+0x16f) [0x7f75fe7b151f]
2021-05-27T23:14:13.386 INFO:tasks.ceph.mds.c.smithi104.stderr: 21: (()+0x7ea5) [0x7f75fc5dbea5]
2021-05-27T23:14:13.386 INFO:tasks.ceph.mds.c.smithi104.stderr: 22: (clone()+0x6d) [0x7f75fb2889fd]
2021-05-27T23:14:13.386 INFO:tasks.ceph.mds.c.smithi104.stderr:2021-05-27 23:14:13.415 7f75eec84700 -1 *** Caught signal (Aborted) **
2021-05-27T23:14:13.386 INFO:tasks.ceph.mds.c.smithi104.stderr: in thread 7f75eec84700 thread_name:fn_anonymous
2021-05-27T23:14:13.387 INFO:tasks.ceph.mds.c.smithi104.stderr: ceph version 14.2.21-281-gb7a4754d941 (b7a4754d941f016678ca17d548fafb10e7444735) nautilus (stable)
2021-05-27T23:14:13.387 INFO:tasks.ceph.mds.c.smithi104.stderr: 1: (()+0xf630) [0x7f75fc5e3630]
2021-05-27T23:14:13.387 INFO:tasks.ceph.mds.c.smithi104.stderr: 2: (gsignal()+0x37) [0x7f75fb1c03d7]
2021-05-27T23:14:13.387 INFO:tasks.ceph.mds.c.smithi104.stderr: 3: (abort()+0x148) [0x7f75fb1c1ac8]
2021-05-27T23:14:13.388 INFO:tasks.ceph.mds.c.smithi104.stderr: 4: (__gnu_cxx::__verbose_terminate_handler()+0x165) [0x7f75fbad0a95]
2021-05-27T23:14:13.388 INFO:tasks.ceph.mds.c.smithi104.stderr: 5: (()+0x5ea06) [0x7f75fbacea06]
2021-05-27T23:14:13.388 INFO:tasks.ceph.mds.c.smithi104.stderr: 6: (()+0x5ea33) [0x7f75fbacea33]
2021-05-27T23:14:13.388 INFO:tasks.ceph.mds.c.smithi104.stderr: 7: (()+0x5ec53) [0x7f75fbacec53]
2021-05-27T23:14:13.389 INFO:tasks.ceph.mds.c.smithi104.stderr: 8: (()+0x1772b) [0x7f75fe0eb72b]
2021-05-27T23:14:13.389 INFO:tasks.ceph.mds.c.smithi104.stderr: 9: (tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0xf3) [0x7f75fe109a03]
2021-05-27T23:14:13.389 INFO:tasks.ceph.mds.c.smithi104.stderr: 10: (std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&)+0x59) [0x55c817ca10f9]
2021-05-27T23:14:13.389 INFO:tasks.ceph.mds.c.smithi104.stderr: 11: (std::string::_Rep::_M_clone(std::allocator<char> const&, unsigned long)+0x1b) [0x55c817ca127b]
2021-05-27T23:14:13.389 INFO:tasks.ceph.mds.c.smithi104.stderr: 12: (std::string::reserve(unsigned long)+0x40) [0x55c817ca1340]
2021-05-27T23:14:13.390 INFO:tasks.ceph.mds.c.smithi104.stderr: 13: (std::string::append(char const*, unsigned long)+0x5e) [0x55c817ca141e]
2021-05-27T23:14:13.390 INFO:tasks.ceph.mds.c.smithi104.stderr: 14: (CDentry::make_path(filepath&, bool) const+0x90) [0x55c817e90230]
2021-05-27T23:14:13.390 INFO:tasks.ceph.mds.c.smithi104.stderr: 15: (operator<<(std::ostream&, CDentry const&)+0x6f) [0x55c817e904cf]
2021-05-27T23:14:13.390 INFO:tasks.ceph.mds.c.smithi104.stderr: 16: (MDCache::_create_system_file_finish(boost::intrusive_ptr<MutationImpl>&, CDentry*, unsigned long, MDSContext*)+0xbd) [0x55c817d7af8d]
2021-05-27T23:14:13.391 INFO:tasks.ceph.mds.c.smithi104.stderr: 17: (MDSContext::complete(int)+0x74) [0x55c817f33ea4]
2021-05-27T23:14:13.391 INFO:tasks.ceph.mds.c.smithi104.stderr: 18: (MDSIOContextBase::complete(int)+0x16f) [0x55c817f340ff]
2021-05-27T23:14:13.391 INFO:tasks.ceph.mds.c.smithi104.stderr: 19: (MDSLogContextBase::complete(int)+0x40) [0x55c817f34310]
2021-05-27T23:14:13.391 INFO:tasks.ceph.mds.c.smithi104.stderr: 20: (Finisher::finisher_thread_entry()+0x16f) [0x7f75fe7b151f]
2021-05-27T23:14:13.391 INFO:tasks.ceph.mds.c.smithi104.stderr: 21: (()+0x7ea5) [0x7f75fc5dbea5]
2021-05-27T23:14:13.392 INFO:tasks.ceph.mds.c.smithi104.stderr: 22: (clone()+0x6d) [0x7f75fb2889fd]
2021-05-27T23:14:13.392 INFO:tasks.ceph.mds.c.smithi104.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2021-05-27T23:14:13.392 INFO:tasks.ceph.mds.c.smithi104.stderr:
2021-05-27T23:14:13.392 INFO:tasks.ceph.mds.c.smithi104.stderr:  -405> 2021-05-27 23:14:10.413 7f75eec84700 -1 mds.0.openfiles _load_finish got (2) No such file or directory
2021-05-27T23:14:13.393 INFO:tasks.ceph.mds.c.smithi104.stderr:     0> 2021-05-27 23:14:13.415 7f75eec84700 -1 *** Caught signal (Aborted) **
2021-05-27T23:14:13.393 INFO:tasks.ceph.mds.c.smithi104.stderr: in thread 7f75eec84700 thread_name:fn_anonymous
2021-05-27T23:14:13.393 INFO:tasks.ceph.mds.c.smithi104.stderr:
2021-05-27T23:14:13.393 INFO:tasks.ceph.mds.c.smithi104.stderr: ceph version 14.2.21-281-gb7a4754d941 (b7a4754d941f016678ca17d548fafb10e7444735) nautilus (stable)

Related issues 2 (0 open — 2 closed)

Copied to CephFS - Backport #53163: octopus: mds: tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0xf3)ResolvedXiubo LiActions
Copied to CephFS - Backport #53164: pacific: mds: tcmalloc::allocate_full_cpp_throw_oom(unsigned long)+0xf3)ResolvedXiubo LiActions
Actions
Copy link

Also available in: Atom PDF