Project

General

Profile

Actions

Bug #16454

closed

"tcmalloc: large alloc 4294967296 bytes" in rados-wip-yuri-testing_2016_6_22-distro-basic-smithi

Added by Yuri Weinstein almost 8 years ago. Updated almost 8 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rados
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Run: http://pulpito.ceph.com/yuriw-2016-06-22_17:09:03-rados-wip-yuri-testing_2016_6_22-distro-basic-smithi/
Job: 271597
Logs: http://qa-proxy.ceph.com/teuthology/yuriw-2016-06-22_17:09:03-rados-wip-yuri-testing_2016_6_22-distro-basic-smithi/271597/teuthology.log

2016-06-22T19:35:44.015 INFO:teuthology.orchestra.run.smithi018.stderr:instructing pg 0.17 on osd.3 to deep-scrub
2016-06-22T19:35:44.935 INFO:tasks.ceph.osd.3.smithi058.stderr:2016-06-23 02:35:45.072538 7f72b7515700 -1 log_channel(cluster) log [ERR] : 0.17 shard 3: soid 0:e80963ad:::benchmark_data_smithi018_12314_object498:head data_digest 0xf3ab4886 != known data_digest 0xc527f3f0 from auth shard 2, size 4097 != known size 4096
2016-06-22T19:35:44.977 INFO:tasks.ceph.osd.3.smithi058.stderr:2016-06-23 02:35:45.115320 7f72b4d10700 -1 log_channel(cluster) log [ERR] : 0.17 deep-scrub 0 missing, 1 inconsistent objects
2016-06-22T19:35:44.978 INFO:tasks.ceph.osd.3.smithi058.stderr:2016-06-23 02:35:45.115332 7f72b4d10700 -1 log_channel(cluster) log [ERR] : 0.17 deep-scrub 1 errors

and then

2016-06-22T19:35:55.337 INFO:teuthology.orchestra.run.smithi058.stderr:mkdir: cannot create directory ‘/var/lib/ceph/osd/ceph-3/fuse/0.17_head/all/#0:e80963ad:::benchmark_data_smithi018_12314_object498:head#’: File exists
2016-06-22T19:35:55.338 INFO:teuthology.orchestra.run.smithi058:Running: "sudo cp /tmp/tmp5DM_1F '/var/lib/ceph/osd/ceph-3/fuse/0.17_head/all/#0:e80963ad:::benchmark_data_smithi018_12314_object498:head#/data'" 
2016-06-22T19:35:55.417 INFO:tasks.ceph.osd.3.smithi058.stderr:tcmalloc: large alloc 4294967296 bytes == 0x7f72e035e000 @  0x7f72d3b28c4c 0x7f72d3b2b547 0x7f72d3b49a02 0x7f72d49faef5 0x7f72d49fb1a5 0x7f72d49fb1d4 0x7f72d4607819 0x7f72d2e6965c 0x7f72d2e697e8 0x7f72d2e72389 0x7f72d2e6ea6c 0x7f72d2e671c8 0x7f72d4606e57 0x7f72d460ce5d 0x7f72d2c47182 0x7f72d0b5a47d (nil)
2016-06-22T19:35:56.914 INFO:tasks.ceph.osd.3.smithi058.stderr:*** Caught signal (Segmentation fault) **
2016-06-22T19:35:56.914 INFO:tasks.ceph.osd.3.smithi058.stderr: in thread 7f72ca6c9700 thread_name:safe_timer
2016-06-22T19:35:56.916 INFO:tasks.ceph.osd.3.smithi058.stderr: ceph version 10.2.0-2691-gc95c188 (c95c188f6c06bf62d75e1a5b4ecf3b5c27939915)
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr: 1: (()+0x94aa22) [0x7f72d48f0a22]
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr: 2: (()+0x10340) [0x7f72d2c4f340]
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr: 3: (SafeTimer::timer_thread()+0xe6) [0x7f72d49da7b6]
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr: 4: (SafeTimerThread::entry()+0xd) [0x7f72d49dc15d]
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr: 5: (()+0x8182) [0x7f72d2c47182]
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr: 6: (clone()+0x6d) [0x7f72d0b5a47d]
2016-06-22T19:35:56.917 INFO:tasks.ceph.osd.3.smithi058.stderr:2016-06-23 02:35:57.054426 7f72ca6c9700 -1 *** Caught signal (Segmentation fault) **
2016-06-22T19:35:56.918 INFO:tasks.ceph.osd.3.smithi058.stderr: in thread 7f72ca6c9700 thread_name:safe_timer
2016-06-22T19:35:56.918 INFO:tasks.ceph.osd.3.smithi058.stderr:
2016-06-22T19:35:56.918 INFO:tasks.ceph.osd.3.smithi058.stderr: ceph version 10.2.0-2691-gc95c188 (c95c188f6c06bf62d75e1a5b4ecf3b5c27939915)
2016-06-22T19:35:56.918 INFO:tasks.ceph.osd.3.smithi058.stderr: 1: (()+0x94aa22) [0x7f72d48f0a22]
2016-06-22T19:35:56.918 INFO:tasks.ceph.osd.3.smithi058.stderr: 2: (()+0x10340) [0x7f72d2c4f340]
2016-06-22T19:35:56.919 INFO:tasks.ceph.osd.3.smithi058.stderr: 3: (SafeTimer::timer_thread()+0xe6) [0x7f72d49da7b6]
2016-06-22T19:35:56.919 INFO:tasks.ceph.osd.3.smithi058.stderr: 4: (SafeTimerThread::entry()+0xd) [0x7f72d49dc15d]
2016-06-22T19:35:56.919 INFO:tasks.ceph.osd.3.smithi058.stderr: 5: (()+0x8182) [0x7f72d2c47182]
2016-06-22T19:35:56.919 INFO:tasks.ceph.osd.3.smithi058.stderr: 6: (clone()+0x6d) [0x7f72d0b5a47d]
2016-06-22T19:35:56.919 INFO:tasks.ceph.osd.3.smithi058.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

@Samuel Hassine complete guess - related to https://github.com/ceph/ceph/pull/9456 ?


Related issues 1 (0 open1 closed)

Is duplicate of Ceph - Bug #16414: scrub_test now failing, probably related to 9e2b455f45cca2f31b19c73d761661a9028c22ffClosedDavid Zafman06/22/2016

Actions
Actions #1

Updated by Yuri Weinstein almost 8 years ago

  • Description updated (diff)
Actions #2

Updated by Yuri Weinstein almost 8 years ago

  • Description updated (diff)
Actions #3

Updated by Samuel Just almost 8 years ago

  • Priority changed from Normal to Urgent
Actions #4

Updated by Yuri Weinstein almost 8 years ago

  • Description updated (diff)
  • Priority changed from Urgent to Normal
Actions #5

Updated by Samuel Just almost 8 years ago

  • Is duplicate of Bug #16414: scrub_test now failing, probably related to 9e2b455f45cca2f31b19c73d761661a9028c22ff added
Actions #6

Updated by Samuel Just almost 8 years ago

I think this is a dup of 16414, that both are use-after-free bugs.

Actions #7

Updated by Samuel Just almost 8 years ago

  • Status changed from New to Duplicate
Actions

Also available in: Atom PDF