Project

General

Profile

Actions
Copy link

Bug #9984

open

lttng_probe_unregister hangs on shutdown

Added by Sage Weil over 9 years ago. Updated almost 4 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
LTTNG
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v16.0.0
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

(gdb) bt
#0  __lll_lock_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1  0x00007fbbb71a7657 in _L_lock_909 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007fbbb71a7480 in __GI___pthread_mutex_lock (mutex=0x7fbbb66a6920) at ../nptl/pthread_mutex_lock.c:79
#3  0x00007fbbb6473f5d in lttng_probe_unregister () from /usr/lib/x86_64-linux-gnu/liblttng-ust.so.0
#4  0x00007fbbb9b2673a in _dl_fini () at dl-fini.c:252
#5  0x00007fbbb68f9519 in __run_exit_handlers (status=0, listp=0x7fbbb6c7c6c8 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true) at exit.c:82
#6  0x00007fbbb68f9565 in __GI_exit (status=<optimized out>) at exit.c:104
#7  0x0000000000405412 in SysTestRunnable::start (this=0x7fff4c394100) at test/system/systest_runnable.cc:103
#8  0x0000000000406d3a in SysTestRunnable::run_until_finished (runnables=...) at test/system/systest_runnable.cc:174
#9  0x00000000004036a1 in main (argc=1, argv=0x7fff4c3943e8) at test/system/rados_delete_pools_parallel.cc:80

ubuntu@teuthology:/a/sage-2014-10-29_13:51:08-rados-master-distro-basic-multi/578520

Actions
Copy link
 #1

Updated by Josh Durgin over 9 years ago

maybe related to dynamic+static linking of lttng?

Actions
Copy link
 #2

Updated by Greg Farnum almost 7 years ago

  • Project changed from Ceph to RADOS
  • Category set to LTTNG
  • Priority changed from High to Normal
Actions
Copy link
 #3

Updated by Brad Hubbard almost 4 years ago

  • Affected Versions v16.0.0 added

/a/yuriw-2020-05-30_02:18:17-rados-wip-yuri-master_5.29.20-distro-basic-smithi/5104372

Possibly an instance of this (although I acknowledge it could be something completely different).

2020-05-30T08:50:42.712 INFO:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c ceph_test_rados_delete_pools_parallel
2020-05-30T08:50:44.954 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: starting.
2020-05-30T08:50:44.954 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: creating pool ceph_test_rados_delete_pools_parallel.smithi119-17935
2020-05-30T08:50:44.955 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: created object 0...
2020-05-30T08:50:44.955 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: created object 25...
2020-05-30T08:50:44.955 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: created object 49...
2020-05-30T08:50:44.955 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: finishing.
2020-05-30T08:50:44.955 INFO:teuthology.orchestra.run.smithi119.stdout:process_1_[17936]: shutting down.
2020-05-30T08:50:45.726 INFO:teuthology.orchestra.run.smithi119.stdout:process_2_[17937]: starting.
2020-05-30T08:50:45.727 INFO:teuthology.orchestra.run.smithi119.stdout:process_2_[17937]: deleting pool ceph_test_rados_delete_pools_parallel.smithi119-17935
2020-05-30T08:50:45.727 INFO:teuthology.orchestra.run.smithi119.stdout:process_2_[17937]: shutting down.
2020-05-30T08:50:48.206 INFO:teuthology.orchestra.run.smithi119.stdout:*******************************
2020-05-30T08:50:48.206 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: starting.
2020-05-30T08:50:48.207 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: creating pool ceph_test_rados_delete_pools_parallel.smithi119-17935
2020-05-30T08:50:48.207 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: created object 0...
2020-05-30T08:50:48.207 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: created object 25...
2020-05-30T08:50:48.207 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: created object 49...
2020-05-30T08:50:48.207 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: finishing.
2020-05-30T08:50:48.207 INFO:teuthology.orchestra.run.smithi119.stdout:process_3_[17972]: shutting down.
2020-05-30T08:50:48.211 INFO:teuthology.orchestra.run.smithi119.stdout:*******************************
2020-05-30T08:50:48.212 INFO:teuthology.orchestra.run.smithi119.stdout:process_5_[17974]: starting.
2020-05-30T08:50:48.212 INFO:teuthology.orchestra.run.smithi119.stdout:process_5_[17974]: listing objects.
2020-05-30T08:50:48.212 INFO:teuthology.orchestra.run.smithi119.stdout:process_5_[17974]: saw 0 objects
2020-05-30T08:50:48.212 INFO:teuthology.orchestra.run.smithi119.stdout:process_5_[17974]: shutting down.

The last time we launch ceph_test_rados_delete_pools_parallel we see that no 'process_4' is started and, after that, the process fails to make progress and the run times out and is marked dead so no logs unfortunately. I'll attempt to reproduce it.

Actions
Copy link

Also available in: Atom PDF