Project

General

Profile

Actions

Bug #38387

closed

[rbd-mirror] LeaderWatcher stuck in loop if pool deleted

Added by Jason Dillaman about 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Jason Dillaman
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
luminous,mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2019-02-19T16:09:17.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi176.stderr:2019-02-19 16:09:17.939 7f82567fc700 -1 rbd::mirror::LeaderWatcher: 0x1e73200 handle_notify_heartbeat: error notifying heartbeat: (2) No such file or directory, releasing leader
2019-02-19T16:09:17.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi176.stderr:2019-02-19 16:09:17.943 7f81d2ffd700 -1 rbd::mirror::image_deleter::TrashWatcher: 0x7f824800bf20 handle_unregister_watcher: error unregistering watcher for trash directory: (2) No such file or directory
2019-02-19T16:09:17.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi176.stderr:2019-02-19 16:09:17.943 7f81d2ffd700 -1 rbd::mirror::PoolWatcher: 0x7f824800e370 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory
2019-02-19T16:09:17.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi176.stderr:2019-02-19 16:09:17.947 7f82567fc700 -1 rbd::mirror::LeaderWatcher: 0x1e73200 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory
2019-02-19T16:09:17.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi176.stderr:2019-02-19 16:09:17.947 7f82567fc700 -1 rbd::mirror::LeaderWatcher: 0x1e73200 handle_release_leader_lock: error releasing lock: (2) No such file or directory
2019-02-19T16:09:18.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:18.083 7f1ad17fa700 -1 rbd::mirror::PoolWatcher: 0x7f1b50002ec0 operator(): error unregistering watcher for rbd_mirroring object: (2) No such file or directory
2019-02-19T16:09:19.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi176.stderr:2019-02-19 16:09:19.335 7f5f61c0e700 -1 rbd::mirror::LeaderWatcher: 0x2905c00 handle_shut_down_status_watcher: error shutting mirror status watcher down: (2) No such file or directory
2019-02-19T16:09:19.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi176.stderr:2019-02-19 16:09:19.335 7f5f61c0e700 -1 rbd::mirror::LeaderWatcher: 0x2905c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory
2019-02-19T16:09:19.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi176.stderr:2019-02-19 16:09:19.335 7f5f61c0e700 -1 rbd::mirror::InstanceWatcher: 0x28fa3f0 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.5390 object: (2) No such file or directory
2019-02-19T16:09:19.340 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi176.stderr:2019-02-19 16:09:19.335 7f5f1ffff700 -1 rbd::mirror::InstanceWatcher: 0x28fa3f0 handle_unregister_instance: error unregistering instance: (2) No such file or directory
2019-02-19T16:09:20.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.851 7f1b6db30700 -1 rbd::mirror::LeaderWatcher: 0x16ce2d0 handle_notify_lock_released: error notifying leader lock released: (2) No such file or directory
2019-02-19T16:09:20.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.851 7f1b6db30700 -1 rbd::mirror::LeaderWatcher: 0x16ce2d0 handle_shut_down_leader_lock: error shutting down leader lock: (2) No such file or directory
2019-02-19T16:09:20.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.851 7f1b6db30700 -1 rbd::mirror::LeaderWatcher: 0x16ce2d0 handle_shut_down_status_watcher: error shutting mirror status watcher down: (2) No such file or directory
2019-02-19T16:09:20.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.851 7f1b6db30700 -1 rbd::mirror::LeaderWatcher: 0x16ce2d0 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory
2019-02-19T16:09:20.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.851 7f1b6db30700 -1 rbd::mirror::InstanceWatcher: 0x16c6490 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4580 object: (2) No such file or directory
2019-02-19T16:09:20.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.851 7f1ae1ffb700 -1 rbd::mirror::InstanceWatcher: 0x16c6490 handle_unregister_instance: error unregistering instance: (2) No such file or directory
2019-02-19T16:09:20.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.859 7f1b6db30700 -1 rbd::mirror::LeaderWatcher: 0x1484c00 handle_shut_down_status_watcher: error shutting mirror status watcher down: (2) No such file or directory
2019-02-19T16:09:20.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.859 7f1b6db30700 -1 rbd::mirror::LeaderWatcher: 0x1484c00 handle_unregister_watch: error unregistering leader watcher for rbd_mirror_leader object: (2) No such file or directory
2019-02-19T16:09:20.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.859 7f1b6db30700 -1 rbd::mirror::InstanceWatcher: 0x14794d0 handle_unregister_watch: error unregistering instance watcher for rbd_mirror_instance.4576 object: (2) No such file or directory
2019-02-19T16:09:20.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi176.stderr:2019-02-19 16:09:20.859 7f1b2b7fe700 -1 rbd::mirror::InstanceWatcher: 0x14794d0 handle_unregister_instance: error unregistering instance: (2) No such file or directory
2019-02-19T16:09:21.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi176.stderr:2019-02-19 16:09:21.447 7f55a1807700 -1 rbd::mirror::LeaderWatcher: 0x1980250 handle_post_acquire_leader_lock: error acquiring leader lock: (2) No such file or directory
2019-02-19T16:09:21.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi176.stderr:2019-02-19 16:09:21.447 7f55a1807700 -1 rbd::mirror::LeaderWatcher: 0x1980250 handle_acquire_leader_lock: error acquiring lock: (2) No such file or directory

...

Related issues 2 (0 open2 closed)

Copied to rbd - Backport #38508: luminous: [rbd-mirror] LeaderWatcher stuck in loop if pool deletedResolvedJason DillamanActions
Copied to rbd - Backport #38509: mimic: [rbd-mirror] LeaderWatcher stuck in loop if pool deletedResolvedJason DillamanActions
Actions #1

Updated by Jason Dillaman about 5 years ago

  • Backport set to luminous,mimic
Actions #2

Updated by Jason Dillaman about 5 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 26523
Actions #3

Updated by Jason Dillaman about 5 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #4

Updated by Nathan Cutler about 5 years ago

  • Copied to Backport #38508: luminous: [rbd-mirror] LeaderWatcher stuck in loop if pool deleted added
Actions #5

Updated by Nathan Cutler about 5 years ago

  • Copied to Backport #38509: mimic: [rbd-mirror] LeaderWatcher stuck in loop if pool deleted added
Actions #6

Updated by Nathan Cutler over 4 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF