Project

General

Profile

Actions
Copy link

Bug #50074

open

[RFE] close idle kafka connections

Added by Yuval Lifshitz about 3 years ago. Updated over 1 year ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Kalpesh Pandya
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
notification backport_processed
Backport:
pacific, octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
42840
Crash signature (v1):
Crash signature (v2):

Description

delete the connection once no topic references them anymore

Related issues 2 (0 open — 2 closed)

Copied to rgw - Backport #52663: octopus: [RFE] close idle kafka connectionsRejectedActions
Copied to rgw - Backport #52664: pacific: [RFE] close idle kafka connectionsRejectedKalpesh PandyaActions
Actions
Copy link
 #1

Updated by Yuval Lifshitz about 3 years ago

connections should just timeout when idle, even if a topic exists.

Actions
Copy link
 #2

Updated by Yuval Lifshitz almost 3 years ago

  • Assignee set to Kalpesh Pandya
Actions
Copy link
 #3

Updated by Yuval Lifshitz almost 3 years ago

  • Subject changed from [RFE] add reference counting to kafka connections to [RFE] close idle kafka connections

currently, connections never expire, the existing "disconnect()" API is not used from inside the RGW.
since multiple topics may be using the same kafka broker, we cannot disconnect based on topic deletion.
not deleting the connection may create a problem for the case when a kafka broker is decommissioned and the RGW keep on trying to reconnect to it, emitting error log messages which may be confusing for the user.

as part of this item:
  • remove the disconnect() API including code handling the marked_for_deletion flag
  • check librdkafka configuration [1] to set a timeout on connections. as well as the callback to remove it from the connection list

[1] https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md

Actions
Copy link
 #4

Updated by Kalpesh Pandya over 2 years ago

  • Status changed from New to Pending Backport
  • Pull request ID set to 42840
Actions
Copy link
 #5

Updated by Backport Bot over 2 years ago

  • Copied to Backport #52663: octopus: [RFE] close idle kafka connections added
Actions
Copy link
 #6

Updated by Backport Bot over 2 years ago

  • Copied to Backport #52664: pacific: [RFE] close idle kafka connections added
Actions
Copy link
 #7

Updated by Backport Bot over 1 year ago

  • Tags changed from notification to notification backport_processed
Actions
Copy link

Also available in: Atom PDF