Project

General

Profile

Actions

Feature #63915

open

propagate kafka errors to client in case of sync notifications

Added by Yuval Lifshitz 5 months ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Target version:
-
% Done:

0%

Source:
Tags:
kafka, notifications
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

when broker is down it is expected to reply with an error immdeiatly.
currently, the errors are reported back to only after connection is marked idle, and being deleted.
since this timeout is only used to "garbage collect" decomissioned connections it is set to a high number, meaning that the reply that the notifications failed takes too long.
  • in case of persistent notifications, it will fill up memory with pending coroutines
  • in case of non-persistent notifications, it will block all frontend coroutines, making the RGW unusable

in addition, any for non-persistent notifications, since this is detected in the "publish_commit()" stage, it will just be reported and there will be no feedback to the client that the notifications failed.
to solve that, the "publish_reserve()" step should be used to verify the connectivity to the broker

Actions

Also available in: Atom PDF