[jira] [Updated] (KAFKA-20898) PushTelemetry sets terminating flag on validation failure, permanently locking out client instance

["Chia-Ping Tsai (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chia-Ping Tsai updated KAFKA-20898:
-----------------------------------
    Fix Version/s: 4.5.0

> PushTelemetry sets terminating flag on validation failure, permanently locking out client instance
> --------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-20898
>                 URL: https://issues.apache.org/jira/browse/KAFKA-20898
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Ken Huang
>            Assignee: Ken Huang
>            Priority: Major
>             Fix For: 4.5.0
>
>
> [https://github.com/apache/kafka/blob/89ccd6a1260130e1a5ea83a0aa98a7d397482ace/server/src/main/java/org/apache/kafka/server/ClientMetricsManager.java#L209]
> [https://cwiki.apache.org/confluence/spaces/KAFKA/pages/173085915/KIP-714+Client+metrics+and+observability]
> {quote}*Client termination*
> When a client with an active metrics subscription is being shut down, it should send its final metrics without waiting for the PushIntervalMs time.
> To avoid the receiving broker’s metrics rate-limiter discarding this out-of-profile push, the {{PushTelemetryRequest.Terminating}} field must be set to true. A broker must only allow one such unthrottled metrics push for each combination of client instance ID and SubscriptionId.
> In the event that the client’s metric subscription has changed and the final metrics push fails with error code {{UNKNOWN_SUBSCRIPTION_ID}} , the terminating client can choose to obtain a new subscription ID by sending a GetTelemetrySubscriptionsRequest and then immediately sending a PushTelemetryRequest with the Terminating flag set to true, or it can choose to abandon sending a final metrics push.{quote}
> The implementation here differs from the behavior described in the KIP. The second terminating request will fail.
> The flow is as follows:
>  # The consumer is running normally, and the previous {{GetTelemetrySubscriptions}} request returns {{{}subscriptionId = X{}}}.
>  # The admin runs {{{}kafka-client-metrics.sh --alter --interval 60000{}}}, causing the broker to recalculate the subscription ID, which changes from *X* to {*}Y{*}.
>  # The consumer begins shutting down and sends its final {{PushTelemetry}} request with {{terminating=true}} and the old {{{}subscriptionId = X{}}}.
>  # The broker fails the subscription ID validation and returns {{{}UNKNOWN_SUBSCRIPTION_ID{}}}, but due to a bug, the {{terminating=true}} state is still recorded.
>  # The consumer attempts to recover by calling {{GetTelemetrySubscriptions}} again, obtains the new {{{}subscriptionId = Y{}}}, and sends another {{PushTelemetry}} request with {{{}terminating=true{}}}.
>  # The broker sees that the instance has already been marked as {{terminating=true}} and returns {{{}INVALID_REQUEST{}}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)