You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "Ritikj55 (via GitHub)" <gi...@apache.org> on 2026/06/26 10:47:35 UTC
[PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Ritikj55 opened a new pull request, #22684:
URL: https://github.com/apache/kafka/pull/22684
Description
Added documentation for new consumer metrics related to the CONSUMER protocol implementation
Below are metrics for which documentation has been added :
1. time-between-network-thread-poll
2. application-event-queue-size
3. application-event-queue-time-avg
4. application-event-queue-processing-time-avg
5. application-events-expired-count
6. background-event-queue-size
7. background-event-queue-time-avg
8. background-event-queue-processing-time-avg
9. unsent-requests-queue-size
10. unsent-requests-queue-time-avg
Testing :
Tested using javascript wrapper in local for checking indentation
<img width="1250" height="494" alt="image" src="https://github.com/user-attachments/assets/bcfbecbe-fadb-4f5c-a289-c8b03e3b1f76" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627759385
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
Review Comment:
There is a max variant as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-4880609841
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-4842547965
Hey @AndrewJSchofield , can you please review this pr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3639501873
##########
docs/operations/monitoring.md:
##########
@@ -3447,44 +3447,45 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
<td>
kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr>
+</td> </tr>
<tr>
<td>
-time-between-network-thread-poll</td>
+application-event-queue-size
+</td>
<td>
-The average delay between invocations of poll (for CONSUMER protocol)
+The current number of events in the queue to send from the application thread to the background thread(for CONSUMER Protocol,will be referred as 'CP' in the below metrics).
Review Comment:
nit: Spacing and a few words can be improved. `background thread (for CONSUMER protocol, will be referred to as 'CP' in the following metrics)`
##########
docs/operations/monitoring.md:
##########
@@ -3493,11 +3494,24 @@ application-event-queue-processing-time-avg
</td>
<td>
-The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+The average time, in ms,that the background thread took to process all available application events((for CP only).
Review Comment:
nit: `events((forCP` should be `events (for CP`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627746756
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
Review Comment:
the metric name says `avg` and description says `maximum` time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-4842567838
Hey @dajac , can you please review this pr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-5058471883
> @Ritikj55 Thanks for the PR, there are some changes required
>
> Some metrics are missing (their -max counterparts, 5-6 entries).
>
> `time-between-network-thread-poll` should have `-max`/`-avg` suffix.
>
> `application-event-queue-processing-time-avg` definition seems incorrect. Verify from code
>
> mbean pattern is not consistent. Please conform to the common style.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627746756
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
Review Comment:
the metric name says `avg` and description says `maximum` time. Also this is different from what the code description says
```
avg time that the background thread takes to process all available application events.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-4808844763
@kirktrue please review this pr
<img width="1250" height="494" alt="image" src="https://github.com/user-attachments/assets/e4f4b728-1937-4489-b2bc-0976751746ac" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627766659
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
the mbean pattern is different here (other places use regex)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627742445
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
Review Comment:
The metric name here does not match the actual exported metric (Kafka exports `time-between-network-thread-poll-avg` and `time-between-network-thread-poll-max`, not a bare `time-between-network-thread-poll`). This makes the documentation inaccurate and harder to query via JMX.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer background event queue (for CONSUMER protocol)
+</td>
+
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that background events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-processing-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that the consumer took to process all available background events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+unsent-requests-queue-size
+</td>
+<td>
+
+The current number of unsent requests in the consumer network (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+unsent-requests-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that requests are taking to be sent in the consumer network (for CONSUMER protocol)
Review Comment:
This description refers to the "consumer network", but the metric tracks time in the background thread (per the metric’s definition). Aligning the wording avoids confusion when interpreting the metric.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer background event queue (for CONSUMER protocol)
+</td>
+
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that background events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer background event queue (for CONSUMER protocol)
+</td>
+
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that background events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-processing-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that the consumer took to process all available background events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer background event queue (for CONSUMER protocol)
+</td>
+
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that background events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-processing-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that the consumer took to process all available background events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+unsent-requests-queue-size
+</td>
+<td>
+
+The current number of unsent requests in the consumer network (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+unsent-requests-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that requests are taking to be sent in the consumer network (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
Review Comment:
This description says "maximum" but the metric name is `...-avg`. The wording should describe the average, and it should align with the metric’s meaning (background thread processing application events).
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer background event queue (for CONSUMER protocol)
+</td>
+
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that application events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-processing-time-avg
+</td>
+<td>
+
+The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-events-expired-count
+</td>
+<td>
+
+The current number of expired application events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer background event queue (for CONSUMER protocol)
+</td>
+
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that background events are taking to be dequeued (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+background-event-queue-processing-time-avg
+</td>
+<td>
+
+The average time, in milliseconds, that the consumer took to process all available background events (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+unsent-requests-queue-size
+</td>
+<td>
+
+The current number of unsent requests in the consumer network (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
Review Comment:
This section uses `client-id={clientId}`, but the surrounding Consumer Metrics table documents MBeans using a regex pattern (e.g., `client-id=([-\.\w]+)` at docs/operations/monitoring.md:3423). For consistency with the rest of the table, use the same pattern here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3645422126
##########
docs/operations/monitoring.md:
##########
@@ -3493,11 +3494,24 @@ application-event-queue-processing-time-avg
</td>
<td>
-The maximum time, in milliseconds, that the consumer network took to process all available application events (for CONSUMER protocol)
+The average time, in ms,that the background thread took to process all available application events((for CP only).
Review Comment:
Hey @AndrewJSchofield , resolved all the comments please rereview it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield merged PR #22684:
URL: https://github.com/apache/kafka/pull/22684
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-5058480016
Hey @smjn, have resolved all the comments; please review them again , Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627757972
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
+<td>
+
+The average delay between invocations of poll (for CONSUMER protocol)
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td> </tr>
+<tr>
+<td>
+
+application-event-queue-size
+</td>
+<td>
+
+The current number of events in the consumer network application event queue (for CONSUMER protocol).
+</td>
+<td>
+
+kafka.consumer:type=consumer-metrics,client-id={clientId}
+</td></tr>
+<tr>
+<td>
+
+application-event-queue-time-avg
Review Comment:
There seems to be a corresponding `-max` metric as well. Are we not documenting that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "smjn (via GitHub)" <gi...@apache.org>.
smjn commented on code in PR #22684:
URL: https://github.com/apache/kafka/pull/22684#discussion_r3627759385
##########
docs/operations/monitoring.md:
##########
@@ -3446,8 +3446,139 @@ The total time the Consumer spent committing offsets in nanoseconds (for AOS).
</td>
<td>
-kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
-</td> </tr> </table>
+kafka.consumer:type=consumer-metrics,client-id=([-.\w]+)
+</td> </tr>
+<tr>
+<td>
+
+time-between-network-thread-poll</td>
Review Comment:
This does not seem correct - the actual metric has `-avg` or `-max` suffix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-19491 CONSUMER protocol metrics documentation [kafka]
Posted by "Ritikj55 (via GitHub)" <gi...@apache.org>.
Ritikj55 commented on PR #22684:
URL: https://github.com/apache/kafka/pull/22684#issuecomment-5070048939
Hey @AndrewJSchofield , have resolved all the comments; please re-review them
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org