[jira] [Comment Edited] (KAFKA-2419) Allow certain Sensors to be garbage collected after inactivity

["Chenxin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17096049#comment-17096049 ] 

Chenxin edited comment on KAFKA-2419 at 4/30/20, 2:57 AM:
----------------------------------------------------------

Thanks for the reply.

I am not very good at kafka. I just found there is thousands of metrics and sensors as my picture shows.

This case happened we us kafka streams.Maybe something wrong with my client config and kafka server. Sometimes it keeps rebalancing all the time, and lots of metrics and sensors were created.

 

I saw there is ExpireSensorTask, but nowhere can use it.

 

My client config as below:
{code:java}
//
Map<String, Object> props = new HashMap<>();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, APPLICATION_ID);
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaBootStrapServers);
props.put("auto.offset.reset", "latest");
props.put("enable.auto.commit", true);
props.put("compression.type", "lz4");
props.put(StreamsConfig.BUFFERED_RECORDS_PER_PARTITION_CONFIG, 600);
props.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 5242880);
props.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 3000);
props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 8);
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
props.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG, OrderTimeTimestampExtractor.class.getName());
{code}


was (Author: icechen):
Thanks for the reply.

I am not very good at kafka. I just found there is thousands of metrics and sensors as my picture shows.

This case happened we us kafka streams.Maybe something wrong with my client config and kafka server. Sometimes it keeps rebalancing all the time, and lots of metrics and sensors were created.

 

My client config as below:
{code:java}
//
Map<String, Object> props = new HashMap<>();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, APPLICATION_ID);
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaBootStrapServers);
props.put("auto.offset.reset", "latest");
props.put("enable.auto.commit", true);
props.put("compression.type", "lz4");
props.put(StreamsConfig.BUFFERED_RECORDS_PER_PARTITION_CONFIG, 600);
props.put(StreamsConfig.CACHE_MAX_BYTES_BUFFERING_CONFIG, 5242880);
props.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 3000);
props.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, 8);
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass().getName());
props.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG, OrderTimeTimestampExtractor.class.getName());
{code}

> Allow certain Sensors to be garbage collected after inactivity
> --------------------------------------------------------------
>
>                 Key: KAFKA-2419
>                 URL: https://issues.apache.org/jira/browse/KAFKA-2419
>             Project: Kafka
>          Issue Type: Improvement
>    Affects Versions: 0.9.0.0
>            Reporter: Aditya Auradkar
>            Assignee: Aditya Auradkar
>            Priority: Blocker
>              Labels: quotas
>             Fix For: 0.9.0.0
>
>         Attachments: Leak1, Leak1.jpg, leak2.jpg
>
>
> Currently, metrics cannot be removed once registered. 
> Implement a feature to remove certain sensors after a certain period of inactivity (perhaps configurable).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)