You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2020/05/18 04:13:18 UTC
[GitHub] [kafka] navina opened a new pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
navina opened a new pull request #8684:
URL: https://github.com/apache/kafka/pull/8684
`SelectorMetrics` has a per-connection metrics, which means the number of `MetricName` objects and the strings associated with it (such as group name and description) grows with the number of connections in the client. This overhead of duplicate string objects is amplified when there are multiple instances of kafka clients within the same JVM.
This patch address some of the memory overhead by making `metricGrpName` a constant and introducing a new constant `perConnectionMetricGrpName`. Additionally, the strings for metric name and description in `createMeter` have been interned since there are about 8 types of meter metric for every single client.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] hachikuji merged pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
hachikuji merged pull request #8684:
URL: https://github.com/apache/kafka/pull/8684
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] navina commented on pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
navina commented on pull request #8684:
URL: https://github.com/apache/kafka/pull/8684#issuecomment-637827964
@ijuma I have removed the `intern()` calls. Please take a look. Thanks!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] navina edited a comment on pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
navina edited a comment on pull request #8684:
URL: https://github.com/apache/kafka/pull/8684#issuecomment-630444877
@ijuma What issues with `intern()` are you referring to? I know that there can be a performance hit when there are a lot of intern strings. I believe string interning mechanism has been improved in the later versions of java such as jdk8 / 9.
I would like to understand the concern better before removing the string interning showed here.
Thanks for the quick feedback!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #8684:
URL: https://github.com/apache/kafka/pull/8684#issuecomment-640295811
@hachikuji feel free to merge, my main concern was the `intern` calls that have been removed. We should update the commit message to remove the bit about interning before merging.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] navina commented on pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
navina commented on pull request #8684:
URL: https://github.com/apache/kafka/pull/8684#issuecomment-630444877
@ijuma What issues with intern are you referring to? I know that there can be a performance hit when there are a lot of intern strings. I believe string interning mechanism has been improved in the later versions of java such as jdk8 / 9.
I would like to understand the concern better before removing the string interning showed here.
Thanks for the quick feedback!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #8684:
URL: https://github.com/apache/kafka/pull/8684#issuecomment-636348427
@navina it's a native call (implying JNI overhead) and somewhat inflexible. We'd want to add a JMH benchmark to understand the full impact.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] ijuma commented on pull request #8684: KAFKA-10012 Reducing memory overhead associated with strings in Metri…
Posted by GitBox <gi...@apache.org>.
ijuma commented on pull request #8684:
URL: https://github.com/apache/kafka/pull/8684#issuecomment-630391729
Thanks for the PR. Java's built in string interning mechanism is known to have issues. Not sure we want to do that. Maybe we can remove that part of the change from this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org