You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2022/12/12 09:11:09 UTC
[GitHub] [kafka] chia7712 opened a new pull request, #12979: MINOR: remove "is-future" from metrics tags after replace current log…
chia7712 opened a new pull request, #12979:
URL: https://github.com/apache/kafka/pull/12979
we don't remove "is-future=true" tag from future log after the future log becomes "current" log. It causes two potential issues:
1. the metrics monitors can't get metrics of Log if they don't trace the property "is-future=true".
2. all Log metrics of specify partition get removed if the partition is moved to another folder again.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a diff in pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
showuon commented on code in PR #12979:
URL: https://github.com/apache/kafka/pull/12979#discussion_r1055056767
##########
core/src/main/scala/kafka/log/LogManager.scala:
##########
@@ -1079,6 +1079,9 @@ class LogManager(logDirs: Seq[File],
if (destLog == null)
throw new KafkaStorageException(s"The future replica for $topicPartition is offline")
+ // the metrics tags still contain "future", so we have to remove it.
+ // we will add metrics back after sourceLog remove the metrics
+ destLog.removeLogMetrics()
destLog.renameDir(UnifiedLog.logDirName(topicPartition), true)
destLog.updateHighWatermark(sourceLog.highWatermark)
Review Comment:
Do you think we should remove log metrics after destLog got renamed? I'm thinking if we'll get stuck for some time during `renameDir` and it'll be helpful to have metrics reporting it. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a diff in pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
showuon commented on code in PR #12979:
URL: https://github.com/apache/kafka/pull/12979#discussion_r1056904807
##########
core/src/main/scala/kafka/log/UnifiedLog.scala:
##########
@@ -1795,10 +1803,10 @@ class UnifiedLog(@volatile var logStartOffset: Long,
* remove deleted log metrics
*/
private[log] def removeLogMetrics(): Unit = {
- removeMetric(LogMetricNames.NumLogSegments, tags)
- removeMetric(LogMetricNames.LogStartOffset, tags)
- removeMetric(LogMetricNames.LogEndOffset, tags)
- removeMetric(LogMetricNames.Size, tags)
+ metricNames.foreach {
+ case (name, tags) => removeMetric(name, tags)
+ }
+ metricNames = Map.empty
Review Comment:
Is it because when we tried to removeMetrics, the tags might become "non-future"?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a diff in pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
showuon commented on code in PR #12979:
URL: https://github.com/apache/kafka/pull/12979#discussion_r1056904583
##########
core/src/main/scala/kafka/log/UnifiedLog.scala:
##########
@@ -1795,10 +1803,10 @@ class UnifiedLog(@volatile var logStartOffset: Long,
* remove deleted log metrics
*/
private[log] def removeLogMetrics(): Unit = {
- removeMetric(LogMetricNames.NumLogSegments, tags)
- removeMetric(LogMetricNames.LogStartOffset, tags)
- removeMetric(LogMetricNames.LogEndOffset, tags)
- removeMetric(LogMetricNames.Size, tags)
+ metricNames.foreach {
+ case (name, tags) => removeMetric(name, tags)
+ }
+ metricNames = Map.empty
Review Comment:
Could you explain why we need this change to remove metrics, instead of the original one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on pull request #12979: MINOR: remove "is-future" from metrics tags after replace current log…
Posted by GitBox <gi...@apache.org>.
showuon commented on PR #12979:
URL: https://github.com/apache/kafka/pull/12979#issuecomment-1359274755
Thanks for the patch. I'll take a look this week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on a diff in pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
chia7712 commented on code in PR #12979:
URL: https://github.com/apache/kafka/pull/12979#discussion_r1056909612
##########
core/src/main/scala/kafka/log/UnifiedLog.scala:
##########
@@ -1795,10 +1803,10 @@ class UnifiedLog(@volatile var logStartOffset: Long,
* remove deleted log metrics
*/
private[log] def removeLogMetrics(): Unit = {
- removeMetric(LogMetricNames.NumLogSegments, tags)
- removeMetric(LogMetricNames.LogStartOffset, tags)
- removeMetric(LogMetricNames.LogEndOffset, tags)
- removeMetric(LogMetricNames.Size, tags)
+ metricNames.foreach {
+ case (name, tags) => removeMetric(name, tags)
+ }
+ metricNames = Map.empty
Review Comment:
> Is it because when we tried to removeMetrics, the tags might become "non-future"?
yep. The tags DOES NOT include "future" after the log dir get renamed. Hence, we have to keep previous metrics name in order to delete correct metrics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on a diff in pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
chia7712 commented on code in PR #12979:
URL: https://github.com/apache/kafka/pull/12979#discussion_r1055146924
##########
core/src/main/scala/kafka/log/LogManager.scala:
##########
@@ -1079,6 +1079,9 @@ class LogManager(logDirs: Seq[File],
if (destLog == null)
throw new KafkaStorageException(s"The future replica for $topicPartition is offline")
+ // the metrics tags still contain "future", so we have to remove it.
+ // we will add metrics back after sourceLog remove the metrics
+ destLog.removeLogMetrics()
destLog.renameDir(UnifiedLog.logDirName(topicPartition), true)
destLog.updateHighWatermark(sourceLog.highWatermark)
Review Comment:
> it'll be helpful to have metrics reporting it
make sense. will copy that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
chia7712 commented on PR #12979:
URL: https://github.com/apache/kafka/pull/12979#issuecomment-1362800962
the new test is unstable. Will fix it later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 merged pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
chia7712 merged PR #12979:
URL: https://github.com/apache/kafka/pull/12979
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
chia7712 commented on PR #12979:
URL: https://github.com/apache/kafka/pull/12979#issuecomment-1361909104
file a jira as this is related to true bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] chia7712 commented on pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
chia7712 commented on PR #12979:
URL: https://github.com/apache/kafka/pull/12979#issuecomment-1364541293
@showuon Could you take a look again? the latest commit includes extra changes to address your comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] showuon commented on a diff in pull request #12979: KAFKA-14544 The "is-future" should be removed from metrics tags after future log becomes current log
Posted by GitBox <gi...@apache.org>.
showuon commented on code in PR #12979:
URL: https://github.com/apache/kafka/pull/12979#discussion_r1055056767
##########
core/src/main/scala/kafka/log/LogManager.scala:
##########
@@ -1079,6 +1079,9 @@ class LogManager(logDirs: Seq[File],
if (destLog == null)
throw new KafkaStorageException(s"The future replica for $topicPartition is offline")
+ // the metrics tags still contain "future", so we have to remove it.
+ // we will add metrics back after sourceLog remove the metrics
+ destLog.removeLogMetrics()
destLog.renameDir(UnifiedLog.logDirName(topicPartition), true)
destLog.updateHighWatermark(sourceLog.highWatermark)
Review Comment:
Do you think we should remove log metrics after destLog got renamed? I'm thinking if we got stuck for some time during `renameDir` and it'll be helpful to have metrics reporting it. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org