You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "tibrewalpratik17 (via GitHub)" <gi...@apache.org> on 2023/03/07 23:37:51 UTC
[GitHub] [pinot] tibrewalpratik17 opened a new issue, #10393: Helix metrics missing from 0.11 version
tibrewalpratik17 opened a new issue, #10393:
URL: https://github.com/apache/pinot/issues/10393
During upgrading our cluster from 0.10 -> 0.11, we also had to upgrade Helix version from 0.9.8 -> 1.0.4. After rolling out the new version, we are seeing Helix metrics like `helix.PendingMessages`, `helix.CompletedMessages`, `helix.FailedMessages` are not getting published. We are trying to get these metrics for Pinot-server.
Is this expected with the upgraded Helix version? We are seeing the same issue on 0.12 and latest master as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] tibrewalpratik17 commented on issue #10393: Helix metrics missing from 0.11 version
Posted by "tibrewalpratik17 (via GitHub)" <gi...@apache.org>.
tibrewalpratik17 commented on issue #10393:
URL: https://github.com/apache/pinot/issues/10393#issuecomment-1487456181
I got the issue. Till Pinot v0.10 (helix 0.9.8), we were publishing these metrics under the mbean `CLMParticipantReport:ParticipantName=<instance_name>` but with Pinot v0.11 (helix 1.0.4) we are publishing this under `CLMParticipantReport:MonitorType=ParticipantMessageMonitor,ParticipantName=<instance_name>`.
Updating the mbean helped us get our metrics back. Thanks for the help @qqu0127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] jackjlli commented on issue #10393: Helix metrics missing from 0.11 version
Posted by "jackjlli (via GitHub)" <gi...@apache.org>.
jackjlli commented on issue #10393:
URL: https://github.com/apache/pinot/issues/10393#issuecomment-1460643463
Yes, the Helix POC and I are looking at it right now.
Detailed discussion can be found in this slack thread:
https://apache-pinot.slack.com/archives/CDRCA57FC/p1677870132262669
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Helix metrics missing from 0.11 version [pinot]
Posted by "tibrewalpratik17 (via GitHub)" <gi...@apache.org>.
tibrewalpratik17 closed issue #10393: Helix metrics missing from 0.11 version
URL: https://github.com/apache/pinot/issues/10393
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10393: Helix metrics missing from 0.11 version
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10393:
URL: https://github.com/apache/pinot/issues/10393#issuecomment-1459093432
@jackjlli Can you help take a look at this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] qqu0127 commented on issue #10393: Helix metrics missing from 0.11 version
Posted by "qqu0127 (via GitHub)" <gi...@apache.org>.
qqu0127 commented on issue #10393:
URL: https://github.com/apache/pinot/issues/10393#issuecomment-1472783492
Helix community is aware of this and investigation is in progress https://github.com/apache/helix/issues/2406
We verified that for pinot-server running helix 1.0.4, the metrics are emitted to the mbean and JMX under `Domain: CLMParticipantReport`. This is also the case for a few other helix applications. An example mbean is `CLMParticipantReport:ParticipantName=<instance_name>,MonitorType=ParticipantMessageMonitor`
Could you please check the mbeans in your application? It could be that your monitoring system depends on a specific implementation that got changed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org