's getChildren response size [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on PR #15345:
URL: https://github.com/apache/pinot/pull/15345#issuecomment-2750282531
> > One concern is that constantly reading the segment list of a large table itself may put pressure on the controller and zookeeper. Is it possible to track this metric when segments are added or removed ? A daily periodic job can refresh the absolute value if required. Another suggestion is to add a config param as getting the list is a heavy operation.
>
> From reading the code, it looks like ZooKeeper access via the property store’s getChildrenNames() method is already cached ([ref](https://sourcegraph.com/github.com/apache/helix/-/blob/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java?L682)). Also, segment status checkers run every 5 minutes by default. Let me know if this addresses your concern.
Got it. Didnt realize this is iterating through an in-memory cached list
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Metrics for monitoring PROPERTYSTORE/SEGMENTS/
's getChildren response size [pinot]
Posted by "rhodo (via GitHub)" <gi...@apache.org>.
rhodo commented on PR #15345:
URL: https://github.com/apache/pinot/pull/15345#issuecomment-2749267004
> One concern is that constantly reading the segment list of a large table itself may put pressure on the controller and zookeeper. Is it possible to track this metric when segments are added or removed ? A daily periodic job can refresh the absolute value if required. Another suggestion is to add a config param as getting the list is a heavy operation.
From reading the code, it looks like ZooKeeper access via the property store’s getChildrenNames() method is already cached ([ref](https://sourcegraph.com/github.com/apache/helix/-/blob/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java?L682)). Also, segment status checkers run every 5 minutes by default. Let me know if this addresses your concern.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Metrics for monitoring PROPERTYSTORE/SEGMENTS/
's getChildren response size [pinot]
Posted by "vrajat (via GitHub)" <gi...@apache.org>.
vrajat commented on PR #15345:
URL: https://github.com/apache/pinot/pull/15345#issuecomment-2747816587
One concern is that constantly reading the segment list of a large table itself may put pressure on the controller and zookeeper. Is it possible to track this metric when segments are added or removed ? A daily periodic job can refresh the absolute value if required.
Another suggestion is to add a config param as getting the list is a heavy operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Metrics for monitoring PROPERTYSTORE/SEGMENTS/
's getChildren response size [pinot]
Posted by "9aman (via GitHub)" <gi...@apache.org>.
9aman commented on PR #15345:
URL: https://github.com/apache/pinot/pull/15345#issuecomment-2756672665
@rhodo
I understand this PR has already been merged but would like to understand following
1. What is the rationale of adding this metric. Where would we need this ?
2. The constant name PROPERTYSTORE_SEGMENT_CHILDREN_BYTE_SIZE gives a perception that it's the size of the child node and not just the segment name sizes. Can we update the name to maybe `PROPERTYSTORE_SEGMENT_CHILDREN_NAME_BYTE_SIZE` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Metrics for monitoring PROPERTYSTORE/SEGMENTS/
's getChildren response size [pinot]
Posted by "rhodo (via GitHub)" <gi...@apache.org>.
rhodo commented on PR #15345:
URL: https://github.com/apache/pinot/pull/15345#issuecomment-2752666256
> Can you fix the tests in `SegmentStatusCheckerTest`?
Sure, just fixed. It is failing because propertyStore can be null in SegmentStatusCheckerTest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [PR] Metrics for monitoring PROPERTYSTORE/SEGMENTS/
's getChildren response size [pinot]
Posted by "rhodo (via GitHub)" <gi...@apache.org>.
rhodo commented on PR #15345:
URL: https://github.com/apache/pinot/pull/15345#issuecomment-2759774679
> @rhodo I understand this PR has already been merged but would like to understand following
>
> 1. What is the rationale of adding this metric. Where would we need this ?
> 2. The constant name PROPERTYSTORE_SEGMENT_CHILDREN_BYTE_SIZE gives a perception that it's the size of the child node and not just the segment name sizes. Can we update the name to maybe `PROPERTYSTORE_SEGMENT_CHILDREN_NAME_BYTE_SIZE` ?
The rationale is that when the number of segments becomes very large, the getChildren() call could fail or impact availability if the returned list exceeds ZooKeeper's buffer size. This metric helps us monitor and stay ahead of that risk.
I don’t have a strong opinion on the naming. I was following the existing convention (e.g., IDEALSTATE_ZNODE_BYTE_SIZE for node size metrics), so I thought PROPERTYSTORE_SEGMENT_ZNODE_BYTE_SIZE might be appropriate if we're tracking znode size. But if you feel strongly about renaming it to something like PROPERTYSTORE_SEGMENT_CHILDREN_NAME_BYTE_SIZE, I’m happy to make that change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org