[GitHub] [pinot] ankitsultana commented on pull request #10937: New API to get tenant tags

["ankitsultana (via GitHub)" <gi...@apache.org>]

ankitsultana commented on PR #10937:
URL: https://github.com/apache/pinot/pull/10937#issuecomment-1605850109

   > @npuppala111 @ankitsultana the instance tag can be retrieved from Pinot controller using the controller's instance api. The tag is stored in Zookeeper and the Pinot controller is usually the right service to got the tag info. I wonder why we can not get these info from the controller?
   
   For some of our observability requirements we need to retrieve tenant tags at instance level (for both brokers and servers). This is done by a separate thread in the JVM which *acts* as a sidecar (it's still within the same process as the instance so not exactly a sidecar per usual definition).
   
   To do this we have the following options:
   
   1. **Hit Controller HTTP API**: This would require that the observability layer knows how to discover the controller for a given instance. It also adds a criss-cross dependency: observability for a given process relies on hitting another component (controller). It also increases controller HTTP load quite a bit as the number of instances increase.
   2. **Use ZK or Helix**: Using ZK directly is not clean since we would have to leak Pinot internal details about where the instance configs are stored to our sidecar. Using Helix directly is probably the closest to what we would be doing with this HTTP API approach as well, but I think the HTTP API approach is cleaner: the observability layer shouldn't need to know about Helix or other details, and this API can also be used generally in true sidecars in the future by not just us but even other users outside Uber.
   
   Given the above I think it's best to have a separate API.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org