[GitHub] [pinot] siddharthteotia commented on issue #8862: Add server api/metrics to track the size of each index in a table

[GitBox <gi...@apache.org>]

siddharthteotia commented on issue #8862:
URL: https://github.com/apache/pinot/issues/8862#issuecomment-1301394396

   On a related note, do we think we can expose such details via SQL which is typically done by DBMSes as this metadata information is also stored in tables (under SYS user) in many systems.
   
   In our case, this information is in per segment flat file but in theory we can try to expose this via SQL and that can be handy for debugging imo. Examples (which I have often done via some CLI script in our production when analyzing issues)....
   
   - Find the segment name with largest forward index for a particular column
   - Find the top 10 segments by size  or by forward index size. 
   - Find the segment name with highest cardinality for a particular column.
   
   API based is also fine I guess. 
   
   Is the goal here to expose all this for some debugging ? From the description, it looks like the suggestion is to move information tracked in `index_map` file into `metadata.properties` file. Curious, what is the benefit ?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org