You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2024/02/15 01:17:32 UTC
[I] [Multi-stage] OpChain stats handling improvement [pinot]
Jackie-Jiang opened a new issue, #12426:
URL: https://github.com/apache/pinot/issues/12426
- OpChain stats is a mapping from worker id to serialized map, then serialized as JSON bytes. It is very inefficient. Since we know the metadata key upfront, we can encode the metadata much more efficient. See `DataTable` for example of how v1 handles query stats
- When tracing is not enabled, no need to send per worker stats. We should aggregate the stats for each stage when finishing the stage, instead of sending stats for each worker
- EOS block containing the stats are duplicated to all workers. Sending the stats to only one worker should be enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [Multi-stage] OpChain stats handling improvement [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #12426:
URL: https://github.com/apache/pinot/issues/12426#issuecomment-1945211294
cc @gortiz @KKcorps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org