You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2024/02/09 01:05:41 UTC
[I] [Multi-stage] Stage stats are very misleading [pinot]
Jackie-Jiang opened a new issue, #12388:
URL: https://github.com/apache/pinot/issues/12388
Currently the stage stats reported in the query response is the aggregated stats over all the operators, which can be quite confusing.
When tracing is disabled, we collect stats for leaf and send operator, and reporting doubled `numDocs` and `numBlocks` than actually sent by the leaf stage.
When tracing is enabled, we collect stats for every operator, and aggregate `numDocs` and `numBlocks` for all the operators.
Instead, we might want to just record the stats for blocks sent out from each stage. We should also revisit other stats, and see if it makes sense to aggregate them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [Multi-stage] Stage stats are very misleading [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #12388:
URL: https://github.com/apache/pinot/issues/12388#issuecomment-1945206587
By default, we collect stats for leaf operator and send operator. The docs gathered for each stage is the aggregated value. IMO we should only count the rows sent by the send operator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [Multi-stage] Stage stats are very misleading [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #12388:
URL: https://github.com/apache/pinot/issues/12388#issuecomment-1935173927
cc @gortiz @KKcorps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [Multi-stage] Stage stats are very misleading [pinot]
Posted by "KKcorps (via GitHub)" <gi...@apache.org>.
KKcorps commented on issue #12388:
URL: https://github.com/apache/pinot/issues/12388#issuecomment-1935178397
That is weird. What's causing the doubling of leaf data? iirc, the stats are only sent when EOS block is received
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [Multi-stage] Stage stats are very misleading [pinot]
Posted by "KKcorps (via GitHub)" <gi...@apache.org>.
KKcorps commented on issue #12388:
URL: https://github.com/apache/pinot/issues/12388#issuecomment-1946184142
Ohhh, iirc, I did that to ensure that all the existing stats for V1 engine are also available for each table. I must missed the tracing duplication part. Assigning this to myself
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org