You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "snleee (via GitHub)" <gi...@apache.org> on 2023/02/17 18:06:48 UTC
[GitHub] [pinot] snleee opened a new issue, #10300: Combine value aggregator
snleee opened a new issue, #10300:
URL: https://github.com/apache/pinot/issues/10300
We have 2 value aggregator interfaces:
1. `org.apache.pinot.core.segment.processing.aggregator.ValueAggregator`: Used in the segment processing framework (mostly used by minion tasks)
2. `org.apache.pinot.segment.local.aggregator.ValueAggregator`: Used by startree generation/aggregation config.
We can combine these aggregators to have the feature parity among all aggregation related features (merge/rollup, startree index, realtime aggregate metrics).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] swaminathanmanish commented on issue #10300: Combine value aggregator interfaces
Posted by "swaminathanmanish (via GitHub)" <gi...@apache.org>.
swaminathanmanish commented on issue #10300:
URL: https://github.com/apache/pinot/issues/10300#issuecomment-1470638099
I'll take this up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on issue #10300: Combine value aggregator interfaces
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #10300:
URL: https://github.com/apache/pinot/issues/10300#issuecomment-1472516778
For some context, the contract over these 2 interfaces are different:
- For the segment processing framework (roll-up), the input and output type for the aggregation must be the same because the values are stored back to the original column
- For the star-tree creation, the aggregated values are stored in a separate column, thus it is not required to have input and output the same type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] swaminathanmanish commented on issue #10300: Combine value aggregator interfaces
Posted by "swaminathanmanish (via GitHub)" <gi...@apache.org>.
swaminathanmanish commented on issue #10300:
URL: https://github.com/apache/pinot/issues/10300#issuecomment-1474270240
Thanks @Jackie-Jiang for the context. I think having 2 separate interfaces makes this contract clear (even though input/output types for the rollup is an Object). A common shared library that encapsulates core aggregation logic can help make the 2 aggregations consistent, but not clear about value prop of doing this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org