You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "Jackie-Jiang (via GitHub)" <gi...@apache.org> on 2024/02/28 00:31:59 UTC
[I] Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server [pinot]
Jackie-Jiang opened a new issue, #12508:
URL: https://github.com/apache/pinot/issues/12508
This is a followup on the discussion in #10137
When `realtime.segment.flush.threshold.rows` is used, the flush threshold for the new CONSUMING segment is determined by both this value and the max partitions consumed by any server. This is not very straight forward, and rebalancing a table could cause new consuming segment size to change.
In order to tackle this problem, we may add a new config (proposing `realtime.segment.flush.threshold.segment.rows`) which works independent of partitions consumed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server [pinot]
Posted by "suyashpatel98 (via GitHub)" <gi...@apache.org>.
suyashpatel98 commented on issue #12508:
URL: https://github.com/apache/pinot/issues/12508#issuecomment-1968467279
@Jackie-Jiang Can you please assign this to me? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server [pinot]
Posted by "suyashpatel98 (via GitHub)" <gi...@apache.org>.
suyashpatel98 commented on issue #12508:
URL: https://github.com/apache/pinot/issues/12508#issuecomment-1991947887
@karthi07 Please take up the issue. I am slightly busy - I will find something else to do when I find time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server [pinot]
Posted by "karthi07 (via GitHub)" <gi...@apache.org>.
karthi07 commented on issue #12508:
URL: https://github.com/apache/pinot/issues/12508#issuecomment-1991683231
@suyashpatel98 are you still working on this task? if not, I would like to take up this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang closed issue #12508: Add a config similar to `realtime.segment.flush.threshold.rows` but independent of partitions consumed by server
URL: https://github.com/apache/pinot/issues/12508
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org