[GitHub] [druid] jihoonson opened a new issue #11252: Checking maxRowsPerSegment after segment merge in dynamic partitioning

[GitBox <gi...@apache.org>]

jihoonson opened a new issue #11252:
URL: https://github.com/apache/druid/issues/11252


   ### Description
   
   In this issue, the segment size refers to the row count in the segment. In dynamic partitioning, the task compares `maxRowsPerSegment` and the current `Sink` size to determine whether the segment hits the max segment size limit and needs to be pushed to deep storage. Here, the `Sink` size is the sum of in-memory segment size and the size of all persisted segments _before_ they are merged. As a result, when the rollup is on, the `Sink` size can be larger than the actual segment size. This can create smaller segments than `maxRowsPerSegment` unnecessarily.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org