You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@auron.apache.org by "lyne7-sc (via GitHub)" <gi...@apache.org> on 2026/09/04 07:28:12 UTC

[I] [Performance] Optimize partial WindowGroupLimit execution [auron]

lyne7-sc opened a new issue, #2507:
URL: https://github.com/apache/auron/issues/2507

   ## Problem
   
   For partial `WindowGroupLimitExec`, where the ranking column is not included in the output, `WindowExec` still creates a regular window processor and materializes a complete ranking array for all input rows.
   
   It then builds a Boolean selection from the ranking array, filters both the ranking array and the input batch, and finally discards the filtered ranking array. This introduces unnecessary allocation and filtering for `row_number`, `rank`, and `dense_rank`.
   
   The selected rows also commonly form contiguous ranges, but the current implementation always uses Arrow filtering.
   
   ## Proposed solution
   
   Add a specialized processor for partial `WindowGroupLimitExec` that tracks partition, ordering, and ranking state across input batches and directly produces selected row ranges.
   
   Use zero-copy `RecordBatch::slice` when the selected rows form one contiguous range. For multiple ranges, build the Boolean selection in blocks and use Arrow filtering.
   
   Keep the existing execution path unchanged when window columns are required in the output.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org