You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "jackylee-ch (via GitHub)" <gi...@apache.org> on 2026/09/03 07:27:57 UTC

[PR] [spark] Use the field index instead of the field id for TopN pushdown [paimon]

jackylee-ch opened a new pull request, #9566:
URL: https://github.com/apache/paimon/pull/9566

   ### Purpose
   
   `PaimonScanBuilder` builds the pushed `SortValue` with `new FieldRef(field.id(), ...)`, but the first argument is a field *index*: the only consumer, `TopNDataSplitEvaluator`, does `schema.fields().get(order.field().index())`. `DROP COLUMN` does not reassign field ids, so after one they no longer agree and the evaluator reads another column's statistics, or throws when the id is past the field count.
   
   ### Tests
   
   `PaimonPushDownTestBase`, "TopN for append-only tables after dropping a column". Without the fix it fails with `IndexOutOfBoundsException: Index 3 out of bounds for length 3` at `TopNDataSplitEvaluator:64`.
   
   Written with Claude Code; reasoning and verification are mine.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Use the field index instead of the field id for TopN pushdown [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #9566:
URL: https://github.com/apache/paimon/pull/9566#issuecomment-5522845142

   +1


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [spark] Use the field index instead of the field id for TopN pushdown [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #9566:
URL: https://github.com/apache/paimon/pull/9566


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org