You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "Smallfu666 (via GitHub)" <gi...@apache.org> on 2026/07/30 10:20:30 UTC
[PR] [GLUTEN-12094][VL] Strip default array_sort comparator for Velox offloading [gluten]
Smallfu666 opened a new pull request, #12661:
URL: https://github.com/apache/gluten/pull/12661
### What changes
Spark represents `array_sort(array)` using a generated null-aware default comparator. Gluten currently passes this comparator to Velox as a lambda, but Velox cannot rewrite it and falls back to Spark execution.
This change detects Spark's default comparator using the bound lambda arguments and omits the lambda only for an exact semantic match (`semanticEquals`), allowing Velox to use its native one-argument `array_sort`.
Custom comparators remain unchanged and continue using the two-argument path.
This revives and revalidates the approach from #12095, which was auto-closed as stale.
Fixes #12094
### Tests
- Spark 4.0.2 targeted native test passed (`MiscOperatorSuite - "array_sort with default comparator"`; full suite blocked by unrelated Velox HashTable SIGFPE)
- Spark 4.1.1 targeted native test passed (`MiscOperatorSuite - "array_sort with default comparator"`, 1/1 PASS, BUILD SUCCESS)
- Spark 3.5 clean compile and test-compile passed (`./build/mvn -ntp clean test-compile -Pspark-3.5 -Pscala-2.12 -Pjava-17 -Pbackends-velox -Pspark-ut`)
- Test coverage: integer arrays, null elements, empty array, null array, double arrays with NaN, descending custom-comparator negative control
- Spark oracle comparison and `ProjectExecTransformer` plan assertion verified
### AI Tooling Disclosure
Was this patch authored or co-authored using generative AI tooling?
Generated-by: Gemini 3.6 Flash
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12094][VL] Strip default array_sort comparator for Velox offloading [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12661:
URL: https://github.com/apache/gluten/pull/12661#issuecomment-5657882265
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12094][VL] Strip default array_sort comparator for Velox offloading [gluten]
Posted by "rdtr (via GitHub)" <gi...@apache.org>.
rdtr commented on PR #12661:
URL: https://github.com/apache/gluten/pull/12661#issuecomment-5967893226
@Smallfu666 Since this was closed, I've taken it over in #13189, rebased onto the current main with you and @surendralilhore credited as co-authors. Thanks for the original work!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12094][VL] Strip default array_sort comparator for Velox offloading [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #12661:
URL: https://github.com/apache/gluten/pull/12661#issuecomment-5825264659
This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [GLUTEN-12094][VL] Strip default array_sort comparator for Velox offloading [gluten]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #12661: [GLUTEN-12094][VL] Strip default array_sort comparator for Velox offloading
URL: https://github.com/apache/gluten/pull/12661
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org