You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@auron.apache.org by "Sigma-Ma (via GitHub)" <gi...@apache.org> on 2026/10/12 14:57:41 UTC
[I] Support arrays_overlap natively [auron]
Sigma-Ma opened a new issue, #2576:
URL: https://github.com/apache/auron/issues/2576
**Is your feature request related to a problem? Please describe.**
Spark's `arrays_overlap` expression currently relies on expression fallback in Auron.
**Describe the solution you'd like**
Add native support for `arrays_overlap(a1, a2)` with Spark-compatible behavior:
- Return true when the arrays share a non-null element.
- Return null if either input array is null.
- When there is no match, return null if both arrays are non-empty and either contains a null element.
- Return false otherwise, including when either non-null array is empty.
Element comparisons should follow Spark semantics, including NaN and signed zero. Unsupported element types should retain the existing fallback.
Add Rust unit tests and Spark compatibility coverage for matches, disjoint arrays, empty arrays, nulls, and floating-point values. Use column inputs in Spark tests and check that the expression executes natively.
**Describe alternatives you've considered**
Continue evaluating the expression through Spark fallback.
**Additional context**
Spark function reference:
https://spark.apache.org/docs/3.5.7/api/sql/index.html#arrays_overlap
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@auron.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org