Re: [I] Support higher-order array functions via JVM UDF bridge [datafusion-comet]

["dwsmith1983 (via GitHub)" <gi...@apache.org>]

dwsmith1983 commented on issue #4224:
URL: https://github.com/apache/datafusion-comet/issues/4224#issuecomment-5637096046

   This landed in #4618 (7e413357e). `transform`, `exists`, `forall` and `aggregate` are `CometCodegenDispatch` serdes in spark/src/main/scala/org/apache/comet/serde/arrays.scala (`CometArrayTransform`, `CometArrayExists`, `CometArrayForAll`, `CometArrayAggregate`), so the lambda is evaluated by Spark's generated code over Arrow batches inside the Comet pipeline with no C2R/R2C transition, which is the outcome the JVM UDF bridge (#4170) was proposed for. `filter` is handled by `CometArrayFilter` in the same file: the `IsNotNull(x)` shape runs natively as `array_compact` and any other lambda goes through the same dispatcher. Suggest closing.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org