You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/10/21 13:32:00 UTC

[GitHub] [spark] LucaCanali commented on pull request #26783: [SPARK-30153][PYTHON][WIP] Extend data exchange options for vectorized UDF functions with vanilla Arrow serialization

LucaCanali commented on pull request #26783:
URL: https://github.com/apache/spark/pull/26783#issuecomment-948621085


   Thanks @HyukjinKwon for following up on this.
   We need something that produces pyarrow Arrays or Tables, then Awkward Array can view them in a zero-copy way.
   What we want to avoid is the extra conversion to pandas, that is not needed for our use case. Something like "mapInArrow" should work too.
     
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org