[GitHub] [spark] viirya commented on issue #24515: [SPARK-14083][WIP] Basic bytecode analyzer to speed up Datasets

[GitBox <gi...@apache.org>]

viirya commented on issue #24515: [SPARK-14083][WIP] Basic bytecode analyzer to speed up Datasets
URL: https://github.com/apache/spark/pull/24515#issuecomment-489281708
 
 
   Thanks for your work!
   
   @rednaxelafx talked about several questions that I think most people are interested to know. For simple typed Dataset operations, it is quite easily to use untyped DataFrame operations instead. But for complex typed Dataset operations, I think it might be more challenging to write uses of the typed Dataset operations to the equivalent untyped DataFrame operations. However, it might be where bytecode analyzer can be really valuable as it automatically converts the operations written in host language to untyped DataFrame operations.
   
   So I'm interested to know that what kind of code can be safely analyzed for speeding up, and what kind of code mightn't be.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org