[GitHub] [arrow-datafusion] liukun4515 commented on issue #1273: Question: Is the Ballista project providing value to the overall DataFusion project?

[GitBox <gi...@apache.org>]

liukun4515 commented on issue #1273:
URL: https://github.com/apache/arrow-datafusion/issues/1273#issuecomment-1014068949


   > I recently started doing some work around DataFusion and seeing good potential with Ballista to become a candidate of our next-gen analytics engine. At this point I am trying to put in some missing pieces and get it working with some benchmark cases we are using for initial validation. If all goes well the plan is to start a team to help drive this forward. I am really interested in working with anyone who share similar interest to kick start the discussion. Please let me know if there are interests and we can use this thread the form an initial group. @andygrove it would be really helpful if we can pick your brain from time to time, and if you have time maybe share some of the things you were thinking to add.
   > 
   > My personal opinion is that Ballista makes sense to be part of DataFusion. It may potentially affect (and depend on) many things such as partition strategy, join strategy, planner, etc. I recognize there might be some overhead in development, maybe this is an opportunity for us to define some APIs to make it easier. @alamb @houqp Do you guys think this is reasonable?
   > 
   > @rdettai I am hoping you are still interested in this topic. Do you think the challenges you had can be solved by introducing an API layer between DataFusion and Ballista and we make some effort to keep it somewhat stable?
   > 
   > I am working on a high level proposal right now and hopefully can share in the next few weeks. Please feel free to let me know if you have any ideas, I am currently working on this besides my full time job so may need a little time.
   
   look forward to your proposal.
   If you need help or discussion, please feel free to let me know.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org