[GitHub] [arrow-datafusion] alamb commented on issue #1273: Question: Is the Ballista project providing value to the overall DataFusion project?

[GitBox <gi...@apache.org>]

alamb commented on issue #1273:
URL: https://github.com/apache/arrow-datafusion/issues/1273#issuecomment-1014661089


   > My personal opinion is that Ballista makes sense to be part of DataFusion. It may potentially affect (and depend on) many things such as partition strategy, join strategy, planner, etc. I recognize there might be some overhead in development, maybe this is an opportunity for us to define some APIs to make it easier. @alamb @houqp Do you guys think this is reasonable?
   
   I think this is very reasonable. I view `datafusion` as a single node, sharable query engine that isn't really designed to be used by itself -- rather it can be used to create systems such as ballista (and [other projects](https://github.com/apache/arrow-datafusion#known-uses))
   
   Having ballista in the same repo as a way to validate datafusion API changes seems valuable to me 👍  and I am happy to have some minor extra overhead on DataFusion API changes. What I really want to see is Ballista actively used / driven forward by the community, which it sounds like @realno  is preparing to propose. 
   
   cc @yahoNanJing and @gaojun2048 who have opened PRs with contributions for Ballista recently 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org