[GitHub] [arrow] westonpace commented on a change in pull request #11707: [WIP] Substrait consumer

[GitBox <gi...@apache.org>]

westonpace commented on a change in pull request #11707:
URL: https://github.com/apache/arrow/pull/11707#discussion_r781720167



##########
File path: cpp/src/arrow/CMakeLists.txt
##########
@@ -712,6 +712,10 @@ if(ARROW_COMPUTE)
   add_subdirectory(compute)
 endif()
 
+if(ARROW_ENGINE)

Review comment:
       I believe `ARROW_ENGINE` may also someday depend on datasets, ipc, parquet, and maybe even flight.  For example, a substrait plan will generally start with a scan node (datasets) and the engine may need to use spillover (ipc / parquet) and we might want to send data to or receive data from flight nodes.
   
   Some of this we could probably avoid using more indirection (e.g. substrait consumer defines a "table provider" and the user can use the "datasets table provider" to link the two modules) but to start with it might be easier to just do the simple thing.
   
   Either way, I think we will eventually want a standalone engine module that isn't really just a child of compute so I'm kind of in favor of it being a peer (and not a child) of compute.
   
   See: ARROW-15238 (which this PR satisfies)




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org