You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by "waynexia (via GitHub)" <gi...@apache.org> on 2024/02/18 15:06:58 UTC

Re: [PR] Improve datafuson-proto and datafusion-substrait docs [arrow-datafusion]

waynexia commented on code in PR #9260:
URL: https://github.com/apache/arrow-datafusion/pull/9260#discussion_r1493784289


##########
datafusion/substrait/src/lib.rs:
##########
@@ -15,6 +15,61 @@
 // specific language governing permissions and limitations
 // under the License.
 
+//! Serialize / Deserialize DataFusion Plans to [Substrait.io]
+//!
+//! This crate provides support for serializing and deserializing DataFusion plans
+//! to and from the generated types in [substrait::proto] from the [substrait] crate.
+//!
+//! [Substrait.io] provides a cross-language serialization format for relational
+//! algebra (e.g. query plans and expressions), based on protocol buffers.
+//!
+//! [Substrait.io]: https://substrait.io/
+//!
+//! [`LogicalPlan`]: datafusion::logical_expr::LogicalPlan
+//! [`ExecutionPlan`]: datafusion::physical_plan::ExecutionPlan
+//!
+//! Potential uses of this crate:
+//! * Use DataFusion run Substrait plans created by other systems (e.g. Apache Calcite)
+//! * Use DataFusion to create plans to run on other systems

Review Comment:
   A famous consumer is DuckDB: https://duckdb.org/docs/extensions/substrait.html



##########
datafusion/substrait/src/lib.rs:
##########
@@ -15,6 +15,61 @@
 // specific language governing permissions and limitations
 // under the License.
 
+//! Serialize / Deserialize DataFusion Plans to [Substrait.io]
+//!
+//! This crate provides support for serializing and deserializing DataFusion plans
+//! to and from the generated types in [substrait::proto] from the [substrait] crate.

Review Comment:
   They seem not to be referenced
   
   ```suggestion
   //! This crate provides support for serializing and deserializing both DataFusion
   //! [`LogicalPlan`] and [`ExecutionPlan`] to and from the generated types in
   //! [substrait::proto] from the [substrait] crate.
   ```



##########
datafusion/substrait/src/lib.rs:
##########
@@ -15,6 +15,61 @@
 // specific language governing permissions and limitations
 // under the License.
 
+//! Serialize / Deserialize DataFusion Plans to [Substrait.io]
+//!
+//! This crate provides support for serializing and deserializing DataFusion plans
+//! to and from the generated types in [substrait::proto] from the [substrait] crate.
+//!
+//! [Substrait.io] provides a cross-language serialization format for relational
+//! algebra (e.g. query plans and expressions), based on protocol buffers.
+//!
+//! [Substrait.io]: https://substrait.io/
+//!
+//! [`LogicalPlan`]: datafusion::logical_expr::LogicalPlan
+//! [`ExecutionPlan`]: datafusion::physical_plan::ExecutionPlan
+//!
+//! Potential uses of this crate:
+//! * Use DataFusion run Substrait plans created by other systems (e.g. Apache Calcite)
+//! * Use DataFusion to create plans to run on other systems
+//! * Pass query plans over FFI boundaries, such as from Python to Rust

Review Comment:
   I'd like to provide another use case from GreptimeDB, where substrait plan is used to pass query plans across compute nodes.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org