You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/08/31 12:56:44 UTC
[GitHub] [arrow] alamb opened a new pull request #8083: ARROW-9889: [Rust][DataFusion] Implement physical plan for EmptyRelation
alamb opened a new pull request #8083:
URL: https://github.com/apache/arrow/pull/8083
This fixes an error I saw in the Datafusion CLI. Running `CREATE EXTERNAL TABLE` errors with `"Unsupported logical plan variant"`
Before this PR, you can't make an external table via the CLI:
{code}
> create external table test(c1 boolean) stored as CSV location '/tmp/foo';
General("Unsupported logical plan variant")
>
{code}
After this PR:
{code}
> create external table test(c1 boolean) stored as CSV location '/tmp/foo';
0 rows in set. Query took 0 seconds.
>
{code}
Other changes:
* Made the list of unsupported plan nodes explicit in the planer
* Added a test for this behavior.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] andygrove closed pull request #8083: ARROW-9889: [Rust][DataFusion] Implement physical plan for EmptyRelation
Posted by GitBox <gi...@apache.org>.
andygrove closed pull request #8083:
URL: https://github.com/apache/arrow/pull/8083
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on pull request #8083: ARROW-9889: [Rust][DataFusion] Implement physical plan for EmptyRelation
Posted by GitBox <gi...@apache.org>.
alamb commented on pull request #8083:
URL: https://github.com/apache/arrow/pull/8083#issuecomment-684743567
Rebased
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8083: ARROW-9889: [Rust][DataFusion] Implement physical plan for EmptyRelation
Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on a change in pull request #8083:
URL: https://github.com/apache/arrow/pull/8083#discussion_r480255476
##########
File path: rust/datafusion/src/execution/physical_plan/planner.rs
##########
@@ -318,12 +330,9 @@ impl DefaultPhysicalPlanner {
format!("{:#?}", input),
));
}
- let schema_ref = Arc::new((**schema).clone());
+ let schema_ref = Arc::new(schema.as_ref().clone());
Review comment:
I prefer to use idiomatic [language being written]. These changes are very welcoming from my side.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] github-actions[bot] commented on pull request #8083: ARROW-9889: [Rust][DataFusion] Implement physical plan for EmptyRelation
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8083:
URL: https://github.com/apache/arrow/pull/8083#issuecomment-683763591
https://issues.apache.org/jira/browse/ARROW-9889
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow] alamb commented on a change in pull request #8083: ARROW-9889: [Rust][DataFusion] Implement physical plan for EmptyRelation
Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8083:
URL: https://github.com/apache/arrow/pull/8083#discussion_r480111822
##########
File path: rust/datafusion/src/execution/physical_plan/planner.rs
##########
@@ -278,6 +278,9 @@ impl DefaultPhysicalPlanner {
ctx_state.config.concurrency,
)?))
}
+ LogicalPlan::EmptyRelation { schema } => {
Review comment:
this is the actual fix
##########
File path: rust/datafusion/src/execution/physical_plan/planner.rs
##########
@@ -318,12 +330,9 @@ impl DefaultPhysicalPlanner {
format!("{:#?}", input),
));
}
- let schema_ref = Arc::new((**schema).clone());
+ let schema_ref = Arc::new(schema.as_ref().clone());
Review comment:
this `**` bothered me, so I switched to the different syntax. It has no intended behavior change
##########
File path: rust/datafusion/src/execution/physical_plan/explain.rs
##########
@@ -44,7 +44,7 @@ pub struct ExplainExec {
}
impl ExplainExec {
- /// Create a new MergeExec
+ /// Create a new ExplainExec
Review comment:
I snuck this comment fix in -- I can remove it if people prefer.
##########
File path: rust/datafusion/tests/sql.rs
##########
@@ -517,9 +518,17 @@ fn register_aggregate_csv_by_sql(ctx: &mut ExecutionContext) {
WITH HEADER ROW
LOCATION '{}/csv/aggregate_test_100.csv'
",
- testdata
- ))
- .unwrap();
+ testdata
+ ))
+ .expect("Creating dataframe for CREATE EXTERNAL TABLE");
+
+ // Mimic the CLI and execute the resulting plan -- even though it
+ // is effectively a no-op (returns zero rows)
+ let results = df.collect().expect("Executing CREATE EXTERNAL TABLE");
Review comment:
Prior to the code change, the tests now fail in the same way as reported in ARROW-9889:
```
---- csv_query_create_external_table stdout ----
thread 'csv_query_create_external_table' panicked at 'Executing CREATE EXTERNAL TABLE: General("Unsupported logical plan variant")', datafusion/tests/sql.rs:526:19
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org