You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2020/10/13 04:29:28 UTC

[GitHub] [arrow] jorgecarleitao opened a new pull request #8452: ARROW-10293: [Rust] [DataFusion] Fixed benchmarks

jorgecarleitao opened a new pull request #8452:
URL: https://github.com/apache/arrow/pull/8452


   The benchmarks were only benchmarking planning, not execution, of the plans. This PR fixes this.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] github-actions[bot] commented on pull request #8452: ARROW-10293: [Rust] [DataFusion] Fixed benchmarks

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #8452:
URL: https://github.com/apache/arrow/pull/8452#issuecomment-707481548


   https://issues.apache.org/jira/browse/ARROW-10293


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorgecarleitao commented on a change in pull request #8452: ARROW-10293: [Rust] [DataFusion] Fixed benchmarks

Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on a change in pull request #8452:
URL: https://github.com/apache/arrow/pull/8452#discussion_r503853695



##########
File path: rust/datafusion/benches/aggregate_query_sql.rs
##########
@@ -38,13 +39,12 @@ use datafusion::datasource::MemTable;
 use datafusion::error::Result;
 use datafusion::execution::context::ExecutionContext;
 
-async fn query(ctx: Arc<Mutex<ExecutionContext>>, sql: &str) {
+fn query(ctx: Arc<Mutex<ExecutionContext>>, sql: &str) {
+    let mut rt = Runtime::new().unwrap();
+
     // execute the query
     let df = ctx.lock().unwrap().sql(&sql).unwrap();
-    let results = df.collect().await.unwrap();
-
-    // display the relation
-    for _batch in results {}
+    rt.block_on(df.collect()).unwrap();

Review comment:
       yes, the block is pick up the task. We also benchmark creating a runtime itself, but that was a very small compared to the task itself, so I left it there.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] alamb commented on a change in pull request #8452: ARROW-10293: [Rust] [DataFusion] Fixed benchmarks

Posted by GitBox <gi...@apache.org>.
alamb commented on a change in pull request #8452:
URL: https://github.com/apache/arrow/pull/8452#discussion_r503831762



##########
File path: rust/datafusion/benches/aggregate_query_sql.rs
##########
@@ -38,13 +39,12 @@ use datafusion::datasource::MemTable;
 use datafusion::error::Result;
 use datafusion::execution::context::ExecutionContext;
 
-async fn query(ctx: Arc<Mutex<ExecutionContext>>, sql: &str) {
+fn query(ctx: Arc<Mutex<ExecutionContext>>, sql: &str) {
+    let mut rt = Runtime::new().unwrap();
+
     // execute the query
     let df = ctx.lock().unwrap().sql(&sql).unwrap();
-    let results = df.collect().await.unwrap();
-
-    // display the relation
-    for _batch in results {}
+    rt.block_on(df.collect()).unwrap();

Review comment:
       👍  this is the major change, right?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] andygrove closed pull request #8452: ARROW-10293: [Rust] [DataFusion] Fixed benchmarks

Posted by GitBox <gi...@apache.org>.
andygrove closed pull request #8452:
URL: https://github.com/apache/arrow/pull/8452


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] jorgecarleitao commented on pull request #8452: ARROW-10293: [Rust] [DataFusion] Fixed benchmarks

Posted by GitBox <gi...@apache.org>.
jorgecarleitao commented on pull request #8452:
URL: https://github.com/apache/arrow/pull/8452#issuecomment-707478913


   FYI @andygrove , as 2.0.0' benchmarks not benchmarking anything atm :/


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org