[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #5308: Add SortExec input cases to bench for SortPreservingMergeExec

["alamb (via GitHub)" <gi...@apache.org>]

alamb commented on code in PR #5308:
URL: https://github.com/apache/arrow-datafusion/pull/5308#discussion_r1108929173


##########
datafusion/core/benches/merge.rs:
##########
@@ -214,6 +266,26 @@ impl MergeBenchCase {
         }
     }
 
+    fn new_with_sort_input(partitions: &[Vec<RecordBatch>]) -> Self {
+        let runtime = tokio::runtime::Builder::new_multi_thread().build().unwrap();
+        let session_ctx = SessionContext::new();
+        let task_ctx = session_ctx.task_ctx();
+
+        let schema = partitions[0][0].schema();
+        let sort = make_sort_exprs(schema.as_ref());
+
+        let projection = None;
+        let exec = Arc::new(MemoryExec::try_new(partitions, schema, projection).unwrap());

Review Comment:
   I double checked that the sort exec does actually make multiple output streams. 👍 
   
   FYI @mustafasrepo  and @ozankabak and @mingmwang   -- not sure if you are following this work from @jaylmiller but I think his work may make repartitioned sorts (in parallel) a more compelling option



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org