[GitHub] [arrow-datafusion] alamb commented on a diff in pull request #4878: Fix column indices in EnforceDistribution optimizer in Partial AggregateMode

[GitBox <gi...@apache.org>]

alamb commented on code in PR #4878:
URL: https://github.com/apache/arrow-datafusion/pull/4878#discussion_r1068729877


##########
datafusion/core/src/physical_optimizer/dist_enforcement.rs:
##########
@@ -431,9 +431,22 @@ fn reorder_aggregate_keys(
                     None
                 };
                 if let Some(partial_agg) = new_partial_agg {
+                    // Build new group expressions that correspond to the schema of partial_agg
                     let mut new_group_exprs = vec![];
-                    for idx in positions.into_iter() {
-                        new_group_exprs.push(group_by.expr()[idx].clone());
+                    for (idx, position) in positions.into_iter().enumerate() {
+                        let (group_expr, name) = group_by.expr()[position].clone();
+
+                        // Update the index of columns so that they correspond to the schema of partial_agg
+                        // rather than input_schema.
+                        let group_expr = if let Some(group_col) =
+                            group_expr.as_any().downcast_ref::<Column>()
+                        {
+                            Arc::new(Column::new(group_col.name(), idx))
+                                as Arc<dyn PhysicalExpr>

Review Comment:
   I think you can do something like this to get the rust compiler to do the right cast for you:
   
   ```suggestion
                                   as _
   ```
   
   However the rest of this module is in the same `as Arc<...>` style so no need to change it in this PR



##########
datafusion/core/tests/sql/joins.rs:
##########
@@ -2810,3 +2810,61 @@ async fn type_coercion_join_with_filter_and_equi_expr() -> Result<()> {
 
     Ok(())
 }
+
+#[tokio::test]
+async fn test_cross_join_to_groupby_with_different_key_ordering() -> Result<()> {

Review Comment:
   I verified that this test does indeed panic without the test code
   
   ```
   ---- sql::joins::test_cross_join_to_groupby_with_different_key_ordering stdout ----
   thread 'sql::joins::test_cross_join_to_groupby_with_different_key_ordering' panicked at 'called `Option::unwrap()` on a `None` value', datafusion/core/src/physical_plan/joins/hash_join.rs:923:17
   stack backtrace:
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org