[GitHub] [arrow-datafusion] Jefffrey commented on a diff in pull request #5612: Miscellaneous ArrayData Cleanup

["Jefffrey (via GitHub)" <gi...@apache.org>]

Jefffrey commented on code in PR #5612:
URL: https://github.com/apache/arrow-datafusion/pull/5612#discussion_r1137606275


##########
datafusion/core/src/dataframe.rs:
##########
@@ -627,22 +627,12 @@ impl DataFrame {
     /// # }
     /// ```
     pub async fn count(self) -> Result<usize> {
-        let rows = self
-            .aggregate(
-                vec![],
-                vec![datafusion_expr::count(Expr::Literal(ScalarValue::Null))],
-            )?
-            .collect()
-            .await?;
-        let len = *rows
-            .first()
-            .and_then(|r| r.columns().first())
-            .and_then(|c| c.as_any().downcast_ref::<Int64Array>())
-            .and_then(|a| a.values().first())
-            .ok_or(DataFusionError::Internal(
-                "Unexpected output when collecting for count()".to_string(),
-            ))? as usize;
-        Ok(len)
+        let mut stream = self.execute_stream().await?;

Review Comment:
   You could keep the old behaviour and use `datafusion_exr::utils::COUNT_STAR_EXPANSION` instead of the `ScalarValue::Null`, since `count(*)` is expanded to that in SQL planner (related bug https://github.com/apache/arrow-datafusion/pull/5518 prevents directly using `Count(Expr:Wildcard)` at the moment)
   
   The reason I had it that way was to let the count optimize out any column projections to only the minimal needed, which I think theoretically is faster than just executing the plan as is and counting the rows



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org