You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2024/11/24 13:51:40 UTC

[I] [EPIC] Improved aggregate function performance [datafusion]

alamb opened a new issue, #13548:
URL: https://github.com/apache/datafusion/issues/13548

   ### Is your feature request related to a problem or challenge?
   
   The basic aggregate functions like `COUNT` and `SUM` in DataFusion are *very* fast (see [Apache DataFusion is now the fastest single node engine for querying Apache Parquet files](https://datafusion.apache.org/blog/2024/11/18/datafusion-fastest-single-node-parquet-clickbench/))
   
   However, many of the other aggregate functions are not particularly fast, and this shows up specifically on some of the H20 benchmarks
   
   We saw this in the results in the [2024 DataFusion SIGMOD paper](https://dl.acm.org/doi/10.1145/3626246.3653368)
   ![Screenshot 2024-11-24 at 8 34 35 AM](https://github.com/user-attachments/assets/72338cd9-3b1d-4feb-ae65-29b9c53ac3da)
   
   (BTW we have made median faster)
   
   @MrPowers has also observed similar results:
   > DataFusion was added to the h2o benchmarks (which are now maintained by duckdb) and DataFusion performs quite well for most of the "basic" groupby queries.  It performs poorly for some of the advanced questions on the 50GB dataset.  Here are the results: 
   > https://duckdblabs.github.io/db-benchmark/
   
   See his version of the benchmarks here
   https://github.com/MrPowers/mrpowers-benchmarks
   
   
   
   ### Describe the solution you'd like
   
   DataFusion has two APIs ways to implement Aggregate functions like `SUM` and `COUNT`
   - Easy (but slow) way: `Accumulator` ([api docs](https://docs.rs/datafusion/latest/datafusion/logical_expr/trait.Accumulator.html))
   - Fast (but complicated way): `GroupsAccumulator` ([api docs](https://docs.rs/datafusion/latest/datafusion/logical_expr/trait.GroupsAccumulator.html))
   
   The basic aggregates are implemented using `GroupsAccumulator` and are part of DataFusions  performance 
   
   This ticket tracks the effort to improve the performance of these  for these "more advanced" aggregate functions, likely by implementing `GroupsAccumulator`
   
   
   
   ### Describe alternatives you've considered
   
   For each function listed above, ideally we would:
   1. Add a new benchmark to the ClickBench extended benchmark [Documentation Here](https://github.com/apache/datafusion/tree/main/benchmarks/queries/clickbench#extended-queries) in one PR
   2. Implement `GroupsAccumulator` for the relevant aggregate function in a second PR (along with tests for correctness). We would use the benchmark to verify the performance
   
   
   Here is a pretty good example of how @eejbyfeldt  did this for `STDDEV`:
   - https://github.com/apache/datafusion/pull/12095
   
   ### Additional context
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2504139573

   > > > > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.
   > > > 
   > > > 
   > > > What version are you working with?
   > > > @Rachelint has some ideas of how to improve this:
   > > > ```
   > > > * [Sketch for aggregation intermediate results blocked management #11943](https://github.com/apache/datafusion/pull/11943)
   > > > 
   > > > * [Manage group values and states by blocks in aggregation #11931](https://github.com/apache/datafusion/issues/11931)
   > > > ```
   > > 
   > > 
   > > 🤔 I guess it may be caused by the similar reason of what we encountered during benchmarking in #11827
   > 
   > Specifically that `power` and `corr` need to support `convert_to_state`?
   
   I think it possible.
   I am running and profiling it to find the answer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2505971804

   > I am using a M3 Macbook with 16 GB of RAM. How much RAM does your machine have? Perhaps DataFusion only struggles with query 9 when the machine doesn't have lots of extra RAM.
   
   This explains 👍🏼 I ran the benchmark on a macbook with 48G of ram.
   It is likely Q9 requires > 16G RAM, and OS memory swapping caused the performance regression.
   
   We should also take a look at how much memory does DataFusion consume for those queries, comparing to other systems. Thanks for the report.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2504898262

   `GroupsAccumulator` for median/corr reduces memory usage (should be by quite a bit).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499442858

   > @Dandandan - thanks to the great work by @SemyonSinchenko, it's easy to generate these datasets with [falsa](https://github.com/mrpowers-io/falsa).
   > 
   > Here's the command to generate the 10 million row dataset: `falsa groupby --path-prefix=~/data --size SMALL --data-format PARQUET`. Just use `MEDIUM` to generate the 100 million row dataset.
   
   Thanks, I will profile and see what happen about the so long time cost in datafusion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499025788

   Hm this seems something quadratic in nature?
   
   > > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.
   > 
   > What version are you working with?
   > 
   > @Rachelint has some ideas of how to improve this:
   > 
   > * [Sketch for aggregation intermediate results blocked management #11943](https://github.com/apache/datafusion/pull/11943)
   > * [Manage group values and states by blocks in aggregation #11931](https://github.com/apache/datafusion/issues/11931)
   
   Does it fully explain the dramatic difference? @MrPowers how do you generate the 10M vs 100M rows?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "MrPowers (via GitHub)" <gi...@apache.org>.
MrPowers commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2504404645

   The h2o benchmarks are run on a Intel(R) Xeon(R) Platinum 8375C CPU @ 2.90GHz machine with 128 cores and 250 GB of RAM.
   
   DataFusion groupby queries perform well on the 100 million row dataset (~5GB of data in a CSV file):
   
   <img width="789" alt="Screenshot 2024-11-27 at 12 10 41 PM" src="https://github.com/user-attachments/assets/a50d9dda-6405-44de-bb99-65c100dc0078">
   
   Some don't run with the 1 billion row dataset (~50GB of data in an uncompressed CSV file):
   
   <img width="802" alt="Screenshot 2024-11-27 at 12 12 20 PM" src="https://github.com/user-attachments/assets/03414d79-3103-4f13-8ff2-7eec325333d2">
   
   I am using a M3 Macbook with 16 GB of RAM.  How much RAM does your machine have?  Perhaps DataFusion only struggles with query 9 when the machine doesn't have lots of extra RAM.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "MrPowers (via GitHub)" <gi...@apache.org>.
MrPowers commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2498033587

   Would like to note that the DataFusion performance really starts to lag when the dataset size grows.
   
   Take a look at this query: `select id2, id4, power(corr(v1, v2), 2) as r2 from x group by id2, id4`.
   
   When the dataset is 10 million rows, then Polars takes 3 seconds and DataFusion takes 3.6 seconds, so pretty similar.
   
   When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.  


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2501810315

   > > > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.
   > > 
   > > 
   > > What version are you working with?
   > > @Rachelint has some ideas of how to improve this:
   > > ```
   > > * [Sketch for aggregation intermediate results blocked management #11943](https://github.com/apache/datafusion/pull/11943)
   > > 
   > > * [Manage group values and states by blocks in aggregation #11931](https://github.com/apache/datafusion/issues/11931)
   > > ```
   > 
   > 🤔 I guess it may be caused by the similar reason of what we encountered during benchmarking in #11827
   
   Specifically that `power` and `corr` need to support `convert_to_state`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499011192

   > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.
   
   What version are you working with?
   
   @Rachelint  has some ideas of how to improve this:
   - https://github.com/apache/datafusion/pull/11943
   - https://github.com/apache/datafusion/issues/11931


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2506349949

   > > I am using a M3 Macbook with 16 GB of RAM. How much RAM does your machine have? Perhaps DataFusion only struggles with query 9 when the machine doesn't have lots of extra RAM.
   > 
   > This explains 👍🏼 I ran the benchmark on a macbook with 48G of ram. It is likely Q9 requires > 16G RAM, and OS memory swapping caused the performance regression.
   > 
   > We should also take a look at how much memory does DataFusion consume for those queries, comparing to other systems. Thanks for the report.
   
   Yes, I run it today, and my machine has only 16GB memory too... and I found the query very very slow due to swapping, too...


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2504258864

   I rerun the H2O Q9 with `GroupsAccumulator` for `corr()`. See https://github.com/apache/datafusion/pull/13581
   h2o dataset is in `parquet` format
   ```
   Result
   ----
   main, h2o_10m: 0.8s
   main, h2o_100m: 12s
   pr, h2o_10m: 0.2s
   pr, h2o_100m: 4s
   ```
   
   I didn't reproduce the drastic slowdown in `main` branch🤔 
   > When the dataset is 10 million rows, then Polars takes 3 seconds and DataFusion takes 3.6 seconds, so pretty similar.
   >
   > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2496263364

   For posterity, here is a link to the discord chat: https://discord.com/channels/885562378132000778/1309883046886903870/1309887744595595324


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2512381581

   I think making DataFusion work better in lower memory situations would certainly be nice


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499442851

   > @Dandandan - thanks to the great work by @SemyonSinchenko, it's easy to generate these datasets with [falsa](https://github.com/mrpowers-io/falsa).
   > 
   > Here's the command to generate the 10 million row dataset: `falsa groupby --path-prefix=~/data --size SMALL --data-format PARQUET`. Just use `MEDIUM` to generate the 100 million row dataset.
   
   Thanks, I will profile and see what happen about the so long time cost in datafusion.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "MrPowers (via GitHub)" <gi...@apache.org>.
MrPowers commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499396354

   @Dandandan - thanks to the great work by @SemyonSinchenko, it's easy to generate these datasets with [falsa](https://github.com/mrpowers-io/falsa).
   
   Here's the command to generate the 10 million row dataset: `falsa groupby --path-prefix=~/data --size SMALL --data-format PARQUET`.  Just use `MEDIUM` to generate the 100 million row dataset.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance (faster H20.ai benchmarks) [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2541865098

   Update here is that @2010YOUY01  has made `corr` much better:
   - https://github.com/apache/datafusion/pull/13581
   
   I also dug up and connected the task to add the H20.ai queries to bench.sh. Check out
   -  https://github.com/apache/datafusion/issues/7209#issuecomment-2541860170


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2504139586

   > > > > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.
   > > > 
   > > > 
   > > > What version are you working with?
   > > > @Rachelint has some ideas of how to improve this:
   > > > ```
   > > > * [Sketch for aggregation intermediate results blocked management #11943](https://github.com/apache/datafusion/pull/11943)
   > > > 
   > > > * [Manage group values and states by blocks in aggregation #11931](https://github.com/apache/datafusion/issues/11931)
   > > > ```
   > > 
   > > 
   > > 🤔 I guess it may be caused by the similar reason of what we encountered during benchmarking in #11827
   > 
   > Specifically that `power` and `corr` need to support `convert_to_state`?
   
   I think it possible.
   I am running and profiling it to find the answer.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499030684

   I would also expect this to help:
   - https://github.com/apache/datafusion/pull/11627


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Rachelint (via GitHub)" <gi...@apache.org>.
Rachelint commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2499632438

   > > When the dataset is 100 million rows, then Polars takes 126 seconds and DataFusion takes 2,100 seconds.
   > 
   > What version are you working with?
   > 
   > @Rachelint has some ideas of how to improve this:
   > 
   >     * [Sketch for aggregation intermediate results blocked management #11943](https://github.com/apache/datafusion/pull/11943)
   > 
   >     * [Manage group values and states by blocks in aggregation #11931](https://github.com/apache/datafusion/issues/11931)
   
   🤔 I guess it may be caused by the similar reason we encountered during benchmarking in https://github.com/apache/datafusion/pull/11827 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [I] [EPIC] Improved aggregate function performance [datafusion]

Posted by "Dandandan (via GitHub)" <gi...@apache.org>.
Dandandan commented on issue #13548:
URL: https://github.com/apache/datafusion/issues/13548#issuecomment-2504899070

   Looking at the benchmark results, I think query 8 is worth analyzing / optimizing as well:
   https://github.com/apache/datafusion/issues/13548


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org