You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "mkleen (via GitHub)" <gi...@apache.org> on 2026/10/01 20:19:25 UTC
[PR] feat: Support Struct type in approx_distinct - #23663 [datafusion]
mkleen opened a new pull request, #25951:
URL: https://github.com/apache/datafusion/pull/25951
## Which issue does this PR close?
- Closes #22989
## Rationale for this change
- Support the Arrow type `RunEndEncoded` type for `approx_distinct`
## What changes are included in this PR?
- Enable `HLLAccumulator` and `HllGroupsAccumulator` to support `RunEndEncoded`
- Tests for the non-grouped and grouped path as part of `aggregate.slt`
## Are these changes tested?
Yes.
## Are there any user-facing changes?
Yes, `approx_distinct` supports now `RunEndEncoded` but no breaking changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Support RunEndEncoded type in approx_distinct [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 commented on PR #25951:
URL: https://github.com/apache/datafusion/pull/25951#issuecomment-5969455393
Thanks @mkleen @nuno-faria !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Support RunEndEncoded type in approx_distinct [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #25951:
URL: https://github.com/apache/datafusion/pull/25951#issuecomment-5940329671
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/25951?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 82.59%. Comparing base ([`141663e`](https://app.codecov.io/gh/apache/datafusion/commit/141663e8c49a484ae84ef4cf67ede80ab56b85d0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`9bfac5f`](https://app.codecov.io/gh/apache/datafusion/commit/9bfac5f3a4ffccbec84112ae579a66231c10b835?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #25951 +/- ##
==========================================
- Coverage 82.59% 82.59% -0.01%
==========================================
Files 1145 1145
Lines 443932 443933 +1
Branches 443932 443933 +1
==========================================
- Hits 366663 366656 -7
- Misses 55012 55017 +5
- Partials 22257 22260 +3
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/25951?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Support RunEndEncoded type in approx_distinct [datafusion]
Posted by "jayzhan211 (via GitHub)" <gi...@apache.org>.
jayzhan211 merged PR #25951:
URL: https://github.com/apache/datafusion/pull/25951
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: Support RunEndEncoded type in approx_distinct [datafusion]
Posted by "mkleen (via GitHub)" <gi...@apache.org>.
mkleen commented on PR #25951:
URL: https://github.com/apache/datafusion/pull/25951#issuecomment-5961012390
> Thanks @mkleen.
>
> The `is_supported_type` reminded me that we don't do the same thing for lists, maps, ... So approx_distinct does not work for (e.g.) floats but works for list of floats. I think it's something we should look at in a follow up.
Thanks for the review @nuno-faria
I agree, the container types such as lists, maps need another iteration. Technically the only unsupported types for `approx_distinct` are Float16, Float32 and Float64. DuckDB supports them. So maybe it's worth thinking to support them in datafusion as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org