(datafusion) 01/01: test: add advanced dictionary test (#23483)

[gi...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

github-merge-queue[bot] pushed a commit to branch gh-readonly-queue/main/pr-23483-0f5790af040fc18cd941f6c6d6f0faeb3df60a9a
in repository https://gitbox.apache.org/repos/asf/datafusion.git

commit 3b1ce16be6beb3377c59ec36fd9f8ae62998c4c0
Author: RIchard Baah <13...@users.noreply.github.com>
AuthorDate: Mon Jul 20 03:40:08 2026 -0400

    test: add advanced dictionary test (#23483)
    
    ## Which issue does this PR close?
    
    <!--
    We generally require a GitHub issue to be filed for all bug fixes and
    enhancements and this helps us generate change logs for our releases.
    You can link an issue to this PR using the GitHub syntax. For example
    `Closes #123` indicates that this PR will close issue #123.
    -->
    
    follow up PR for #23280
    
    ## Rationale for this change
    
    see comment
    https://github.com/apache/datafusion/pull/23280#discussion_r3548804883
    <!--
    Why are you proposing this change? If this is already explained clearly
    in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand
    your changes and offer better suggestions for fixes.
    -->
    
    ## What changes are included in this PR?
    
    Adds SQL logic tests to verify that GROUP BY on dictionary-encoded
    columns resolves on values rather than raw dictionary key integers. The
    tests use UNION ALL between independently-encoded subqueries (mixing
    Int32/Int16/Int8 key types and Utf8/LargeUtf8 value types)
    
    <!--
    There is no need to duplicate the description in the issue here but it
    is sometimes worth providing a summary of the individual changes in this
    PR.
    -->
    
    ## Are these changes tested?
    
    the test cover
    
    - Correct grouping when the same value maps to different keys in
    different batches
    - No false merging of distinct values that happen to share the same key
    number
    - Null handling across batches with mismatched key assignments
    
    <!--
    We typically require tests for all PRs in order to:
    1. Prevent the code from being accidentally broken by subsequent changes
    2. Serve as another way to document the expected behavior of the code
    
    If tests are not included in your PR, please explain why (for example,
    are they covered by existing tests)?
    -->
    
    ## Are there any user-facing changes?
    no
    <!--
    If there are user-facing changes then we may require documentation to be
    updated before approving the PR.
    -->
    
    <!--
    If there are any breaking changes to public APIs, please add the `api
    change` label.
    -->
---
 datafusion/sqllogictest/test_files/dictionary.slt | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/datafusion/sqllogictest/test_files/dictionary.slt b/datafusion/sqllogictest/test_files/dictionary.slt
index 105523ab50..f314254955 100644
--- a/datafusion/sqllogictest/test_files/dictionary.slt
+++ b/datafusion/sqllogictest/test_files/dictionary.slt
@@ -632,4 +632,22 @@ south 2
 statement ok
 DROP TABLE dict_count_distinct;
 
-
+# same dictionary type but value order differs across batches so key ids refer to different strings;
+# grouping must use the logical value, not the raw key id
+query TI rowsort
+WITH
+  first_batch AS (
+    SELECT arrow_cast(column1, 'Dictionary(Int32, Utf8)') AS region
+    FROM (VALUES ('west'), ('west'), ('west'), ('east'), (NULL)) AS t(column1)
+  ),
+  second_batch AS (
+    SELECT arrow_cast(column1, 'Dictionary(Int32, Utf8)') AS region
+    FROM (VALUES ('east'), ('east'), ('east'), ('west'), (NULL)) AS t(column1)
+  )
+SELECT region, count(*)
+FROM (SELECT region FROM first_batch UNION ALL SELECT region FROM second_batch)
+GROUP BY region;
+----
+NULL 2
+east 4
+west 4


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org