You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2026/03/04 15:53:53 UTC
Re: [PR] fix: HashJoin panic with dictionary-encoded columns in multi-key joins [datafusion]
alamb commented on code in PR #20441:
URL: https://github.com/apache/datafusion/pull/20441#discussion_r2884593782
##########
datafusion/physical-plan/src/joins/hash_join/inlist_builder.rs:
##########
@@ -33,15 +33,16 @@ pub(super) fn build_struct_fields(data_types: &[DataType]) -> Result<Fields> {
.collect()
}
-/// Flattens dictionary-encoded arrays to their underlying value arrays.
+/// Casts dictionary-encoded arrays to their underlying value type, preserving row count.
/// Non-dictionary arrays are returned as-is.
-fn flatten_dictionary_array(array: &ArrayRef) -> ArrayRef {
- downcast_dictionary_array! {
- array => {
+fn flatten_dictionary_array(array: &ArrayRef) -> Result<ArrayRef> {
+ match array.data_type() {
+ DataType::Dictionary(_, value_type) => {
+ let casted = cast(array, value_type)?;
Review Comment:
I should have followed my instincts a bit more. It turns out we found another issue with this code at InfluxData
- https://github.com/apache/datafusion/issues/20696
Thankfully this PR fixes it
- https://github.com/apache/datafusion/pull/20505
##########
datafusion/physical-plan/src/joins/hash_join/inlist_builder.rs:
##########
@@ -33,15 +33,16 @@ pub(super) fn build_struct_fields(data_types: &[DataType]) -> Result<Fields> {
.collect()
}
-/// Flattens dictionary-encoded arrays to their underlying value arrays.
+/// Casts dictionary-encoded arrays to their underlying value type, preserving row count.
/// Non-dictionary arrays are returned as-is.
-fn flatten_dictionary_array(array: &ArrayRef) -> ArrayRef {
- downcast_dictionary_array! {
- array => {
+fn flatten_dictionary_array(array: &ArrayRef) -> Result<ArrayRef> {
+ match array.data_type() {
+ DataType::Dictionary(_, value_type) => {
+ let casted = cast(array, value_type)?;
Review Comment:
I should have followed my instincts a bit more. It turns out we found another issue with this code at InfluxData
- https://github.com/apache/datafusion/issues/20696
Thankfully the following PR fixes it:
- https://github.com/apache/datafusion/pull/20505
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org