You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "gianm (via GitHub)" <gi...@apache.org> on 2025/07/08 23:20:49 UTC
[PR] MSQ: Fix composing channels losing partitionless frames. (druid)
gianm opened a new pull request, #18220:
URL: https://github.com/apache/druid/pull/18220
The main change is that single-partition ComposingWritableFrameChannels (i.e. those created by `ChannelOutputFactory#openChannel`) now associate all incoming frames with that partition. Previously, frames might have come in with partition set to `NO_PARTITION`, which would cause them to get "lost" by the composing channel.
Fixes a bug introduced in #18144 when composed intermediate stage-internal channels started being used for the output of hash partitioning. Prior to #18144, they were only used for internal channels of the SuperSorter. This bug could cause frames to go missing during sortMerge joins.
This patch also adds an embedded test for various durable storage scenarios, including sortMerge join tests that would have caught the original bug.
Finally, this patch adjusts the way that Calcites escapes string literals, to use the actual characters more often when possible. This helps format the test SQLs generated by the embedded test more nicely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] MSQ: Fix composing channels losing partitionless frames. (druid)
Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe commented on code in PR #18220:
URL: https://github.com/apache/druid/pull/18220#discussion_r2194232805
##########
embedded-tests/pom.xml:
##########
@@ -182,6 +182,11 @@
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.hamcrest</groupId>
+ <artifactId>hamcrest-all</artifactId>
+ <scope>test</scope>
+ </dependency>
<dependency>
Review Comment:
```suggestion
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org
Re: [PR] MSQ: Fix composing channels losing partitionless frames. (druid)
Posted by "cryptoe (via GitHub)" <gi...@apache.org>.
cryptoe merged PR #18220:
URL: https://github.com/apache/druid/pull/18220
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org