You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by GitBox <gi...@apache.org> on 2021/04/29 19:03:06 UTC
[GitHub] [kafka] spena opened a new pull request #10612: KAFKA-10847: Add internal flag to disable KAFKA-10847 fix
spena opened a new pull request #10612:
URL: https://github.com/apache/kafka/pull/10612
Adds an internal flag that can be used to disable the fixes in KAFKA-10847. It defaults to `true` if the flag is not set or has an invalid boolean value.
The flag is named `__enable.kstreams.outer.join.spurious.results.fix__`. This flag is considered internal only. It is a temporary flag that will be used to help users to disable the join fixes while they do a transition from the previous semantics of left/outer joins. The flag may be removed in future releases.
How was it tested?
- I added a few tests to in left and outer joins to verify the previous semantics work when the flag is disabled.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on a change in pull request #10612: KAFKA-10847: Add internal flag to disable KAFKA-10847 fix
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on a change in pull request #10612:
URL: https://github.com/apache/kafka/pull/10612#discussion_r625369866
##########
File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/StreamStreamJoinNode.java
##########
@@ -95,7 +98,10 @@ public void writeToTopology(final InternalTopologyBuilder topologyBuilder) {
topologyBuilder.addStateStore(thisWindowStoreBuilder, thisWindowedStreamProcessorName, otherProcessorName);
topologyBuilder.addStateStore(otherWindowStoreBuilder, otherWindowedStreamProcessorName, thisProcessorName);
- outerJoinWindowStoreBuilder.ifPresent(builder -> topologyBuilder.addStateStore(builder, thisProcessorName, otherProcessorName));
+ final StreamsConfig streamsConfig = topologyBuilder.getStreamsConfig();
+ if (streamsConfig == null || StreamsConfig.InternalConfig.getBoolean(streamsConfig.originals(), ENABLE_KSTREAMS_OUTER_JOIN_SPURIOUS_RESULTS_FIX, true)) {
Review comment:
Cool, sgtm.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] spena commented on a change in pull request #10612: KAFKA-10847: Add internal flag to disable KAFKA-10847 fix
Posted by GitBox <gi...@apache.org>.
spena commented on a change in pull request #10612:
URL: https://github.com/apache/kafka/pull/10612#discussion_r625179200
##########
File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/StreamStreamJoinNode.java
##########
@@ -95,7 +98,10 @@ public void writeToTopology(final InternalTopologyBuilder topologyBuilder) {
topologyBuilder.addStateStore(thisWindowStoreBuilder, thisWindowedStreamProcessorName, otherProcessorName);
topologyBuilder.addStateStore(otherWindowStoreBuilder, otherWindowedStreamProcessorName, thisProcessorName);
- outerJoinWindowStoreBuilder.ifPresent(builder -> topologyBuilder.addStateStore(builder, thisProcessorName, otherProcessorName));
+ final StreamsConfig streamsConfig = topologyBuilder.getStreamsConfig();
+ if (streamsConfig == null || StreamsConfig.InternalConfig.getBoolean(streamsConfig.originals(), ENABLE_KSTREAMS_OUTER_JOIN_SPURIOUS_RESULTS_FIX, true)) {
Review comment:
I don't expect the user to always set the flag. The assumption is if the user did not set any flag, then flag should default to true. This applies to both cases, when the flag was not set or the streamsConfig was not set. I don't know in which scenarios streamsConfig won't be set, but just in case that happens, I prefer to enable the new join semantics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang commented on a change in pull request #10612: KAFKA-10847: Add internal flag to disable KAFKA-10847 fix
Posted by GitBox <gi...@apache.org>.
guozhangwang commented on a change in pull request #10612:
URL: https://github.com/apache/kafka/pull/10612#discussion_r624078706
##########
File path: streams/src/main/java/org/apache/kafka/streams/kstream/internals/graph/StreamStreamJoinNode.java
##########
@@ -95,7 +98,10 @@ public void writeToTopology(final InternalTopologyBuilder topologyBuilder) {
topologyBuilder.addStateStore(thisWindowStoreBuilder, thisWindowedStreamProcessorName, otherProcessorName);
topologyBuilder.addStateStore(otherWindowStoreBuilder, otherWindowedStreamProcessorName, thisProcessorName);
- outerJoinWindowStoreBuilder.ifPresent(builder -> topologyBuilder.addStateStore(builder, thisProcessorName, otherProcessorName));
+ final StreamsConfig streamsConfig = topologyBuilder.getStreamsConfig();
+ if (streamsConfig == null || StreamsConfig.InternalConfig.getBoolean(streamsConfig.originals(), ENABLE_KSTREAMS_OUTER_JOIN_SPURIOUS_RESULTS_FIX, true)) {
Review comment:
Do we expect the user of this internal flag to always set it? If say the caller had a bug to forget set it, should we default it to false instead?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [kafka] guozhangwang merged pull request #10612: KAFKA-10847: Add internal flag to disable KAFKA-10847 fix
Posted by GitBox <gi...@apache.org>.
guozhangwang merged pull request #10612:
URL: https://github.com/apache/kafka/pull/10612
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org