You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "aliehsaeedii (via GitHub)" <gi...@apache.org> on 2026/03/19 22:05:05 UTC
[PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
aliehsaeedii opened a new pull request, #21827:
URL: https://github.com/apache/kafka/pull/21827
Parametrized 28 DSL and processor test classes to validate functionality with both `headers` and `default` store configurations. This ensures backward compatibility and correctness as Kafka Streams transitions to supporting record headers in state stores.
Modified Tests:
- Materializer Tests
- Windowed KStream Tests
- Aggregation Tests
- Cogroup Tests
- Join Tests
- KTable Tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041103330
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -312,9 +334,13 @@ public void shouldRemoveMergedSessionsFromStateStore(final EmitStrategy.Strategy
}
@ParameterizedTest
- @EnumSource(EmitStrategy.StrategyType.class)
- public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType) {
- setup(inputType, true);
+ @MethodSource("data")
+ public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType, final boolean enableCaching, final boolean withHeaders) {
+ setup(inputType, enableCaching, withHeaders);
+ // This test expects caching behavior for accurate result counts
+ if (!enableCaching && !emitFinal) {
Review Comment:
same question
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on PR #21827:
URL: https://github.com/apache/kafka/pull/21827#issuecomment-4210068714
retrigger CI
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041129372
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableAggregateTest.java:
##########
@@ -186,9 +201,19 @@ public void testAggRepartition() {
}
}
- @Test
- public void testAggOfVersionedStore() {
- final StreamsBuilder builder = new StreamsBuilder();
+ @ParameterizedTest
+ @MethodSource("data")
+ public void testAggOfVersionedStore(final boolean withHeaders) {
+ final StreamsBuilder builder;
+ if (withHeaders) {
+ final Properties props = new Properties();
+ props.putAll(CONFIG);
+ props.putAll(StreamsTestUtils.getStreamsConfig(Serdes.String(), Serdes.String()));
+ props.put(StreamsConfig.DSL_STORE_FORMAT_CONFIG, StreamsConfig.DSL_STORE_FORMAT_HEADERS);
+ builder = new StreamsBuilder(new TopologyConfig(new StreamsConfig(props)));
Review Comment:
Do we actually need this change at all, given that versions stores don't support headers yet?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041011795
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -144,18 +146,21 @@ private void makeJoin(final Duration grace) {
Joined.with(Serdes.Integer(), Serdes.String(), Serdes.String(), "Grace", grace)
).process(supplier);
final Properties props = StreamsTestUtils.getStreamsConfig(Serdes.Integer(), Serdes.String());
+ if (withHeaders) {
+ props.put(StreamsConfig.DSL_STORE_FORMAT_CONFIG, StreamsConfig.DSL_STORE_FORMAT_HEADERS);
+ }
driver = new TopologyTestDriver(builder.build(), props);
inputStreamTopic = driver.createInputTopic(streamTopic, new IntegerSerializer(), new StringSerializer(), Instant.ofEpochMilli(0L), Duration.ZERO);
inputTableTopic = driver.createInputTopic("tableTopic2", new IntegerSerializer(), new StringSerializer(), Instant.ofEpochMilli(0L), Duration.ZERO);
processor = supplier.theCapturedProcessor();
}
- @Test
- public void shouldFailIfTableIsNotVersioned() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldFailIfTableIsNotVersioned(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
- final Properties props = new Properties();
- props.put(StreamsConfig.TOPOLOGY_OPTIMIZATION_CONFIG, StreamsConfig.NO_OPTIMIZATION);
Review Comment:
Why remove this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041134774
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableAggregateTest.java:
##########
@@ -300,9 +327,19 @@ public void testCountWithInternalStore() {
testCountHelper(builder, input, supplier);
}
- @Test
- public void testCountOfVersionedStore() {
- final StreamsBuilder builder = new StreamsBuilder();
+ @ParameterizedTest
+ @MethodSource("data")
+ public void testCountOfVersionedStore(final boolean withHeaders) {
+ final StreamsBuilder builder;
+ if (withHeaders) {
Review Comment:
Same question as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041178099
##########
streams/src/test/java/org/apache/kafka/streams/processor/internals/KeyValueStoreMaterializerTest.java:
##########
@@ -78,8 +78,7 @@ public class KeyValueStoreMaterializerTest {
@Mock
private StreamsConfig streamsConfig;
- @BeforeEach
- public void setUp() {
+ public void setUp(final boolean withHeaders) {
Review Comment:
Does it make sense to parameterize this test (same question for "sliding" and "windowed" case). This test already has explicit setup to test w/ and w/o header stores, for different test methods.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3052281867
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -93,17 +95,29 @@ public class KStreamSessionWindowAggregateProcessorTest {
private KStreamSessionWindowAggregate<String, String, Long> sessionAggregator;
private Processor<String, String, Windowed<String>, Change<Long>> processor;
private SessionStoreWithHeaders<String, Long> sessionStore;
-
+
public EmitStrategy.StrategyType type;
private EmitStrategy emitStrategy;
private boolean emitFinal;
- private void setup(final EmitStrategy.StrategyType inputType, final boolean enableCaching) {
+ public static Stream<Arguments> data() {
Review Comment:
Could we have some meaningful name?
for example `emitStrategyAndCachingMatrix`
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSlidingWindowAggregateTest.java:
##########
@@ -92,14 +92,22 @@ public class KStreamSlidingWindowAggregateTest {
public static Stream<Arguments> data() {
Review Comment:
ditto, we should rename it.
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableAggregateTest.java:
##########
@@ -76,17 +78,22 @@ public class KTableAggregateTest {
private final MockApiProcessorSupplier<String, Object, Void, Void> supplier = new MockApiProcessorSupplier<>();
private static final Properties CONFIG = mkProperties(mkMap(
mkEntry(StreamsConfig.STATE_DIR_CONFIG, TestUtils.tempDirectory("kafka-test").getAbsolutePath())));
-
- private StreamsBuilder createStreamBuilderInMemory() {
+
Review Comment:
redundant empty line
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableImplTest.java:
##########
@@ -80,18 +82,28 @@ public class KTableImplTest {
private final Consumed<String, String> stringConsumed = Consumed.with(Serdes.String(), Serdes.String());
private final Consumed<String, String> consumed = Consumed.with(Serdes.String(), Serdes.String());
private final Produced<String, String> produced = Produced.with(Serdes.String(), Serdes.String());
- private final Properties props = StreamsTestUtils.getStreamsConfig(Serdes.String(), Serdes.String());
+ private Properties props = StreamsTestUtils.getStreamsConfig(Serdes.String(), Serdes.String());
private final Serde<String> mySerde = new Serdes.StringSerde();
private KTable<String, String> table;
+ private void setupProps(final boolean withHeaders) {
Review Comment:
I think `setDslStoreFormat` is a better name and it also align with other test files.
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableKTableOuterJoinTest.java:
##########
@@ -434,4 +445,12 @@ private void assertOutputKeyValueTimestamp(final TestOutputTopic<Integer, String
assertThat(outputTopic.readRecord(), equalTo(new TestRecord<>(expectedKey, expectedValue, null, expectedTimestamp)));
}
+ private void setDslStoreFormat(final boolean withHeaders) {
Review Comment:
ditto
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableKTableInnerJoinTest.java:
##########
@@ -477,4 +491,12 @@ private void assertOutputKeyValueTimestamp(final TestOutputTopic<Integer, String
assertThat(outputTopic.readRecord(), equalTo(new TestRecord<>(expectedKey, expectedValue, null, expectedTimestamp)));
}
+ private void setDslStoreFormat(final boolean withHeaders) {
Review Comment:
I noticed there's Javadoc for this method in other test files. I’m fine with or without it—up to you—but the key is to stay consistent.
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableMapValuesTest.java:
##########
@@ -321,4 +336,12 @@ private void testSendingOldValues(
);
}
}
+
+ private void setDslStoreFormat(final boolean withHeaders) {
Review Comment:
ditto
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableFilterTest.java:
##########
@@ -59,12 +59,16 @@
@SuppressWarnings("unchecked")
public class KTableFilterTest {
private final Consumed<String, Integer> consumed = Consumed.with(Serdes.String(), Serdes.Integer());
- private final Properties props = StreamsTestUtils.getStreamsConfig(Serdes.String(), Serdes.Integer());
+ private Properties props = StreamsTestUtils.getStreamsConfig(Serdes.String(), Serdes.Integer());
- @BeforeEach
- public void setUp() {
+ private void setupProps(final boolean withHeaders) {
Review Comment:
nit:
I suppose we could keep the original name.
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamWindowAggregateTest.java:
##########
@@ -102,26 +103,35 @@ public class KStreamWindowAggregateTest {
private boolean emitFinal;
- public static Stream<Arguments> getEmitStrategy() {
+ public static Stream<Arguments> data() {
Review Comment:
ditto
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableKTableRightJoinTest.java:
##########
@@ -67,4 +70,12 @@ public void shouldLogAndMeterSkippedRecordsDueToNullLeftKeyWithBuiltInMetricsVer
);
}
}
+
+ private void setDslStoreFormat(final boolean withHeaders) {
Review Comment:
ditto
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KTableMapKeysTest.java:
##########
@@ -76,4 +80,12 @@ public void testMapKeysConvertingToStream() {
assertEquals(expected[i], supplier.theCapturedProcessor().processed().get(i));
}
}
+
+ private void setDslStoreFormat(final boolean withHeaders) {
Review Comment:
ditto
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041102438
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -233,9 +251,13 @@ public void shouldUpdateSessionIfTheSameTime(final EmitStrategy.StrategyType inp
}
@ParameterizedTest
- @EnumSource(EmitStrategy.StrategyType.class)
- public void shouldHaveMultipleSessionsForSameIdWhenTimestampApartBySessionGap(final EmitStrategy.StrategyType inputType) {
- setup(inputType, true);
+ @MethodSource("data")
+ public void shouldHaveMultipleSessionsForSameIdWhenTimestampApartBySessionGap(final EmitStrategy.StrategyType inputType, final boolean enableCaching, final boolean withHeaders) {
+ setup(inputType, enableCaching, withHeaders);
+ // This test expects caching behavior for accurate result counts
+ if (!enableCaching && !emitFinal) {
Review Comment:
Why do we run this test only with caching disabled, and emit-on-change, but not with emit-on-final? Seems the original test setup did includ emit-on-final?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3040994394
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -167,11 +172,11 @@ public void shouldFailIfTableIsNotVersioned() {
);
}
- @Test
- public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
- final Properties props = new Properties();
- props.put(StreamsConfig.TOPOLOGY_OPTIMIZATION_CONFIG, StreamsConfig.NO_OPTIMIZATION);
Review Comment:
Not sure why we did disable this explicitly as it's disabled by default -- seems we have similar stuff in other tests on this class. Should we also remove it?
Still wondering, why we have this explicit disabling 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041025337
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -187,11 +192,16 @@ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
);
}
- @Test
- public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
final Properties props = new Properties();
Review Comment:
`setUp` create it's own local `Properties` which is passed into TTD. So if we create new one here, that we need to pass into `builder.build()` below, why do we need to call `setUp()` at all?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3047455425
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -187,11 +192,16 @@ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
);
}
- @Test
- public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
final Properties props = new Properties();
Review Comment:
`setUp` must be called on every test as it used to a `@beforeEach` method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3040994394
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -167,11 +172,11 @@ public void shouldFailIfTableIsNotVersioned() {
);
}
- @Test
- public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
- final Properties props = new Properties();
- props.put(StreamsConfig.TOPOLOGY_OPTIMIZATION_CONFIG, StreamsConfig.NO_OPTIMIZATION);
Review Comment:
Not sure why we did disable this explicitly as it's disabled by default -- seems we have similar stuff in other tests on this class. Should we also remove it? -- Any reason why you remove it here, but not elsewhere?
Still wondering, why we have this explicit disabling 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041025337
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -187,11 +192,16 @@ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
);
}
- @Test
- public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
final Properties props = new Properties();
Review Comment:
`setUp` create it's own local `Properties` which is passed into TTD. So if we create new one here, that we need to pass into `builder.build()` below, why do we need to call `setUp()` at all?
This pattern of calling `setUp` and using local `Properties` is on other methods, too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3047455425
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -187,11 +192,16 @@ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
);
}
- @Test
- public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
final Properties props = new Properties();
Review Comment:
`setUp` must be called on every test as it used to be a `@beforeEach` method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3047455425
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -187,11 +192,16 @@ public void shouldFailIfTableIsNotVersionedButMaterializationIsInherited() {
);
}
- @Test
- public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited() {
+ @ParameterizedTest
+ @ValueSource(booleans = {false, true})
+ public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited(final boolean withHeaders) {
+ setUp(withHeaders);
final StreamsBuilder builder = new StreamsBuilder();
final Properties props = new Properties();
Review Comment:
`setUp` must be called on every test as it used to a @beforeEach method
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "frankvicky (via GitHub)" <gi...@apache.org>.
frankvicky merged PR #21827:
URL: https://github.com/apache/kafka/pull/21827
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3041041718
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamKTableJoinTest.java:
##########
@@ -201,14 +211,19 @@ public void shouldNotFailIfTableIsVersionedButMaterializationIsInherited() {
streamA.join(tableB, (value1, value2) -> value1 + value2, Joined.with(Serdes.String(), Serdes.String(), Serdes.String(), "first-join", Duration.ofMillis(6))).to("out-one");
//should not throw an error
- builder.build();
+ builder.build(props);
Review Comment:
Why do we need to pass in `props` now, but not before? I understand that the "headers" flag is a DSL config, so if the `build(...)` method needs that we pass `props` to pick it up, it seems we need to make this change everywhere to actually enable header store? This would also apply to your other PRs (maybe even the already merged N/2? -- If we need to pass in here, all your other changes seems to actually _not_ enable header stores?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3048019867
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -233,9 +251,13 @@ public void shouldUpdateSessionIfTheSameTime(final EmitStrategy.StrategyType inp
}
@ParameterizedTest
- @EnumSource(EmitStrategy.StrategyType.class)
- public void shouldHaveMultipleSessionsForSameIdWhenTimestampApartBySessionGap(final EmitStrategy.StrategyType inputType) {
- setup(inputType, true);
+ @MethodSource("data")
+ public void shouldHaveMultipleSessionsForSameIdWhenTimestampApartBySessionGap(final EmitStrategy.StrategyType inputType, final boolean enableCaching, final boolean withHeaders) {
+ setup(inputType, enableCaching, withHeaders);
+ // This test expects caching behavior for accurate result counts
+ if (!enableCaching && !emitFinal) {
Review Comment:
The original test considers caching as enabled: https://github.com/apache/kafka/blob/794d2b625fa83ea2208c96682239ca8261d5cc3d/streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java#L238
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -312,9 +334,13 @@ public void shouldRemoveMergedSessionsFromStateStore(final EmitStrategy.Strategy
}
@ParameterizedTest
- @EnumSource(EmitStrategy.StrategyType.class)
- public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType) {
- setup(inputType, true);
+ @MethodSource("data")
+ public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType, final boolean enableCaching, final boolean withHeaders) {
+ setup(inputType, enableCaching, withHeaders);
+ // This test expects caching behavior for accurate result counts
+ if (!enableCaching && !emitFinal) {
Review Comment:
the same as above: https://github.com/apache/kafka/blob/794d2b625fa83ea2208c96682239ca8261d5cc3d/streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java#L317
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -312,9 +334,13 @@ public void shouldRemoveMergedSessionsFromStateStore(final EmitStrategy.Strategy
}
@ParameterizedTest
- @EnumSource(EmitStrategy.StrategyType.class)
- public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType) {
- setup(inputType, true);
+ @MethodSource("data")
+ public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType, final boolean enableCaching, final boolean withHeaders) {
+ setup(inputType, enableCaching, withHeaders);
+ // This test expects caching behavior for accurate result counts
+ if (!enableCaching && !emitFinal) {
Review Comment:
And regarding `!enableCaching`: Without it, we'd also skip enableCaching=false + emitFinal=true (ON_WINDOW_CLOSE), but that case works fine! Because ON_WINDOW_CLOSE has its own buffering mechanism independent of caching.
We only want to skip when there's absolutely no buffering - which is only when: !enableCaching AND !emitFinal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "aliehsaeedii (via GitHub)" <gi...@apache.org>.
aliehsaeedii commented on code in PR #21827:
URL: https://github.com/apache/kafka/pull/21827#discussion_r3048157157
##########
streams/src/test/java/org/apache/kafka/streams/kstream/internals/KStreamSessionWindowAggregateProcessorTest.java:
##########
@@ -312,9 +334,13 @@ public void shouldRemoveMergedSessionsFromStateStore(final EmitStrategy.Strategy
}
@ParameterizedTest
- @EnumSource(EmitStrategy.StrategyType.class)
- public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType) {
- setup(inputType, true);
+ @MethodSource("data")
+ public void shouldHandleMultipleSessionsAndMerging(final EmitStrategy.StrategyType inputType, final boolean enableCaching, final boolean withHeaders) {
+ setup(inputType, enableCaching, withHeaders);
+ // This test expects caching behavior for accurate result counts
+ if (!enableCaching && !emitFinal) {
Review Comment:
And regarding `!enableCaching`: Without it, we'd also skip enableCaching=false + emitFinal=true (ON_WINDOW_CLOSE), but that case works fine! Because ON_WINDOW_CLOSE has its own buffering mechanism independent of caching.
We only want to skip when there's absolutely no buffering - which is only when: !enableCaching AND !emitFinal
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-20329: Test headers dsl.store.format further (4/N) [kafka]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #21827:
URL: https://github.com/apache/kafka/pull/21827#issuecomment-4139988331
A label of 'needs-attention' was automatically added to this PR in order to raise the
attention of the committers. Once this issue has been triaged, the `triage` label
should be removed to prevent this automation from happening again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org