You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "mjsax (via GitHub)" <gi...@apache.org> on 2024/09/06 03:25:11 UTC
[PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
mjsax opened a new pull request, #17108:
URL: https://github.com/apache/kafka/pull/17108
We will remove EOSv1 in 4.0, and thus can remove the corresponding system tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #17108:
URL: https://github.com/apache/kafka/pull/17108#discussion_r1752430411
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java:
##########
@@ -742,9 +742,9 @@ boolean runLoop() {
errorMessage.startsWith("Broker unexpectedly doesn't support requireStable flag on version ")) {
log.error("Shutting down because the Kafka cluster seems to be on a too old version. " +
- "Setting {}=\"{}\"/\"{}\" requires broker version 2.5 or higher.",
+ "Setting {}=\"{}\" requires broker version 2.5 or higher.",
StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
- StreamsConfig.EXACTLY_ONCE_V2, StreamsConfig.EXACTLY_ONCE_BETA);
Review Comment:
Could you please remove the `@SuppressWarnings("deprecation")` from line:688 as `StreamsConfig.EXACTLY_ONCE_BETA` is removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #17108:
URL: https://github.com/apache/kafka/pull/17108#discussion_r1769344246
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java:
##########
@@ -742,9 +742,9 @@ boolean runLoop() {
errorMessage.startsWith("Broker unexpectedly doesn't support requireStable flag on version ")) {
log.error("Shutting down because the Kafka cluster seems to be on a too old version. " +
- "Setting {}=\"{}\"/\"{}\" requires broker version 2.5 or higher.",
+ "Setting {}=\"{}\" requires broker version 2.5 or higher.",
StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
- StreamsConfig.EXACTLY_ONCE_V2, StreamsConfig.EXACTLY_ONCE_BETA);
Review Comment:
Addressed via https://github.com/apache/kafka/pull/17170
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax merged PR #17108:
URL: https://github.com/apache/kafka/pull/17108
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #17108:
URL: https://github.com/apache/kafka/pull/17108#discussion_r1746521228
##########
streams/src/test/java/org/apache/kafka/streams/tests/BrokerCompatibilityTest.java:
##########
@@ -68,8 +68,8 @@ public static void main(final String[] args) throws IOException {
streamsProperties.put(StreamsConfig.APPLICATION_ID_CONFIG, "kafka-streams-system-test-broker-compatibility");
streamsProperties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
- streamsProperties.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
- streamsProperties.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
+ streamsProperties.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.StringSerde.class);
+ streamsProperties.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.StringSerde.class);
Review Comment:
Side improvement
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on PR #17108:
URL: https://github.com/apache/kafka/pull/17108#issuecomment-2333166082
Triggered a system test run to verify that we did not break anything: https://semaphore.ci.confluent.io/workflows/4481964d-cbd3-4982-959f-98d9f2cfa4bf
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #17108:
URL: https://github.com/apache/kafka/pull/17108#discussion_r1752977363
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/StreamThread.java:
##########
@@ -742,9 +742,9 @@ boolean runLoop() {
errorMessage.startsWith("Broker unexpectedly doesn't support requireStable flag on version ")) {
log.error("Shutting down because the Kafka cluster seems to be on a too old version. " +
- "Setting {}=\"{}\"/\"{}\" requires broker version 2.5 or higher.",
+ "Setting {}=\"{}\" requires broker version 2.5 or higher.",
StreamsConfig.PROCESSING_GUARANTEE_CONFIG,
- StreamsConfig.EXACTLY_ONCE_V2, StreamsConfig.EXACTLY_ONCE_BETA);
Review Comment:
Good point. Will do in a follow up PR. (EOSv1 removal will be many PRs...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "chia7712 (via GitHub)" <gi...@apache.org>.
chia7712 commented on code in PR #17108:
URL: https://github.com/apache/kafka/pull/17108#discussion_r1750765034
##########
streams/src/test/java/org/apache/kafka/streams/tests/StreamsEosTest.java:
##########
@@ -49,12 +48,9 @@ public static void main(final String[] args) throws IOException {
}
if ("process".equals(command) || "process-complex".equals(command)) {
- if (!StreamsConfig.EXACTLY_ONCE.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE_BETA.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE_V2.equals(processingGuarantee)) {
+ if (!StreamsConfig.EXACTLY_ONCE_V2.equals(processingGuarantee)) {
- System.err.println("processingGuarantee must be either " + StreamsConfig.EXACTLY_ONCE + " or " +
- StreamsConfig.EXACTLY_ONCE_BETA + " or " + StreamsConfig.EXACTLY_ONCE_V2);
+ System.err.println("processingGuarantee must be " + StreamsConfig.EXACTLY_ONCE_V2);
Review Comment:
nit: `processingGuarantee` -> `processing.guarantee`
##########
streams/src/test/java/org/apache/kafka/streams/tests/StreamsSmokeTest.java:
##########
@@ -60,14 +59,10 @@ public static void main(final String[] args) throws IOException {
if ("process".equals(command)) {
if (!StreamsConfig.AT_LEAST_ONCE.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE_BETA.equals(processingGuarantee) &&
!StreamsConfig.EXACTLY_ONCE_V2.equals(processingGuarantee)) {
System.err.println("processingGuarantee must be either " +
Review Comment:
nit: `processingGuarantee` -> `processing.guarantee`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on PR #17108:
URL: https://github.com/apache/kafka/pull/17108#issuecomment-2336213631
Another re-run: https://semaphore.ci.confluent.io/workflows/82a881c7-93f6-4a1e-91a5-d0fb634b3212?pipeline_id=af4f87e6-1aa3-4300-9993-6f23774e0169
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on PR #17108:
URL: https://github.com/apache/kafka/pull/17108#issuecomment-2335033736
`streams_broker_compatibility_test` and `streams_eos_test` failed. Pushed a fix and triggered re-runs
- https://semaphore.ci.confluent.io/workflows/87fd8199-6507-4e40-83b7-2e9500f87626
- https://semaphore.ci.confluent.io/workflows/b08bdfcd-6c7f-4ae1-8e92-dead9c121fcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16331: Remove EOSv1 from Kafka Streams system tests [kafka]
Posted by "mjsax (via GitHub)" <gi...@apache.org>.
mjsax commented on code in PR #17108:
URL: https://github.com/apache/kafka/pull/17108#discussion_r1752413121
##########
streams/src/test/java/org/apache/kafka/streams/tests/StreamsEosTest.java:
##########
@@ -49,12 +48,9 @@ public static void main(final String[] args) throws IOException {
}
if ("process".equals(command) || "process-complex".equals(command)) {
- if (!StreamsConfig.EXACTLY_ONCE.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE_BETA.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE_V2.equals(processingGuarantee)) {
+ if (!StreamsConfig.EXACTLY_ONCE_V2.equals(processingGuarantee)) {
- System.err.println("processingGuarantee must be either " + StreamsConfig.EXACTLY_ONCE + " or " +
- StreamsConfig.EXACTLY_ONCE_BETA + " or " + StreamsConfig.EXACTLY_ONCE_V2);
+ System.err.println("processingGuarantee must be " + StreamsConfig.EXACTLY_ONCE_V2);
Review Comment:
Actually correct as-is, as we refer to the variable `processingGuarantee` here -- note: it's internal test code; not user facing.
##########
streams/src/test/java/org/apache/kafka/streams/tests/StreamsSmokeTest.java:
##########
@@ -60,14 +59,10 @@ public static void main(final String[] args) throws IOException {
if ("process".equals(command)) {
if (!StreamsConfig.AT_LEAST_ONCE.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE.equals(processingGuarantee) &&
- !StreamsConfig.EXACTLY_ONCE_BETA.equals(processingGuarantee) &&
!StreamsConfig.EXACTLY_ONCE_V2.equals(processingGuarantee)) {
System.err.println("processingGuarantee must be either " +
Review Comment:
Same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org