You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/06/16 21:17:36 UTC
[GitHub] [flink] MartijnVisser opened a new pull request, #19994: [FLINK-28060][Connector/Kafka] Updated Kafka Clients to 3.1.1
MartijnVisser opened a new pull request, #19994:
URL: https://github.com/apache/flink/pull/19994
## What is the purpose of the change
Updated Kafka Clients to resolve the issue where Flink is unable to commit its offset back to Kafka in case of Kafka Broker becoming unavailable. This should be resolved when the broker comes back up, but due to KAFKA-13563 that doesn't work. Since that fix has only become available with Kafka Clients 3.1.1, this commit updates the Kafka Clients dependency from 2.8.4 to 3.1.1.
No interfaces needed to be adjusted.
It was necessary to change some of our Bash e2e tests since they still relied on the Zookeeper parameter which has been removed in this version.
The other necessary change was adjusting the `KafkaConsumerTestBase` class since the level of exception is changed in the new Kafka Clients so exception.getCause().getMessage() throws an NPE in the test case.
## Brief change log
* Updated Kafka Clients dependency
* Adjusted Bash e2e tests to connect to the brokers instead of zookeeper
* Small change to `KafkaConsumerTestBase` due to change in level of exception
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): yes
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] MartijnVisser commented on pull request #19994: [FLINK-28060][Connector/Kafka] Updated Kafka Clients to 3.1.1
Posted by GitBox <gi...@apache.org>.
MartijnVisser commented on PR #19994:
URL: https://github.com/apache/flink/pull/19994#issuecomment-1159371183
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] snuyanzin commented on a diff in pull request #19994: [FLINK-28060][Connector/Kafka] Updated Kafka Clients to 3.1.1
Posted by GitBox <gi...@apache.org>.
snuyanzin commented on code in PR #19994:
URL: https://github.com/apache/flink/pull/19994#discussion_r899798301
##########
flink-connectors/flink-connector-kafka/src/test/java/org/apache/flink/streaming/connectors/kafka/KafkaConsumerTestBase.java:
##########
@@ -192,7 +192,7 @@ public void runFailOnNoBrokerTest() throws Exception {
final TimeoutException timeoutException = optionalTimeoutException.get();
if (useNewSource) {
- assertThat(timeoutException.getCause().getMessage())
+ assertThat(timeoutException.getMessage())
.contains("Timed out waiting for a node assignment.");
Review Comment:
Probably `hasMessageContaining` could be used here like
```java
assertThat(timeoutException)
.hasMessageContaining("Timed out waiting for a node assignment.");
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #19994: [FLINK-28060][Connector/Kafka] Updated Kafka Clients to 3.1.1
Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #19994:
URL: https://github.com/apache/flink/pull/19994#issuecomment-1158142197
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "afdd18d6402f6c2aa0b6a342618c8251a473ee61",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "afdd18d6402f6c2aa0b6a342618c8251a473ee61",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* afdd18d6402f6c2aa0b6a342618c8251a473ee61 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] MartijnVisser merged pull request #19994: [FLINK-28060][Connector/Kafka] Updated Kafka Clients to 3.1.1
Posted by GitBox <gi...@apache.org>.
MartijnVisser merged PR #19994:
URL: https://github.com/apache/flink/pull/19994
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org