You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by GitBox <gi...@apache.org> on 2021/04/24 22:16:48 UTC
[GitHub] [spark] dongjoon-hyun opened a new pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
dongjoon-hyun opened a new pull request #32325:
URL: https://github.com/apache/spark/pull/32325
### What changes were proposed in this pull request?
This PR aims to upgrade Kafka client to 2.8.0.
### Why are the changes needed?
This will bring the latest client-side improvement and bug fixes like the following examples.
- KAFKA-10631 ProducerFencedException is not Handled on Offest Commit
- KAFKA-10134 High CPU issue during rebalance in Kafka consumer after upgrading to 2.5
- KAFKA-12193 Re-resolve IPs when a client is disconnected
- KAFKA-10090 Misleading warnings: The configuration was supplied but isn't a known config
- KAFKA-9263 The new hw is added to incorrect log when ReplicaAlterLogDirsThread is replacing log
- KAFKA-10607 Ensure the error counts contains the NONE
- KAFKA-10458 Need a way to update quota for TokenBucket registered with Sensor
- KAFKA-9587 Producer configs are omitted in the documentation
- KAFKA-10503 MockProducer doesn't throw ClassCastException when no partition for topic
**RELEASE NOTE**
- https://downloads.apache.org/kafka/2.8.0/RELEASE_NOTES.html
- https://downloads.apache.org/kafka/2.7.0/RELEASE_NOTES.html
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the CIs with the existing tests because this is a dependency change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826165955
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42420/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826165545
Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42420/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826354574
Thank you so much, @viirya and @HyukjinKwon !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826179404
**[Test build #137895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137895/testReport)** for PR 32325 at commit [`76ad092`](https://github.com/apache/spark/commit/76ad0928ee428c37b4dbdd3898015ee4e9a6a42a).
* This patch passes all tests.
* This patch merges cleanly.
* This patch adds no public classes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826165955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826160828
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #32325:
URL: https://github.com/apache/spark/pull/32325
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826160828
**[Test build #137895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137895/testReport)** for PR 32325 at commit [`76ad092`](https://github.com/apache/spark/commit/76ad0928ee428c37b4dbdd3898015ee4e9a6a42a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826188590
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137895/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826230009
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826188590
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137895/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA removed a comment on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826160828
**[Test build #137895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137895/testReport)** for PR 32325 at commit [`76ad092`](https://github.com/apache/spark/commit/76ad0928ee428c37b4dbdd3898015ee4e9a6a42a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826165955
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42420/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
dongjoon-hyun commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826230009
Could you review this, @viirya ? This is for Structured Streaming and ZSTD dependency improvement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826272877
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826165949
Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42420/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
SparkQA commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826160828
**[Test build #137895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137895/testReport)** for PR 32325 at commit [`76ad092`](https://github.com/apache/spark/commit/76ad0928ee428c37b4dbdd3898015ee4e9a6a42a).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon closed pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
HyukjinKwon closed pull request #32325:
URL: https://github.com/apache/spark/pull/32325
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
AmplabJenkins removed a comment on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826165955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
[GitHub] [spark] HyukjinKwon commented on pull request #32325: [SPARK-33913][SS] Upgrade Kafka to 2.8.0
Posted by GitBox <gi...@apache.org>.
HyukjinKwon commented on pull request #32325:
URL: https://github.com/apache/spark/pull/32325#issuecomment-826272877
Merged to master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org