You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "navina (via GitHub)" <gi...@apache.org> on 2023/04/21 17:34:10 UTC
[GitHub] [pinot] navina opened a new issue, #10664: Upgrade kafka clients to 3.* version
navina opened a new issue, #10664:
URL: https://github.com/apache/pinot/issues/10664
kafka 3.* has some really [good upgrades](https://www.confluent.io/blog/apache-kafka-3-0-major-improvements-and-new-features/) (both on clients and server side). some highlights:
* deprecate support for java 8 and deprecate support for scala 2.12
* producer's default behavior is to have idempotency which is good.
* Client API slightly changed to use CompleteableFuture instead of KafkaFuture -> this might require some code changes. but overall in the right direction
* kip-734 listOffset admin operation returns the offset of the largest offset and timestamp, instead of the next "upcoming" offset -> this is a huge semantic improvement which will also likely trigger changes in our code, although we don't yet use admin client.
Using admin client is one of the changes we should make in pinot , instead of relying on short-lived consumers for metadata fetch operations.
[release notes](https://kafka.apache.org/30/documentation/streams/upgrade-guide) don't mention anything about client compatibility. So I am assuming the compatibility remains with the older clients
Labels: `ingestion`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Upgrade kafka clients to 3.* version [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on issue #10664:
URL: https://github.com/apache/pinot/issues/10664#issuecomment-4113370235
Close as we already have Kafka 3 (https://github.com/apache/pinot/pull/13891) and Kafka 4 (https://github.com/apache/pinot/pull/17633) support now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Upgrade kafka clients to 3.* version [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 closed issue #10664: Upgrade kafka clients to 3.* version
URL: https://github.com/apache/pinot/issues/10664
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org