You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Dima Brodsky <dd...@gmail.com> on 2023/12/05 15:04:09 UTC
In Kafka KRaft can controllers participate as bootstrap servers
Hello, question,
If I have my kafka cluster behind a VIP for bootstrapping, is it possible
to have the controllers participate in the bootstrap process or only
brokers can?
Thanks!
ttyl
Dima
--
ddbrodsky@gmail.com
"The price of reliability is the pursuit of the utmost simplicity.
It is a price which the very rich find the most hard to pay."
(Sir
Antony Hoare, 1980)
Re: In Kafka KRaft can controllers participate as bootstrap servers
Posted by Vikram Singh <vi...@clouzersolutions.com.INVALID>.
Hello,
Why am I getting below logs continuously, how can I configure to avoid this.
11:32:43.469 WARN org.apache.kafka.clients.producer.internals.Sender -
[Producer clientId=javaproducer11703811766106] Received invalid metadata
error in produce request on partition
CCR_CLZ_COM_STATUS_MONITOR-PREDEV_123-0 due to
org.apache.kafka.common.errors.NotLeaderOrFollowerException: For requests
intended only for the leader, this error indicates that the broker is not
the current leader. For requests intended for any replica, this error
indicates that the broker is not a replica of the topic partition.. Going
to request metadata update now
On Thu, Dec 14, 2023 at 1:50 AM David Arthur
<da...@confluent.io.invalid> wrote:
> Only brokers can be specified as --bootstrap-servers for AdminClient (the
> bin/kafka-* scripts).
>
> In 3.7, we are adding the ability to bootstrap from KRaft controllers for
> certain scripts. In this case, the scripts will use --bootstrap-controllers
> (the details are in
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-919%3A+Allow+AdminClient+to+Talk+Directly+with+the+KRaft+Controller+Quorum+and+add+Controller+Registration
> )
>
> But in general, no controllers cannot be used as bootstrap servers.
>
> -David
>
> On Tue, Dec 5, 2023 at 10:05 AM Dima Brodsky <dd...@gmail.com> wrote:
>
> > Hello, question,
> >
> > If I have my kafka cluster behind a VIP for bootstrapping, is it possible
> > to have the controllers participate in the bootstrap process or only
> > brokers can?
> >
> > Thanks!
> > ttyl
> > Dima
> >
> > --
> > ddbrodsky@gmail.com
> >
> > "The price of reliability is the pursuit of the utmost simplicity.
> > It is a price which the very rich find the most hard to pay."
> > (Sir
> > Antony Hoare, 1980)
> >
>
>
> --
> -David
>
--
Thanks & Regards
*VIKRAM S SINGH*
Re: In Kafka KRaft can controllers participate as bootstrap servers
Posted by David Arthur <da...@confluent.io.INVALID>.
Only brokers can be specified as --bootstrap-servers for AdminClient (the
bin/kafka-* scripts).
In 3.7, we are adding the ability to bootstrap from KRaft controllers for
certain scripts. In this case, the scripts will use --bootstrap-controllers
(the details are in
https://cwiki.apache.org/confluence/display/KAFKA/KIP-919%3A+Allow+AdminClient+to+Talk+Directly+with+the+KRaft+Controller+Quorum+and+add+Controller+Registration
)
But in general, no controllers cannot be used as bootstrap servers.
-David
On Tue, Dec 5, 2023 at 10:05 AM Dima Brodsky <dd...@gmail.com> wrote:
> Hello, question,
>
> If I have my kafka cluster behind a VIP for bootstrapping, is it possible
> to have the controllers participate in the bootstrap process or only
> brokers can?
>
> Thanks!
> ttyl
> Dima
>
> --
> ddbrodsky@gmail.com
>
> "The price of reliability is the pursuit of the utmost simplicity.
> It is a price which the very rich find the most hard to pay."
> (Sir
> Antony Hoare, 1980)
>
--
-David
Re: Can a controller in a kafka kraft cluster be a bootstrap server
Posted by Vikram Singh <vi...@clouzersolutions.com.INVALID>.
Hello,
Why am I getting below logs continuously, how can I configure to avoid this.
11:32:43.469 WARN org.apache.kafka.clients.producer.internals.Sender -
[Producer clientId=javaproducer11703811766106] Received invalid metadata
error in produce request on partition
CCR_CLZ_COM_STATUS_MONITOR-PREDEV_123-0 due to
org.apache.kafka.common.errors.NotLeaderOrFollowerException: For requests
intended only for the leader, this error indicates that the broker is not
the current leader. For requests intended for any replica, this error
indicates that the broker is not a replica of the topic partition.. Going
to request metadata update now
On Wed, Dec 13, 2023 at 11:51 AM Vikram Singh <
vikram.singh@clouzersolutions.com> wrote:
> Hello Luke,
> Please look into below logs,
>
> 12:32:15.813 WARN org.apache.kafka.clients.NetworkClient - [AdminClient
> clientId=test-lgn-clz-com-v0.0.43-INS_CLZ_COM-TEST_123-7e87eb30-69e9-4746-b351-beac3a085383-admin]
> Error connecting to node kafka-0-0.kafka.test.svc.cluster.local:9092 (id: 0
> rack: null)
> java.net.UnknownHostException: kafka-0-0.kafka.test.svc.cluster.local
> at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[?:1.8.0_212]
> at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[?:1.8.0_212]
> at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[?:1.8.0_212]
> at
> org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
> ~[kafka-clients-3.4.1.jar:?]
> at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
> ~[kafka-clients-3.4.1.jar:?]
> at
> org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
> ~[kafka-clients-3.4.1.jar:?]
>
>
>
> 12:36:42.366 ERROR
> org.apache.kafka.streams.processor.internals.TaskExecutor - stream-thread
> [test-lgn-clz-com-v0.0.43-INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-61549c35-df2a-4b55-bb27-c2eb49855944-StreamThread-2]
> Committing task(s) 0_1 failed.
> org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms
> expired before successfully committing offsets
> {INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-1=OffsetAndMetadata{offset=329,
> leaderEpoch=null, metadata='AgAAAYxY3+k+'}}
> 12:37:12.814 WARN
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator -
> [Consumer
> clientId=test-lgn-clz-com-v0.0.43-INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-61549c35-df2a-4b55-bb27-c2eb49855944-StreamThread-2-consumer,
> groupId=test-lgn-clz-com-v0.0.43-INS_CLZ_COM_BCKGRND_PROCESS-TEST_123]
> Offset commit failed on partition INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-1 at
> offset 329: This is not the correct coordinator.
>
> On Wed, Dec 13, 2023 at 11:40 AM Luke Chen <sh...@gmail.com> wrote:
>
>> Hi Vikram,
>>
>> It would be good if you could share client and broker logs for
>> troubleshooting.
>>
>> Thanks.
>> Luke
>>
>> On Wed, Dec 13, 2023 at 1:15 PM Vikram Singh
>> <vi...@clouzersolutions.com.invalid> wrote:
>> >
>> > Hello,
>> > I have 3 node kafka cluster, when one node goes down for some reason the
>> > request which are serving on down node is not routing to other running
>> > node, It takes me always to restart the services,
>> > Running on kafka version 3.2.1 (kraft mode)
>> >
>> > On Mon, Dec 11, 2023 at 12:33 PM Luke Chen <sh...@gmail.com> wrote:
>> >
>> > > Hi Dima,
>> > >
>> > > You can set "process.roles=controller,broker" to get what you want.
>> > > Otherwise, the controller role cannot be served as a broker.
>> > >
>> > > Thanks.
>> > > Luke
>> > >
>> > > On Sat, Dec 9, 2023 at 3:59 AM Dima Brodsky <dd...@gmail.com>
>> wrote:
>> > >
>> > > > Hello,
>> > > >
>> > > > Would the following configuration be valid in a kafka kraft cluster
>> > > >
>> > > > So lets say we had the following configs for a controller and a
>> broker:
>> > > >
>> > > > === controller -
>> > > >
>> > > >
>> > >
>> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/controller.properties
>> > > >
>> > > > process.roles=controller
>> > > > node.id=1
>> > > > controller.quorum.voters=1@host1:9093
>> > > > listeners=CONTROLLER://:9093,BROKER://:9092
>> > > > controller.listener.names=CONTROLLER
>> > > >
>> > > > advertised.listeners=BROKER://:9092,CONTROLLER://:9093
>> > > > inter.broker.listener.name=BROKER
>> > > >
>> > > > === broker -
>> > > >
>> > > >
>> > >
>> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/broker.properties
>> > > >
>> > > > process.roles=broker
>> > > > node.id=2
>> > > > controller.quorum.voters=1@host1:9093
>> > > > listeners=BROKER://:9092
>> > > > inter.broker.listener.name=BROKER
>> > > > advertised.listeners=BROKER://:9092
>> > > >
>> > > > The controller, not only advertises itself as a controller but also
>> as a
>> > > > broker. If the controller is contacted by a client as a bootstrap
>> server
>> > > > will the controller be able to perform the duties of a bootstrap
>> server
>> > > or
>> > > > that role has to be left to the brokers. What I have in green has
>> been
>> > > > added to the controller config.
>> > > >
>> > > > Thanks!
>> > > > ttyl
>> > > > Dima
>> > > >
>> > >
>> >
>> >
>> > --
>> > Thanks & Regards
>> > *VIKRAM S SINGH*
>>
>
>
> --
> Thanks & Regards
> *VIKRAM S SINGH*
>
--
Thanks & Regards
*VIKRAM S SINGH*
Re: Can a controller in a kafka kraft cluster be a bootstrap server
Posted by Vikram Singh <vi...@clouzersolutions.com.INVALID>.
Hello Luke,
Please look into below logs,
12:32:15.813 WARN org.apache.kafka.clients.NetworkClient - [AdminClient
clientId=test-lgn-clz-com-v0.0.43-INS_CLZ_COM-TEST_123-7e87eb30-69e9-4746-b351-beac3a085383-admin]
Error connecting to node kafka-0-0.kafka.test.svc.cluster.local:9092 (id: 0
rack: null)
java.net.UnknownHostException: kafka-0-0.kafka.test.svc.cluster.local
at java.net.InetAddress.getAllByName0(InetAddress.java:1281) ~[?:1.8.0_212]
at java.net.InetAddress.getAllByName(InetAddress.java:1193) ~[?:1.8.0_212]
at java.net.InetAddress.getAllByName(InetAddress.java:1127) ~[?:1.8.0_212]
at
org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
~[kafka-clients-3.4.1.jar:?]
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
~[kafka-clients-3.4.1.jar:?]
at
org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
~[kafka-clients-3.4.1.jar:?]
12:36:42.366 ERROR
org.apache.kafka.streams.processor.internals.TaskExecutor - stream-thread
[test-lgn-clz-com-v0.0.43-INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-61549c35-df2a-4b55-bb27-c2eb49855944-StreamThread-2]
Committing task(s) 0_1 failed.
org.apache.kafka.common.errors.TimeoutException: Timeout of 60000ms expired
before successfully committing offsets
{INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-1=OffsetAndMetadata{offset=329,
leaderEpoch=null, metadata='AgAAAYxY3+k+'}}
12:37:12.814 WARN
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator -
[Consumer
clientId=test-lgn-clz-com-v0.0.43-INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-61549c35-df2a-4b55-bb27-c2eb49855944-StreamThread-2-consumer,
groupId=test-lgn-clz-com-v0.0.43-INS_CLZ_COM_BCKGRND_PROCESS-TEST_123]
Offset commit failed on partition INS_CLZ_COM_BCKGRND_PROCESS-TEST_123-1 at
offset 329: This is not the correct coordinator.
On Wed, Dec 13, 2023 at 11:40 AM Luke Chen <sh...@gmail.com> wrote:
> Hi Vikram,
>
> It would be good if you could share client and broker logs for
> troubleshooting.
>
> Thanks.
> Luke
>
> On Wed, Dec 13, 2023 at 1:15 PM Vikram Singh
> <vi...@clouzersolutions.com.invalid> wrote:
> >
> > Hello,
> > I have 3 node kafka cluster, when one node goes down for some reason the
> > request which are serving on down node is not routing to other running
> > node, It takes me always to restart the services,
> > Running on kafka version 3.2.1 (kraft mode)
> >
> > On Mon, Dec 11, 2023 at 12:33 PM Luke Chen <sh...@gmail.com> wrote:
> >
> > > Hi Dima,
> > >
> > > You can set "process.roles=controller,broker" to get what you want.
> > > Otherwise, the controller role cannot be served as a broker.
> > >
> > > Thanks.
> > > Luke
> > >
> > > On Sat, Dec 9, 2023 at 3:59 AM Dima Brodsky <dd...@gmail.com>
> wrote:
> > >
> > > > Hello,
> > > >
> > > > Would the following configuration be valid in a kafka kraft cluster
> > > >
> > > > So lets say we had the following configs for a controller and a
> broker:
> > > >
> > > > === controller -
> > > >
> > > >
> > >
> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/controller.properties
> > > >
> > > > process.roles=controller
> > > > node.id=1
> > > > controller.quorum.voters=1@host1:9093
> > > > listeners=CONTROLLER://:9093,BROKER://:9092
> > > > controller.listener.names=CONTROLLER
> > > >
> > > > advertised.listeners=BROKER://:9092,CONTROLLER://:9093
> > > > inter.broker.listener.name=BROKER
> > > >
> > > > === broker -
> > > >
> > > >
> > >
> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/broker.properties
> > > >
> > > > process.roles=broker
> > > > node.id=2
> > > > controller.quorum.voters=1@host1:9093
> > > > listeners=BROKER://:9092
> > > > inter.broker.listener.name=BROKER
> > > > advertised.listeners=BROKER://:9092
> > > >
> > > > The controller, not only advertises itself as a controller but also
> as a
> > > > broker. If the controller is contacted by a client as a bootstrap
> server
> > > > will the controller be able to perform the duties of a bootstrap
> server
> > > or
> > > > that role has to be left to the brokers. What I have in green has
> been
> > > > added to the controller config.
> > > >
> > > > Thanks!
> > > > ttyl
> > > > Dima
> > > >
> > >
> >
> >
> > --
> > Thanks & Regards
> > *VIKRAM S SINGH*
>
--
Thanks & Regards
*VIKRAM S SINGH*
Re: Can a controller in a kafka kraft cluster be a bootstrap server
Posted by Luke Chen <sh...@gmail.com>.
Hi Vikram,
It would be good if you could share client and broker logs for troubleshooting.
Thanks.
Luke
On Wed, Dec 13, 2023 at 1:15 PM Vikram Singh
<vi...@clouzersolutions.com.invalid> wrote:
>
> Hello,
> I have 3 node kafka cluster, when one node goes down for some reason the
> request which are serving on down node is not routing to other running
> node, It takes me always to restart the services,
> Running on kafka version 3.2.1 (kraft mode)
>
> On Mon, Dec 11, 2023 at 12:33 PM Luke Chen <sh...@gmail.com> wrote:
>
> > Hi Dima,
> >
> > You can set "process.roles=controller,broker" to get what you want.
> > Otherwise, the controller role cannot be served as a broker.
> >
> > Thanks.
> > Luke
> >
> > On Sat, Dec 9, 2023 at 3:59 AM Dima Brodsky <dd...@gmail.com> wrote:
> >
> > > Hello,
> > >
> > > Would the following configuration be valid in a kafka kraft cluster
> > >
> > > So lets say we had the following configs for a controller and a broker:
> > >
> > > === controller -
> > >
> > >
> > https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/controller.properties
> > >
> > > process.roles=controller
> > > node.id=1
> > > controller.quorum.voters=1@host1:9093
> > > listeners=CONTROLLER://:9093,BROKER://:9092
> > > controller.listener.names=CONTROLLER
> > >
> > > advertised.listeners=BROKER://:9092,CONTROLLER://:9093
> > > inter.broker.listener.name=BROKER
> > >
> > > === broker -
> > >
> > >
> > https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/broker.properties
> > >
> > > process.roles=broker
> > > node.id=2
> > > controller.quorum.voters=1@host1:9093
> > > listeners=BROKER://:9092
> > > inter.broker.listener.name=BROKER
> > > advertised.listeners=BROKER://:9092
> > >
> > > The controller, not only advertises itself as a controller but also as a
> > > broker. If the controller is contacted by a client as a bootstrap server
> > > will the controller be able to perform the duties of a bootstrap server
> > or
> > > that role has to be left to the brokers. What I have in green has been
> > > added to the controller config.
> > >
> > > Thanks!
> > > ttyl
> > > Dima
> > >
> >
>
>
> --
> Thanks & Regards
> *VIKRAM S SINGH*
Re: Can a controller in a kafka kraft cluster be a bootstrap server
Posted by Vikram Singh <vi...@clouzersolutions.com.INVALID>.
Hello,
I have 3 node kafka cluster, when one node goes down for some reason the
request which are serving on down node is not routing to other running
node, It takes me always to restart the services,
Running on kafka version 3.2.1 (kraft mode)
On Mon, Dec 11, 2023 at 12:33 PM Luke Chen <sh...@gmail.com> wrote:
> Hi Dima,
>
> You can set "process.roles=controller,broker" to get what you want.
> Otherwise, the controller role cannot be served as a broker.
>
> Thanks.
> Luke
>
> On Sat, Dec 9, 2023 at 3:59 AM Dima Brodsky <dd...@gmail.com> wrote:
>
> > Hello,
> >
> > Would the following configuration be valid in a kafka kraft cluster
> >
> > So lets say we had the following configs for a controller and a broker:
> >
> > === controller -
> >
> >
> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/controller.properties
> >
> > process.roles=controller
> > node.id=1
> > controller.quorum.voters=1@host1:9093
> > listeners=CONTROLLER://:9093,BROKER://:9092
> > controller.listener.names=CONTROLLER
> >
> > advertised.listeners=BROKER://:9092,CONTROLLER://:9093
> > inter.broker.listener.name=BROKER
> >
> > === broker -
> >
> >
> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/broker.properties
> >
> > process.roles=broker
> > node.id=2
> > controller.quorum.voters=1@host1:9093
> > listeners=BROKER://:9092
> > inter.broker.listener.name=BROKER
> > advertised.listeners=BROKER://:9092
> >
> > The controller, not only advertises itself as a controller but also as a
> > broker. If the controller is contacted by a client as a bootstrap server
> > will the controller be able to perform the duties of a bootstrap server
> or
> > that role has to be left to the brokers. What I have in green has been
> > added to the controller config.
> >
> > Thanks!
> > ttyl
> > Dima
> >
>
--
Thanks & Regards
*VIKRAM S SINGH*
Re: Can a controller in a kafka kraft cluster be a bootstrap server
Posted by Luke Chen <sh...@gmail.com>.
Hi Dima,
You can set "process.roles=controller,broker" to get what you want.
Otherwise, the controller role cannot be served as a broker.
Thanks.
Luke
On Sat, Dec 9, 2023 at 3:59 AM Dima Brodsky <dd...@gmail.com> wrote:
> Hello,
>
> Would the following configuration be valid in a kafka kraft cluster
>
> So lets say we had the following configs for a controller and a broker:
>
> === controller -
>
> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/controller.properties
>
> process.roles=controller
> node.id=1
> controller.quorum.voters=1@host1:9093
> listeners=CONTROLLER://:9093,BROKER://:9092
> controller.listener.names=CONTROLLER
>
> advertised.listeners=BROKER://:9092,CONTROLLER://:9093
> inter.broker.listener.name=BROKER
>
> === broker -
>
> https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/broker.properties
>
> process.roles=broker
> node.id=2
> controller.quorum.voters=1@host1:9093
> listeners=BROKER://:9092
> inter.broker.listener.name=BROKER
> advertised.listeners=BROKER://:9092
>
> The controller, not only advertises itself as a controller but also as a
> broker. If the controller is contacted by a client as a bootstrap server
> will the controller be able to perform the duties of a bootstrap server or
> that role has to be left to the brokers. What I have in green has been
> added to the controller config.
>
> Thanks!
> ttyl
> Dima
>
Can a controller in a kafka kraft cluster be a bootstrap server
Posted by Dima Brodsky <dd...@gmail.com>.
Hello,
Would the following configuration be valid in a kafka kraft cluster
So lets say we had the following configs for a controller and a broker:
=== controller -
https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/controller.properties
process.roles=controller
node.id=1
controller.quorum.voters=1@host1:9093
listeners=CONTROLLER://:9093,BROKER://:9092
controller.listener.names=CONTROLLER
advertised.listeners=BROKER://:9092,CONTROLLER://:9093
inter.broker.listener.name=BROKER
=== broker -
https://github.com/apache/kafka/blob/6d1d68617ecd023b787f54aafc24a4232663428d/config/kraft/broker.properties
process.roles=broker
node.id=2
controller.quorum.voters=1@host1:9093
listeners=BROKER://:9092
inter.broker.listener.name=BROKER
advertised.listeners=BROKER://:9092
The controller, not only advertises itself as a controller but also as a
broker. If the controller is contacted by a client as a bootstrap server
will the controller be able to perform the duties of a bootstrap server or
that role has to be left to the brokers. What I have in green has been
added to the controller config.
Thanks!
ttyl
Dima