You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Sivaprakash <si...@gmail.com> on 2018/02/15 11:50:29 UTC
Static IP Configuration
Hi,
Just trying to understand how Zookeeper works with Kafka in recent
versions. Document says Zookeeper need not be connected by Producer and
Subscriber.
How to assign a static IP to the whole cluster (anything like load
balancer) ? I want my producers to use only one IP or domain to publish
data irrespective of Kafka Broker Leader.
Re: Static IP Configuration
Posted by Martin Gainty <mg...@hotmail.com>.
more like amazon supporting multiple-devices from multiple-distribution-sources than mom & pop?
get a CA (certificate authentication) provider ..CA providers protect websites for a living..here is a list
https://www.pluralsight.com/blog/software-development/top-reliable-ssl-certificates
[https://www.pluralsight.com/content/dam/pluralsight/blog/thumbnails/default/blog-thumb-dev-default@2x.png]<https://www.pluralsight.com/blog/software-development/top-reliable-ssl-certificates>
The Top 7 Most Reliable SSL Certificate Providers ...<https://www.pluralsight.com/blog/software-development/top-reliable-ssl-certificates>
www.pluralsight.com
These days, we do most of our stuff online. We pay our bills online, we bank online, meet our friends online, shop online, etc. It is easier, faster and more ...
2 that you'll recognise immediately are verisign and digicert ..
Good Luck!
Martin
______________________________________________
________________________________
From: Sivaprakash <si...@gmail.com>
Sent: Friday, February 16, 2018 2:50 AM
To: users@kafka.apache.org
Subject: Re: Static IP Configuration
My clients (producers) are combination of devices, sensors, application
hence volume of producers would be high. All I want to ensure Broker
receives data from trusted (my own) devices.
On Fri, Feb 16, 2018 at 9:33 AM, Martin Gainty <mg...@hotmail.com> wrote:
> majority of commercial entities (banks/financial-houses) pass this to a CA
> (certifying authority) who will be responsible for
> generating X509 certificates
> generating private key
> generating public key
> generating username (aka alias)
> generating password
>
> this depends on what type of PKI your client wants to setup at the very
> least you need to identify target env:
> which browser?
> which mobile device?
>
> also you will need to be aware of
> selected provider capabilities
> algorithms supported within provider
>
> also are you using Zookeeper?
> in which case will you be implementing either
> Kerberos5
> OR
> MD5
>
> implementing DIGEST-MD5 in ZK you will need to properly configure
> java.security as follows:
>
> MG>$JRE_HOME/lib/security/java.security entries verification :
> > MG>can you verify
> > Context.SECURITY_AUTHENTICATION="DIGEST-MD5"
> ?
>
> MG>can you verify authentication Principal is set to EITHER u as in
> > Context.SECURITY_PRINCIPAL="u: cuser"
> > MG>OR authentication Principal is set to DistinguishedName
> > Context.SECURITY_PRINCIPAL="dn: cn=C. User, ou=NewHires, o=JNDITutorial"
>
> ?
>
> MG>what is value of zookeeper.sasl.client.username System Property ?
>
> Martin
> ________________________________
> From: Sivaprakash <si...@gmail.com>
> Sent: Thursday, February 15, 2018 11:24 AM
> To: users@kafka.apache.org
> Subject: Re: Static IP Configuration
>
> Any general best practice which can be followed in Kafka on this?
>
> On Thu, Feb 15, 2018 at 9:39 PM, Jakub Scholz <ja...@scholz.cz> wrote:
>
> > I'm afraid Kafka will not offer any help with this - at least not as far
> as
> > I know. You have to implement it on your own.
> >
> > Jakub
> >
> > On Thu, Feb 15, 2018 at 3:58 PM, Sivaprakash <sivaprakashshanmugam@gmail.
> > com
> > > wrote:
> >
> > > In this case how authentication keys can be maintained? If I want to
> > change
> > > key/password for my cluster how can I change it in my clients
> (producers)
> > > because they might be sitting in my remote place (sensors/mobile apps)?
> > >
> > > On Thu, Feb 15, 2018 at 7:30 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> > >
> > > > Yes, the clients do not connect to Zookeeper anymore. You can create
> a
> > > > loadbalancer which will point to all your Kafka brokers and use the
> > > address
> > > > of such loadbalancer as the "bootstrap server" in your clients. The
> > > clients
> > > > will use the loadbalancer and connect to one of the Kafka brokers and
> > get
> > > > metadata from this broker. And with these metadata they will connect
> to
> > > the
> > > > different leaders as needed.
> > > >
> > > > Jakub
> > > >
> > > > On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
> > > > sivaprakashshanmugam@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Just trying to understand how Zookeeper works with Kafka in recent
> > > > > versions. Document says Zookeeper need not be connected by Producer
> > and
> > > > > Subscriber.
> > > > >
> > > > > How to assign a static IP to the whole cluster (anything like load
> > > > > balancer) ? I want my producers to use only one IP or domain to
> > publish
> > > > > data irrespective of Kafka Broker Leader.
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > - Prakash.
> > >
> >
>
>
>
> --
> - Prakash.
>
--
- Prakash.
Re: Static IP Configuration
Posted by Sivaprakash <si...@gmail.com>.
My clients (producers) are combination of devices, sensors, application
hence volume of producers would be high. All I want to ensure Broker
receives data from trusted (my own) devices.
On Fri, Feb 16, 2018 at 9:33 AM, Martin Gainty <mg...@hotmail.com> wrote:
> majority of commercial entities (banks/financial-houses) pass this to a CA
> (certifying authority) who will be responsible for
> generating X509 certificates
> generating private key
> generating public key
> generating username (aka alias)
> generating password
>
> this depends on what type of PKI your client wants to setup at the very
> least you need to identify target env:
> which browser?
> which mobile device?
>
> also you will need to be aware of
> selected provider capabilities
> algorithms supported within provider
>
> also are you using Zookeeper?
> in which case will you be implementing either
> Kerberos5
> OR
> MD5
>
> implementing DIGEST-MD5 in ZK you will need to properly configure
> java.security as follows:
>
> MG>$JRE_HOME/lib/security/java.security entries verification :
> > MG>can you verify
> > Context.SECURITY_AUTHENTICATION="DIGEST-MD5"
> ?
>
> MG>can you verify authentication Principal is set to EITHER u as in
> > Context.SECURITY_PRINCIPAL="u: cuser"
> > MG>OR authentication Principal is set to DistinguishedName
> > Context.SECURITY_PRINCIPAL="dn: cn=C. User, ou=NewHires, o=JNDITutorial"
>
> ?
>
> MG>what is value of zookeeper.sasl.client.username System Property ?
>
> Martin
> ________________________________
> From: Sivaprakash <si...@gmail.com>
> Sent: Thursday, February 15, 2018 11:24 AM
> To: users@kafka.apache.org
> Subject: Re: Static IP Configuration
>
> Any general best practice which can be followed in Kafka on this?
>
> On Thu, Feb 15, 2018 at 9:39 PM, Jakub Scholz <ja...@scholz.cz> wrote:
>
> > I'm afraid Kafka will not offer any help with this - at least not as far
> as
> > I know. You have to implement it on your own.
> >
> > Jakub
> >
> > On Thu, Feb 15, 2018 at 3:58 PM, Sivaprakash <sivaprakashshanmugam@gmail.
> > com
> > > wrote:
> >
> > > In this case how authentication keys can be maintained? If I want to
> > change
> > > key/password for my cluster how can I change it in my clients
> (producers)
> > > because they might be sitting in my remote place (sensors/mobile apps)?
> > >
> > > On Thu, Feb 15, 2018 at 7:30 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> > >
> > > > Yes, the clients do not connect to Zookeeper anymore. You can create
> a
> > > > loadbalancer which will point to all your Kafka brokers and use the
> > > address
> > > > of such loadbalancer as the "bootstrap server" in your clients. The
> > > clients
> > > > will use the loadbalancer and connect to one of the Kafka brokers and
> > get
> > > > metadata from this broker. And with these metadata they will connect
> to
> > > the
> > > > different leaders as needed.
> > > >
> > > > Jakub
> > > >
> > > > On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
> > > > sivaprakashshanmugam@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > Just trying to understand how Zookeeper works with Kafka in recent
> > > > > versions. Document says Zookeeper need not be connected by Producer
> > and
> > > > > Subscriber.
> > > > >
> > > > > How to assign a static IP to the whole cluster (anything like load
> > > > > balancer) ? I want my producers to use only one IP or domain to
> > publish
> > > > > data irrespective of Kafka Broker Leader.
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > - Prakash.
> > >
> >
>
>
>
> --
> - Prakash.
>
--
- Prakash.
Re: Static IP Configuration
Posted by Martin Gainty <mg...@hotmail.com>.
majority of commercial entities (banks/financial-houses) pass this to a CA (certifying authority) who will be responsible for
generating X509 certificates
generating private key
generating public key
generating username (aka alias)
generating password
this depends on what type of PKI your client wants to setup at the very least you need to identify target env:
which browser?
which mobile device?
also you will need to be aware of
selected provider capabilities
algorithms supported within provider
also are you using Zookeeper?
in which case will you be implementing either
Kerberos5
OR
MD5
implementing DIGEST-MD5 in ZK you will need to properly configure java.security as follows:
MG>$JRE_HOME/lib/security/java.security entries verification :
> MG>can you verify
> Context.SECURITY_AUTHENTICATION="DIGEST-MD5"
?
MG>can you verify authentication Principal is set to EITHER u as in
> Context.SECURITY_PRINCIPAL="u: cuser"
> MG>OR authentication Principal is set to DistinguishedName
> Context.SECURITY_PRINCIPAL="dn: cn=C. User, ou=NewHires, o=JNDITutorial"
?
MG>what is value of zookeeper.sasl.client.username System Property ?
Martin
________________________________
From: Sivaprakash <si...@gmail.com>
Sent: Thursday, February 15, 2018 11:24 AM
To: users@kafka.apache.org
Subject: Re: Static IP Configuration
Any general best practice which can be followed in Kafka on this?
On Thu, Feb 15, 2018 at 9:39 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> I'm afraid Kafka will not offer any help with this - at least not as far as
> I know. You have to implement it on your own.
>
> Jakub
>
> On Thu, Feb 15, 2018 at 3:58 PM, Sivaprakash <sivaprakashshanmugam@gmail.
> com
> > wrote:
>
> > In this case how authentication keys can be maintained? If I want to
> change
> > key/password for my cluster how can I change it in my clients (producers)
> > because they might be sitting in my remote place (sensors/mobile apps)?
> >
> > On Thu, Feb 15, 2018 at 7:30 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> >
> > > Yes, the clients do not connect to Zookeeper anymore. You can create a
> > > loadbalancer which will point to all your Kafka brokers and use the
> > address
> > > of such loadbalancer as the "bootstrap server" in your clients. The
> > clients
> > > will use the loadbalancer and connect to one of the Kafka brokers and
> get
> > > metadata from this broker. And with these metadata they will connect to
> > the
> > > different leaders as needed.
> > >
> > > Jakub
> > >
> > > On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
> > > sivaprakashshanmugam@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > Just trying to understand how Zookeeper works with Kafka in recent
> > > > versions. Document says Zookeeper need not be connected by Producer
> and
> > > > Subscriber.
> > > >
> > > > How to assign a static IP to the whole cluster (anything like load
> > > > balancer) ? I want my producers to use only one IP or domain to
> publish
> > > > data irrespective of Kafka Broker Leader.
> > > >
> > >
> >
> >
> >
> > --
> > - Prakash.
> >
>
--
- Prakash.
Re: Static IP Configuration
Posted by Sivaprakash <si...@gmail.com>.
Any general best practice which can be followed in Kafka on this?
On Thu, Feb 15, 2018 at 9:39 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> I'm afraid Kafka will not offer any help with this - at least not as far as
> I know. You have to implement it on your own.
>
> Jakub
>
> On Thu, Feb 15, 2018 at 3:58 PM, Sivaprakash <sivaprakashshanmugam@gmail.
> com
> > wrote:
>
> > In this case how authentication keys can be maintained? If I want to
> change
> > key/password for my cluster how can I change it in my clients (producers)
> > because they might be sitting in my remote place (sensors/mobile apps)?
> >
> > On Thu, Feb 15, 2018 at 7:30 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> >
> > > Yes, the clients do not connect to Zookeeper anymore. You can create a
> > > loadbalancer which will point to all your Kafka brokers and use the
> > address
> > > of such loadbalancer as the "bootstrap server" in your clients. The
> > clients
> > > will use the loadbalancer and connect to one of the Kafka brokers and
> get
> > > metadata from this broker. And with these metadata they will connect to
> > the
> > > different leaders as needed.
> > >
> > > Jakub
> > >
> > > On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
> > > sivaprakashshanmugam@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > Just trying to understand how Zookeeper works with Kafka in recent
> > > > versions. Document says Zookeeper need not be connected by Producer
> and
> > > > Subscriber.
> > > >
> > > > How to assign a static IP to the whole cluster (anything like load
> > > > balancer) ? I want my producers to use only one IP or domain to
> publish
> > > > data irrespective of Kafka Broker Leader.
> > > >
> > >
> >
> >
> >
> > --
> > - Prakash.
> >
>
--
- Prakash.
Re: Static IP Configuration
Posted by Jakub Scholz <ja...@scholz.cz>.
I'm afraid Kafka will not offer any help with this - at least not as far as
I know. You have to implement it on your own.
Jakub
On Thu, Feb 15, 2018 at 3:58 PM, Sivaprakash <sivaprakashshanmugam@gmail.com
> wrote:
> In this case how authentication keys can be maintained? If I want to change
> key/password for my cluster how can I change it in my clients (producers)
> because they might be sitting in my remote place (sensors/mobile apps)?
>
> On Thu, Feb 15, 2018 at 7:30 PM, Jakub Scholz <ja...@scholz.cz> wrote:
>
> > Yes, the clients do not connect to Zookeeper anymore. You can create a
> > loadbalancer which will point to all your Kafka brokers and use the
> address
> > of such loadbalancer as the "bootstrap server" in your clients. The
> clients
> > will use the loadbalancer and connect to one of the Kafka brokers and get
> > metadata from this broker. And with these metadata they will connect to
> the
> > different leaders as needed.
> >
> > Jakub
> >
> > On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
> > sivaprakashshanmugam@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > Just trying to understand how Zookeeper works with Kafka in recent
> > > versions. Document says Zookeeper need not be connected by Producer and
> > > Subscriber.
> > >
> > > How to assign a static IP to the whole cluster (anything like load
> > > balancer) ? I want my producers to use only one IP or domain to publish
> > > data irrespective of Kafka Broker Leader.
> > >
> >
>
>
>
> --
> - Prakash.
>
Re: Static IP Configuration
Posted by Sivaprakash <si...@gmail.com>.
In this case how authentication keys can be maintained? If I want to change
key/password for my cluster how can I change it in my clients (producers)
because they might be sitting in my remote place (sensors/mobile apps)?
On Thu, Feb 15, 2018 at 7:30 PM, Jakub Scholz <ja...@scholz.cz> wrote:
> Yes, the clients do not connect to Zookeeper anymore. You can create a
> loadbalancer which will point to all your Kafka brokers and use the address
> of such loadbalancer as the "bootstrap server" in your clients. The clients
> will use the loadbalancer and connect to one of the Kafka brokers and get
> metadata from this broker. And with these metadata they will connect to the
> different leaders as needed.
>
> Jakub
>
> On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
> sivaprakashshanmugam@gmail.com> wrote:
>
> > Hi,
> >
> > Just trying to understand how Zookeeper works with Kafka in recent
> > versions. Document says Zookeeper need not be connected by Producer and
> > Subscriber.
> >
> > How to assign a static IP to the whole cluster (anything like load
> > balancer) ? I want my producers to use only one IP or domain to publish
> > data irrespective of Kafka Broker Leader.
> >
>
--
- Prakash.
Re: Static IP Configuration
Posted by Jakub Scholz <ja...@scholz.cz>.
Yes, the clients do not connect to Zookeeper anymore. You can create a
loadbalancer which will point to all your Kafka brokers and use the address
of such loadbalancer as the "bootstrap server" in your clients. The clients
will use the loadbalancer and connect to one of the Kafka brokers and get
metadata from this broker. And with these metadata they will connect to the
different leaders as needed.
Jakub
On Thu, Feb 15, 2018 at 12:50 PM, Sivaprakash <
sivaprakashshanmugam@gmail.com> wrote:
> Hi,
>
> Just trying to understand how Zookeeper works with Kafka in recent
> versions. Document says Zookeeper need not be connected by Producer and
> Subscriber.
>
> How to assign a static IP to the whole cluster (anything like load
> balancer) ? I want my producers to use only one IP or domain to publish
> data irrespective of Kafka Broker Leader.
>