You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by James Ziesig <ja...@broadcom.com.INVALID> on 2022/09/15 19:55:05 UTC
mTLS client hostname validation is not working
Hi,
I have configured mTLS on a three server Kafka cluster. The servers and
clients are all communicating properly, except I am having trouble with
client hostname validation when the client is using a cert from a different
host. I would expect this to fail on handshake like it does when the certs
are expired, but I have found that clients can use any cert signed by the
trusted CA regardless of whether the cert matches their hostname/IP. The
certs are generated from an intermediate CA in Vault, and the server and
client settings are configured via PEM properties. The intermediate CA
cert is the only cert in ssl.truststore.certificates.
Example server config (ssl settings only):
listeners=SSL://server-a.b.net:9092
advertised.listeners=SSL://server-a.b.net:9092
security.inter.broker.protocol=SSL
ssl.client.auth=required
#Default is HTTPS, but try explicitly setting it
ssl.endpoint.identification.algorithm=HTTPS
security.protocol=SSL
ssl.keystore.type=PEM
ssl.truststore.type=PEM
ssl.key.password=mypassword
ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<server private key
info>\n...-----END ENCRYPTED PRIVATE KEY-----
ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<server
cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
ssl.truststore.certificates=-----BEGIN
CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
Example client config (ssl settings only):
security.protocol=SSL
ssl.keystore.type=PEM
ssl.truststore.type=PEM
ssl.key.password=mypassword
#Default is HTTPS, but try explicitly setting it
ssl.endpoint.identification.algorithm=HTTPS
ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<client-1 private
key info>\n...-----END ENCRYPTED PRIVATE KEY-----
ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<client-1
cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
ssl.truststore.certificates=-----BEGIN
CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
I have two properties files with certs for client-1.b.net and client-2.b.net
(producer.properties.client1 and producer.properties.client2 respectively).
On client-1.b.net, I can use either of those properties files and
successfully connect and send data to the cluster using
kafka-console-producer.sh:
./kafka-console-producer.sh --bootstrap-server server-a.b.net:9092 --topic
foo --producer.config producer.properties.client2
I have extracted the \n concatenated string out to a file, and verified
that the CN and SAN include only the FQDN and IP of client-2.b.net.
Am I missing a configuration setting somewhere, or is this broken?
Thanks in advance,
Jim Ziesig
--
This electronic communication and the information and any files transmitted
with it, or attached to it, are confidential and are intended solely for
the use of the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or otherwise restricted from disclosure to anyone else. If you are
not the intended recipient or the person responsible for delivering the
e-mail to the intended recipient, you are hereby notified that any use,
copying, distributing, dissemination, forwarding, printing, or copying of
this e-mail is strictly prohibited. If you received this e-mail in error,
please return the e-mail to the sender, delete it from your computer, and
destroy any printed copy of it.
Re: mTLS client hostname validation is not working
Posted by Richard Bosch <ri...@axual.com>.
You're very welcome, and good luck with your installation.
Kind regards,
Richard Bosch
On Fri, 16 Sep 2022, 17:56 James Ziesig, <ja...@broadcom.com.invalid>
wrote:
> Hi Richard,
>
> Thank you for the response. That does clear things up for me. I thought
> it would be good to avoid MITM attacks by validating SAN, if CN is a
> service identifier, without the need for ACLs. However, I do appreciate
> the added flexibility provided by the ACLs, and the lack of client hostname
> validation on handshake simplifies the client cert deployment.
>
> Thanks again, and have a nice weekend.
>
> Jim Ziesig
>
> On Fri, Sep 16, 2022 at 4:31 AM Richard Bosch <ri...@axual.com>
> wrote:
>
>> Hi Jim,
>>
>> The broker setting for endpoint identification is used when a broker
>> connects to another broker.
>> For client connection the handshake is performed by verifying that the
>> certificate presented by the client is signed by a CA that's in the
>> truststore, and that it hasn't expired yet.
>>
>> If you want to limit access to resources on the location, you can set up
>> ACLs to match on the reported hostname of the client as well.
>> This allows you to define an ACL with a meaning like this: Principal
>> CN=MyClientCert has Consume access for Topic Y if it connects from Host
>> 10.1.2.100
>>
>> I hope this helps you to understand the Kafka Broker setup a bit better.
>>
>> Kind regards,
>>
>>
>> Richard Bosch
>>
>> Developer Advocate
>>
>> Axual BV
>>
>> https://axual.com/
>>
>>
>> On Thu, Sep 15, 2022 at 9:55 PM James Ziesig
>> <ja...@broadcom.com.invalid> wrote:
>>
>> > Hi,
>> >
>> > I have configured mTLS on a three server Kafka cluster. The servers and
>> > clients are all communicating properly, except I am having trouble with
>> > client hostname validation when the client is using a cert from a
>> different
>> > host. I would expect this to fail on handshake like it does when the
>> certs
>> > are expired, but I have found that clients can use any cert signed by
>> the
>> > trusted CA regardless of whether the cert matches their hostname/IP.
>> The
>> > certs are generated from an intermediate CA in Vault, and the server and
>> > client settings are configured via PEM properties. The intermediate CA
>> > cert is the only cert in ssl.truststore.certificates.
>> >
>> > Example server config (ssl settings only):
>> > listeners=SSL://server-a.b.net:9092
>> > advertised.listeners=SSL://server-a.b.net:9092
>> > security.inter.broker.protocol=SSL
>> > ssl.client.auth=required
>> > #Default is HTTPS, but try explicitly setting it
>> > ssl.endpoint.identification.algorithm=HTTPS
>> > security.protocol=SSL
>> > ssl.keystore.type=PEM
>> > ssl.truststore.type=PEM
>> > ssl.key.password=mypassword
>> > ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<server private
>> > key info>\n...-----END ENCRYPTED PRIVATE KEY-----
>> > ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<server
>> > cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
>> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
>> > ssl.truststore.certificates=-----BEGIN
>> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
>> >
>> > Example client config (ssl settings only):
>> > security.protocol=SSL
>> > ssl.keystore.type=PEM
>> > ssl.truststore.type=PEM
>> > ssl.key.password=mypassword
>> > #Default is HTTPS, but try explicitly setting it
>> > ssl.endpoint.identification.algorithm=HTTPS
>> > ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<client-1
>> private
>> > key info>\n...-----END ENCRYPTED PRIVATE KEY-----
>> > ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<client-1
>> > cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
>> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
>> > ssl.truststore.certificates=-----BEGIN
>> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
>> >
>> > I have two properties files with certs for client-1.b.net and
>> > client-2.b.net (producer.properties.client1 and
>> > producer.properties.client2 respectively).
>> > On client-1.b.net, I can use either of those properties files and
>> > successfully connect and send data to the cluster using
>> > kafka-console-producer.sh:
>> > ./kafka-console-producer.sh --bootstrap-server server-a.b.net:9092
>> > --topic foo --producer.config producer.properties.client2
>> >
>> > I have extracted the \n concatenated string out to a file, and verified
>> > that the CN and SAN include only the FQDN and IP of client-2.b.net.
>> >
>> > Am I missing a configuration setting somewhere, or is this broken?
>> >
>> > Thanks in advance,
>> >
>> > Jim Ziesig
>> >
>> > This electronic communication and the information and any files
>> > transmitted with it, or attached to it, are confidential and are
>> intended
>> > solely for the use of the individual or entity to whom it is addressed
>> and
>> > may contain information that is confidential, legally privileged,
>> protected
>> > by privacy laws, or otherwise restricted from disclosure to anyone
>> else. If
>> > you are not the intended recipient or the person responsible for
>> delivering
>> > the e-mail to the intended recipient, you are hereby notified that any
>> use,
>> > copying, distributing, dissemination, forwarding, printing, or copying
>> of
>> > this e-mail is strictly prohibited. If you received this e-mail in
>> error,
>> > please return the e-mail to the sender, delete it from your computer,
>> and
>> > destroy any printed copy of it.
>>
>
> This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.
Re: mTLS client hostname validation is not working
Posted by James Ziesig <ja...@broadcom.com.INVALID>.
Hi Richard,
Thank you for the response. That does clear things up for me. I thought
it would be good to avoid MITM attacks by validating SAN, if CN is a
service identifier, without the need for ACLs. However, I do appreciate
the added flexibility provided by the ACLs, and the lack of client hostname
validation on handshake simplifies the client cert deployment.
Thanks again, and have a nice weekend.
Jim Ziesig
On Fri, Sep 16, 2022 at 4:31 AM Richard Bosch <ri...@axual.com>
wrote:
> Hi Jim,
>
> The broker setting for endpoint identification is used when a broker
> connects to another broker.
> For client connection the handshake is performed by verifying that the
> certificate presented by the client is signed by a CA that's in the
> truststore, and that it hasn't expired yet.
>
> If you want to limit access to resources on the location, you can set up
> ACLs to match on the reported hostname of the client as well.
> This allows you to define an ACL with a meaning like this: Principal
> CN=MyClientCert has Consume access for Topic Y if it connects from Host
> 10.1.2.100
>
> I hope this helps you to understand the Kafka Broker setup a bit better.
>
> Kind regards,
>
>
> Richard Bosch
>
> Developer Advocate
>
> Axual BV
>
> https://axual.com/
>
>
> On Thu, Sep 15, 2022 at 9:55 PM James Ziesig
> <ja...@broadcom.com.invalid> wrote:
>
> > Hi,
> >
> > I have configured mTLS on a three server Kafka cluster. The servers and
> > clients are all communicating properly, except I am having trouble with
> > client hostname validation when the client is using a cert from a
> different
> > host. I would expect this to fail on handshake like it does when the
> certs
> > are expired, but I have found that clients can use any cert signed by the
> > trusted CA regardless of whether the cert matches their hostname/IP. The
> > certs are generated from an intermediate CA in Vault, and the server and
> > client settings are configured via PEM properties. The intermediate CA
> > cert is the only cert in ssl.truststore.certificates.
> >
> > Example server config (ssl settings only):
> > listeners=SSL://server-a.b.net:9092
> > advertised.listeners=SSL://server-a.b.net:9092
> > security.inter.broker.protocol=SSL
> > ssl.client.auth=required
> > #Default is HTTPS, but try explicitly setting it
> > ssl.endpoint.identification.algorithm=HTTPS
> > security.protocol=SSL
> > ssl.keystore.type=PEM
> > ssl.truststore.type=PEM
> > ssl.key.password=mypassword
> > ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<server private
> > key info>\n...-----END ENCRYPTED PRIVATE KEY-----
> > ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<server
> > cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
> > ssl.truststore.certificates=-----BEGIN
> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
> >
> > Example client config (ssl settings only):
> > security.protocol=SSL
> > ssl.keystore.type=PEM
> > ssl.truststore.type=PEM
> > ssl.key.password=mypassword
> > #Default is HTTPS, but try explicitly setting it
> > ssl.endpoint.identification.algorithm=HTTPS
> > ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<client-1 private
> > key info>\n...-----END ENCRYPTED PRIVATE KEY-----
> > ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<client-1
> > cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
> > ssl.truststore.certificates=-----BEGIN
> > CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
> >
> > I have two properties files with certs for client-1.b.net and
> > client-2.b.net (producer.properties.client1 and
> > producer.properties.client2 respectively).
> > On client-1.b.net, I can use either of those properties files and
> > successfully connect and send data to the cluster using
> > kafka-console-producer.sh:
> > ./kafka-console-producer.sh --bootstrap-server server-a.b.net:9092
> > --topic foo --producer.config producer.properties.client2
> >
> > I have extracted the \n concatenated string out to a file, and verified
> > that the CN and SAN include only the FQDN and IP of client-2.b.net.
> >
> > Am I missing a configuration setting somewhere, or is this broken?
> >
> > Thanks in advance,
> >
> > Jim Ziesig
> >
> > This electronic communication and the information and any files
> > transmitted with it, or attached to it, are confidential and are intended
> > solely for the use of the individual or entity to whom it is addressed
> and
> > may contain information that is confidential, legally privileged,
> protected
> > by privacy laws, or otherwise restricted from disclosure to anyone else.
> If
> > you are not the intended recipient or the person responsible for
> delivering
> > the e-mail to the intended recipient, you are hereby notified that any
> use,
> > copying, distributing, dissemination, forwarding, printing, or copying of
> > this e-mail is strictly prohibited. If you received this e-mail in error,
> > please return the e-mail to the sender, delete it from your computer, and
> > destroy any printed copy of it.
>
--
This electronic communication and the information and any files transmitted
with it, or attached to it, are confidential and are intended solely for
the use of the individual or entity to whom it is addressed and may contain
information that is confidential, legally privileged, protected by privacy
laws, or otherwise restricted from disclosure to anyone else. If you are
not the intended recipient or the person responsible for delivering the
e-mail to the intended recipient, you are hereby notified that any use,
copying, distributing, dissemination, forwarding, printing, or copying of
this e-mail is strictly prohibited. If you received this e-mail in error,
please return the e-mail to the sender, delete it from your computer, and
destroy any printed copy of it.
Re: mTLS client hostname validation is not working
Posted by Richard Bosch <ri...@axual.com>.
Hi Jim,
The broker setting for endpoint identification is used when a broker
connects to another broker.
For client connection the handshake is performed by verifying that the
certificate presented by the client is signed by a CA that's in the
truststore, and that it hasn't expired yet.
If you want to limit access to resources on the location, you can set up
ACLs to match on the reported hostname of the client as well.
This allows you to define an ACL with a meaning like this: Principal
CN=MyClientCert has Consume access for Topic Y if it connects from Host
10.1.2.100
I hope this helps you to understand the Kafka Broker setup a bit better.
Kind regards,
Richard Bosch
Developer Advocate
Axual BV
https://axual.com/
On Thu, Sep 15, 2022 at 9:55 PM James Ziesig
<ja...@broadcom.com.invalid> wrote:
> Hi,
>
> I have configured mTLS on a three server Kafka cluster. The servers and
> clients are all communicating properly, except I am having trouble with
> client hostname validation when the client is using a cert from a different
> host. I would expect this to fail on handshake like it does when the certs
> are expired, but I have found that clients can use any cert signed by the
> trusted CA regardless of whether the cert matches their hostname/IP. The
> certs are generated from an intermediate CA in Vault, and the server and
> client settings are configured via PEM properties. The intermediate CA
> cert is the only cert in ssl.truststore.certificates.
>
> Example server config (ssl settings only):
> listeners=SSL://server-a.b.net:9092
> advertised.listeners=SSL://server-a.b.net:9092
> security.inter.broker.protocol=SSL
> ssl.client.auth=required
> #Default is HTTPS, but try explicitly setting it
> ssl.endpoint.identification.algorithm=HTTPS
> security.protocol=SSL
> ssl.keystore.type=PEM
> ssl.truststore.type=PEM
> ssl.key.password=mypassword
> ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<server private
> key info>\n...-----END ENCRYPTED PRIVATE KEY-----
> ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<server
> cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
> CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
> ssl.truststore.certificates=-----BEGIN
> CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
>
> Example client config (ssl settings only):
> security.protocol=SSL
> ssl.keystore.type=PEM
> ssl.truststore.type=PEM
> ssl.key.password=mypassword
> #Default is HTTPS, but try explicitly setting it
> ssl.endpoint.identification.algorithm=HTTPS
> ssl.keystore.key=-----BEGIN ENCRYPTED PRIVATE KEY-----\n<client-1 private
> key info>\n...-----END ENCRYPTED PRIVATE KEY-----
> ssl.keystore.certificate.chain=-----BEGIN CERTIFICATE-----\n<client-1
> cert>\n...\n-----END CERTIFICATE-----\n-----BEGIN
> CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
> ssl.truststore.certificates=-----BEGIN
> CERTIFICATE-----\n<int ca info>\n...n-----END CERTIFICATE-----
>
> I have two properties files with certs for client-1.b.net and
> client-2.b.net (producer.properties.client1 and
> producer.properties.client2 respectively).
> On client-1.b.net, I can use either of those properties files and
> successfully connect and send data to the cluster using
> kafka-console-producer.sh:
> ./kafka-console-producer.sh --bootstrap-server server-a.b.net:9092
> --topic foo --producer.config producer.properties.client2
>
> I have extracted the \n concatenated string out to a file, and verified
> that the CN and SAN include only the FQDN and IP of client-2.b.net.
>
> Am I missing a configuration setting somewhere, or is this broken?
>
> Thanks in advance,
>
> Jim Ziesig
>
> This electronic communication and the information and any files
> transmitted with it, or attached to it, are confidential and are intended
> solely for the use of the individual or entity to whom it is addressed and
> may contain information that is confidential, legally privileged, protected
> by privacy laws, or otherwise restricted from disclosure to anyone else. If
> you are not the intended recipient or the person responsible for delivering
> the e-mail to the intended recipient, you are hereby notified that any use,
> copying, distributing, dissemination, forwarding, printing, or copying of
> this e-mail is strictly prohibited. If you received this e-mail in error,
> please return the e-mail to the sender, delete it from your computer, and
> destroy any printed copy of it.