You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by Men Lim <zu...@gmail.com> on 2021/04/01 01:21:54 UTC
MirrorMaker 2 with SSL
Hello. I was wondering if someone can help answer my question. I'm trying
to run MirrorMaker 2 in distributed mode using SSL. I have the distributor
running in SSL but when I can't get the curl REST api to do so. I saw that
kif-208 fixed this but I can't seem to implement it.
in my mm2-dist.prop file I have set:
////
listeners=https://localhost:8443
security.protocol=SSL
ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
////
my connector.json file look like this:
////
{
"name": "mm2-connect-cluster",
"config":{
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
"connector.client.config.override.policy": "All",
"name": "mm2-connect-cluster",
"topics": "test.*",
"tasks.max": "1",
"source.cluster.alias": "source",
"target.cluster.alias": "target",
"source.cluster.bootstrap.servers": "source:9094",
"target.cluster.bootstrap.servers": "target:9094",
"source->target.enabled": "true",
"target->source.enabled": "false",
"offset-syncs.topic.replication.factor": "4",
"topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
__consumer_offsets",
"groups.blacklist": "console-consumer-.*, connect-.*, __.*",
"topic.creation.enabled": "true",
"topic.creation.default.replication.factor": "4",
"topic.creation.default.partitions": "1"
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"security.protocol": "SSL",
"ssl.truststore.password":
"/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
}
}
////
I would then start up the distributor and it launched fine. So I try to
run the CURl command
////
curl -s -X POST -H 'Content-Type: application/json' --data @connector.json
https://localhost:8443/connectors
////
nada. nothing. no error. no reasons for not starting.
Is it possible to run MM2 with SSL? If so, can someone point me to a
working example?
thanks.
Re: MirrorMaker 2 with SSL
Posted by Men Lim <zu...@gmail.com>.
I got this working late last week. I did the shotgun approach with all the
values in both the distributor and connector.json and once it worked, I
started to remove things to see when it stop working.
On Sun, Apr 18, 2021 at 10:00 PM Ning Zhang <ni...@gmail.com> wrote:
> if the source kafka cluster is SSL-enabled, then the consumer of mm2
> should be configured to read from SSL-enabled cluster
> if the target kafka cluster is SSL-enabled, then the producer of mm2
> should be configured to write to SSL-enabled cluster.
>
> On 2021/04/16 03:23:21, Men Lim <zu...@gmail.com> wrote:
> > well the 405 is due to a syntax error in the connector.json. after
> fixing
> > that, passing the -k switch, it started. but when looking at the
> > connect.log, mm2 is only talking in plaintext rather than SSL. After a
> > while it timed out because the port 9094 is ssl while mm2 is trying to
> use
> > plaintext. at least got past the curl problem.
> >
> > On Wed, Apr 14, 2021 at 12:11 PM Men Lim <zu...@gmail.com> wrote:
> >
> > > I read thru the security_ssl page when I started this, it doesn't apply
> > > much to me because I'm running this in AWS MSK, where I can't access
> the
> > > broker. so my hands are tied there when it come to certificate.
> > >
> > > however, this morning, I decided to work on creating a self sign cert
> for
> > > the CURL command. I was able to get past the ssl handshake error and
> now
> > > running into another issue.
> > >
> > > * Trying 127.0.0.1...
> > > * TCP_NODELAY set
> > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > > * ALPN, offering h2
> > > * ALPN, offering http/1.1
> > > * Cipher selection:
> > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > > * successfully set certificate verify locations:
> > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > > CApath: none
> > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > > * TLSv1.2 (IN), TLS handshake, Server hello (2):
> > > * TLSv1.2 (IN), TLS handshake, Certificate (11):
> > > * TLSv1.2 (OUT), TLS alert, unknown CA (560):
> > >
> > > ** SSL certificate problem: self signed certificate* Closing
> connection 0*
> > >
> > > so I tried passing the -k flag and got the following message
> > >
> > > * Trying 127.0.0.1...
> > > * TCP_NODELAY set
> > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > > * ALPN, offering h2
> > > * ALPN, offering http/1.1
> > > * Cipher selection:
> > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > > * successfully set certificate verify locations:
> > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > > CApath: none
> > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > > * TLSv1.2 (IN), TLS handshake, Server hello (2):
> > > * TLSv1.2 (IN), TLS handshake, Certificate (11):
> > > * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> > > * TLSv1.2 (IN), TLS handshake, Server finished (14):
> > > * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> > > * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
> > > * TLSv1.2 (OUT), TLS handshake, Finished (20):
> > > * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
> > > * TLSv1.2 (IN), TLS handshake, Finished (20):
> > > * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> > > * ALPN, server did not agree to a protocol
> > > * Server certificate:
> > > * subject: C=us; ST=il; L=Unknown; O=usfoods; OU=Unknown; CN=Unknown
> > > * start date: Apr 14 16:52:07 2021 GMT
> > > * expire date: Apr 12 16:52:07 2031 GMT
> > > * issuer: C=US; ST=MI; L=Unknown; O=FUD; OU=Unknown; CN=Unknown
> > > * SSL certificate verify result: self signed certificate (18),
> continuing
> > > anyway.
> > > > PUT /connectors HTTP/1.1
> > > > Host: localhost:8443
> > > > User-Agent: curl/7.61.1
> > > > Accept: */*
> > > > Content-Type: application/json
> > > > Content-Length: 1251
> > > > Expect: 100-continue
> > > >
> > > < HTTP/1.1 100 Continue
> > > * We are completely uploaded and fine
> > > < HTTP/1.1 405 Method Not Allowed
> > > < Date: Wed, 14 Apr 2021 19:07:01 GMT
> > > < Content-Length: 58
> > > < Server: Jetty(9.4.33.v20201020)
> > > <
> > > * Connection #0 to host localhost left intact
> > > *{"error_code":405,"message":"HTTP 405 Method Not Allowed"}*
> > >
> > > with the error, the mm2 process still isn't running. i'm not sure how
> to
> > > go about troubleshooting this HTTP 405 msg. Maybe the cert can't be
> self
> > > signed and need an official one.
> > >
> > >
> > > On Tue, Apr 13, 2021 at 8:41 PM Ning Zhang <ni...@gmail.com>
> wrote:
> > >
> > >> assume your target / destination kafka cluster is SSL enabled. If your
> > >> MM2 wants to write to such cluster, you may have the following config
> in
> > >> your MM2:
> > >>
> > >>
> > >>
> https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
> > >>
> > >> on the broker (even client) side, you may refer to:
> > >>
> > >> http://kafka.apache.org/090/documentation.html#security_ssl
> > >>
> > >> On 2021/04/09 15:01:26, Men Lim <zu...@gmail.com> wrote:
> > >> > Hi Ning,
> > >> >
> > >> > thanks for the response. This self sign cert stays on the ec2
> instance,
> > >> > specifically for the curl command and I don't have to share it with
> the
> > >> > brokers correct?
> > >> >
> > >> > thanks,
> > >> >
> > >> >
> > >> >
> > >> > On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ni...@gmail.com>
> > >> wrote:
> > >> >
> > >> > > Hi Men,
> > >> > >
> > >> > > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a
> try of
> > >> > > self-signing certs and key for testing purpose:
> > >> > > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
> > >> > >
> > >> > > On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
> > >> > > > Hi Ryanne,
> > >> > > >
> > >> > > > thanks for the reply. My kafka clusters are on AWS, their
> > >> serverless
> > >> > > > platform, MSK. I'm stuck with using the default java cacerts
> > >> unless I
> > >> > > use
> > >> > > > their AWS PCA which is pretty pricey.
> > >> > > >
> > >> > > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag
> and
> > >> got
> > >> > > the
> > >> > > > following verbose message:
> > >> > > >
> > >> > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > >> > > @connector.json
> > >> > > > https://localhost:8443/connectors -v --tlsv1.2
> > >> > > > * Trying 127.0.0.1...
> > >> > > > * TCP_NODELAY set
> > >> > > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > >> > > > * ALPN, offering h2
> > >> > > > * ALPN, offering http/1.1
> > >> > > > * Cipher selection:
> > >> > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > >> > > > * successfully set certificate verify locations:
> > >> > > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > >> > > > CApath: none
> > >> > > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > >> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > >> > > > * TLSv1.2 (IN), TLS header, Unknown (21):
> > >> > > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
> > >> > > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
> > >> > > handshake
> > >> > > > failure
> > >> > > >
> > >> > > > Thanks
> > >> > > >
> > >> > > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <
> ryannedolan@gmail.com>
> > >> > > wrote:
> > >> > > >
> > >> > > > > Yes it's possible. The most common issue in my experience is
> the
> > >> > > location
> > >> > > > > of the trust store and key store being different or absent on
> some
> > >> > > hosts.
> > >> > > > > You need to make sure that these locations are consistent
> across
> > >> all
> > >> > > hosts
> > >> > > > > in your Connect cluster, or use a ConfigProvider to provide
> the
> > >> > > location
> > >> > > > > dynamically. Otherwise, a task will get scheduled on some
> host and
> > >> > > fail to
> > >> > > > > find these files.
> > >> > > > >
> > >> > > > > Ryanne
> > >> > > > >
> > >> > > > >
> > >> > > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com>
> wrote:
> > >> > > > >
> > >> > > > > > Hello. I was wondering if someone can help answer my
> > >> question. I'm
> > >> > > > > trying
> > >> > > > > > to run MirrorMaker 2 in distributed mode using SSL. I have
> the
> > >> > > > > distributor
> > >> > > > > > running in SSL but when I can't get the curl REST api to do
> so.
> > >> I saw
> > >> > > > > that
> > >> > > > > > kif-208 fixed this but I can't seem to implement it.
> > >> > > > > >
> > >> > > > > > in my mm2-dist.prop file I have set:
> > >> > > > > > ////
> > >> > > > > > listeners=https://localhost:8443
> > >> > > > > > security.protocol=SSL
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > >
> > >>
> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > >> > > > > > ////
> > >> > > > > > my connector.json file look like this:
> > >> > > > > >
> > >> > > > > > ////
> > >> > > > > > {
> > >> > > > > > "name": "mm2-connect-cluster",
> > >> > > > > > "config":{
> > >> > > > > > "connector.class":
> > >> > > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > >> > > > > > "connector.client.config.override.policy": "All",
> > >> > > > > > "name": "mm2-connect-cluster",
> > >> > > > > > "topics": "test.*",
> > >> > > > > > "tasks.max": "1",
> > >> > > > > > "source.cluster.alias": "source",
> > >> > > > > > "target.cluster.alias": "target",
> > >> > > > > > "source.cluster.bootstrap.servers": "source:9094",
> > >> > > > > > "target.cluster.bootstrap.servers": "target:9094",
> > >> > > > > > "source->target.enabled": "true",
> > >> > > > > > "target->source.enabled": "false",
> > >> > > > > > "offset-syncs.topic.replication.factor": "4",
> > >> > > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > >> > > > > > __consumer_offsets",
> > >> > > > > > "groups.blacklist": "console-consumer-.*,
> connect-.*,
> > >> __.*",
> > >> > > > > > "topic.creation.enabled": "true",
> > >> > > > > > "topic.creation.default.replication.factor": "4",
> > >> > > > > > "topic.creation.default.partitions": "1"
> > >> > > > > > "key.converter":
> > >> > > "org.apache.kafka.connect.json.JsonConverter",
> > >> > > > > > "value.converter":
> > >> > > "org.apache.kafka.connect.json.JsonConverter",
> > >> > > > > > "security.protocol": "SSL",
> > >> > > > > > "ssl.truststore.password":
> > >> > > > > >
> > >> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > >> > > > > > }
> > >> > > > > > }
> > >> > > > > > ////
> > >> > > > > >
> > >> > > > > > I would then start up the distributor and it launched
> fine. So
> > >> I
> > >> > > try to
> > >> > > > > > run the CURl command
> > >> > > > > >
> > >> > > > > > ////
> > >> > > > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > >> > > > > @connector.json
> > >> > > > > > https://localhost:8443/connectors
> > >> > > > > > ////
> > >> > > > > > nada. nothing. no error. no reasons for not starting.
> > >> > > > > >
> > >> > > > > > Is it possible to run MM2 with SSL? If so, can someone
> point
> > >> me to a
> > >> > > > > > working example?
> > >> > > > > >
> > >> > > > > > thanks.
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> >
>
Re: MirrorMaker 2 with SSL
Posted by Ning Zhang <ni...@gmail.com>.
if the source kafka cluster is SSL-enabled, then the consumer of mm2 should be configured to read from SSL-enabled cluster
if the target kafka cluster is SSL-enabled, then the producer of mm2 should be configured to write to SSL-enabled cluster.
On 2021/04/16 03:23:21, Men Lim <zu...@gmail.com> wrote:
> well the 405 is due to a syntax error in the connector.json. after fixing
> that, passing the -k switch, it started. but when looking at the
> connect.log, mm2 is only talking in plaintext rather than SSL. After a
> while it timed out because the port 9094 is ssl while mm2 is trying to use
> plaintext. at least got past the curl problem.
>
> On Wed, Apr 14, 2021 at 12:11 PM Men Lim <zu...@gmail.com> wrote:
>
> > I read thru the security_ssl page when I started this, it doesn't apply
> > much to me because I'm running this in AWS MSK, where I can't access the
> > broker. so my hands are tied there when it come to certificate.
> >
> > however, this morning, I decided to work on creating a self sign cert for
> > the CURL command. I was able to get past the ssl handshake error and now
> > running into another issue.
> >
> > * Trying 127.0.0.1...
> > * TCP_NODELAY set
> > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > * ALPN, offering h2
> > * ALPN, offering http/1.1
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > * successfully set certificate verify locations:
> > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > CApath: none
> > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > * TLSv1.2 (IN), TLS handshake, Server hello (2):
> > * TLSv1.2 (IN), TLS handshake, Certificate (11):
> > * TLSv1.2 (OUT), TLS alert, unknown CA (560):
> >
> > ** SSL certificate problem: self signed certificate* Closing connection 0*
> >
> > so I tried passing the -k flag and got the following message
> >
> > * Trying 127.0.0.1...
> > * TCP_NODELAY set
> > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > * ALPN, offering h2
> > * ALPN, offering http/1.1
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > * successfully set certificate verify locations:
> > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > CApath: none
> > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > * TLSv1.2 (IN), TLS handshake, Server hello (2):
> > * TLSv1.2 (IN), TLS handshake, Certificate (11):
> > * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> > * TLSv1.2 (IN), TLS handshake, Server finished (14):
> > * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> > * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
> > * TLSv1.2 (OUT), TLS handshake, Finished (20):
> > * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
> > * TLSv1.2 (IN), TLS handshake, Finished (20):
> > * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> > * ALPN, server did not agree to a protocol
> > * Server certificate:
> > * subject: C=us; ST=il; L=Unknown; O=usfoods; OU=Unknown; CN=Unknown
> > * start date: Apr 14 16:52:07 2021 GMT
> > * expire date: Apr 12 16:52:07 2031 GMT
> > * issuer: C=US; ST=MI; L=Unknown; O=FUD; OU=Unknown; CN=Unknown
> > * SSL certificate verify result: self signed certificate (18), continuing
> > anyway.
> > > PUT /connectors HTTP/1.1
> > > Host: localhost:8443
> > > User-Agent: curl/7.61.1
> > > Accept: */*
> > > Content-Type: application/json
> > > Content-Length: 1251
> > > Expect: 100-continue
> > >
> > < HTTP/1.1 100 Continue
> > * We are completely uploaded and fine
> > < HTTP/1.1 405 Method Not Allowed
> > < Date: Wed, 14 Apr 2021 19:07:01 GMT
> > < Content-Length: 58
> > < Server: Jetty(9.4.33.v20201020)
> > <
> > * Connection #0 to host localhost left intact
> > *{"error_code":405,"message":"HTTP 405 Method Not Allowed"}*
> >
> > with the error, the mm2 process still isn't running. i'm not sure how to
> > go about troubleshooting this HTTP 405 msg. Maybe the cert can't be self
> > signed and need an official one.
> >
> >
> > On Tue, Apr 13, 2021 at 8:41 PM Ning Zhang <ni...@gmail.com> wrote:
> >
> >> assume your target / destination kafka cluster is SSL enabled. If your
> >> MM2 wants to write to such cluster, you may have the following config in
> >> your MM2:
> >>
> >>
> >> https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
> >>
> >> on the broker (even client) side, you may refer to:
> >>
> >> http://kafka.apache.org/090/documentation.html#security_ssl
> >>
> >> On 2021/04/09 15:01:26, Men Lim <zu...@gmail.com> wrote:
> >> > Hi Ning,
> >> >
> >> > thanks for the response. This self sign cert stays on the ec2 instance,
> >> > specifically for the curl command and I don't have to share it with the
> >> > brokers correct?
> >> >
> >> > thanks,
> >> >
> >> >
> >> >
> >> > On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ni...@gmail.com>
> >> wrote:
> >> >
> >> > > Hi Men,
> >> > >
> >> > > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
> >> > > self-signing certs and key for testing purpose:
> >> > > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
> >> > >
> >> > > On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
> >> > > > Hi Ryanne,
> >> > > >
> >> > > > thanks for the reply. My kafka clusters are on AWS, their
> >> serverless
> >> > > > platform, MSK. I'm stuck with using the default java cacerts
> >> unless I
> >> > > use
> >> > > > their AWS PCA which is pretty pricey.
> >> > > >
> >> > > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and
> >> got
> >> > > the
> >> > > > following verbose message:
> >> > > >
> >> > > > curl -s -X POST -H 'Content-Type: application/json' --data
> >> > > @connector.json
> >> > > > https://localhost:8443/connectors -v --tlsv1.2
> >> > > > * Trying 127.0.0.1...
> >> > > > * TCP_NODELAY set
> >> > > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> >> > > > * ALPN, offering h2
> >> > > > * ALPN, offering http/1.1
> >> > > > * Cipher selection:
> >> > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> >> > > > * successfully set certificate verify locations:
> >> > > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> >> > > > CApath: none
> >> > > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> >> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> >> > > > * TLSv1.2 (IN), TLS header, Unknown (21):
> >> > > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
> >> > > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
> >> > > handshake
> >> > > > failure
> >> > > >
> >> > > > Thanks
> >> > > >
> >> > > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com>
> >> > > wrote:
> >> > > >
> >> > > > > Yes it's possible. The most common issue in my experience is the
> >> > > location
> >> > > > > of the trust store and key store being different or absent on some
> >> > > hosts.
> >> > > > > You need to make sure that these locations are consistent across
> >> all
> >> > > hosts
> >> > > > > in your Connect cluster, or use a ConfigProvider to provide the
> >> > > location
> >> > > > > dynamically. Otherwise, a task will get scheduled on some host and
> >> > > fail to
> >> > > > > find these files.
> >> > > > >
> >> > > > > Ryanne
> >> > > > >
> >> > > > >
> >> > > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
> >> > > > >
> >> > > > > > Hello. I was wondering if someone can help answer my
> >> question. I'm
> >> > > > > trying
> >> > > > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
> >> > > > > distributor
> >> > > > > > running in SSL but when I can't get the curl REST api to do so.
> >> I saw
> >> > > > > that
> >> > > > > > kif-208 fixed this but I can't seem to implement it.
> >> > > > > >
> >> > > > > > in my mm2-dist.prop file I have set:
> >> > > > > > ////
> >> > > > > > listeners=https://localhost:8443
> >> > > > > > security.protocol=SSL
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > >
> >> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> >> > > > > > ////
> >> > > > > > my connector.json file look like this:
> >> > > > > >
> >> > > > > > ////
> >> > > > > > {
> >> > > > > > "name": "mm2-connect-cluster",
> >> > > > > > "config":{
> >> > > > > > "connector.class":
> >> > > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> >> > > > > > "connector.client.config.override.policy": "All",
> >> > > > > > "name": "mm2-connect-cluster",
> >> > > > > > "topics": "test.*",
> >> > > > > > "tasks.max": "1",
> >> > > > > > "source.cluster.alias": "source",
> >> > > > > > "target.cluster.alias": "target",
> >> > > > > > "source.cluster.bootstrap.servers": "source:9094",
> >> > > > > > "target.cluster.bootstrap.servers": "target:9094",
> >> > > > > > "source->target.enabled": "true",
> >> > > > > > "target->source.enabled": "false",
> >> > > > > > "offset-syncs.topic.replication.factor": "4",
> >> > > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> >> > > > > > __consumer_offsets",
> >> > > > > > "groups.blacklist": "console-consumer-.*, connect-.*,
> >> __.*",
> >> > > > > > "topic.creation.enabled": "true",
> >> > > > > > "topic.creation.default.replication.factor": "4",
> >> > > > > > "topic.creation.default.partitions": "1"
> >> > > > > > "key.converter":
> >> > > "org.apache.kafka.connect.json.JsonConverter",
> >> > > > > > "value.converter":
> >> > > "org.apache.kafka.connect.json.JsonConverter",
> >> > > > > > "security.protocol": "SSL",
> >> > > > > > "ssl.truststore.password":
> >> > > > > >
> >> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> >> > > > > > }
> >> > > > > > }
> >> > > > > > ////
> >> > > > > >
> >> > > > > > I would then start up the distributor and it launched fine. So
> >> I
> >> > > try to
> >> > > > > > run the CURl command
> >> > > > > >
> >> > > > > > ////
> >> > > > > > curl -s -X POST -H 'Content-Type: application/json' --data
> >> > > > > @connector.json
> >> > > > > > https://localhost:8443/connectors
> >> > > > > > ////
> >> > > > > > nada. nothing. no error. no reasons for not starting.
> >> > > > > >
> >> > > > > > Is it possible to run MM2 with SSL? If so, can someone point
> >> me to a
> >> > > > > > working example?
> >> > > > > >
> >> > > > > > thanks.
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: MirrorMaker 2 with SSL
Posted by Men Lim <zu...@gmail.com>.
well the 405 is due to a syntax error in the connector.json. after fixing
that, passing the -k switch, it started. but when looking at the
connect.log, mm2 is only talking in plaintext rather than SSL. After a
while it timed out because the port 9094 is ssl while mm2 is trying to use
plaintext. at least got past the curl problem.
On Wed, Apr 14, 2021 at 12:11 PM Men Lim <zu...@gmail.com> wrote:
> I read thru the security_ssl page when I started this, it doesn't apply
> much to me because I'm running this in AWS MSK, where I can't access the
> broker. so my hands are tied there when it come to certificate.
>
> however, this morning, I decided to work on creating a self sign cert for
> the CURL command. I was able to get past the ssl handshake error and now
> running into another issue.
>
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> CApath: none
> * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (OUT), TLS alert, unknown CA (560):
>
> ** SSL certificate problem: self signed certificate* Closing connection 0*
>
> so I tried passing the -k flag and got the following message
>
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> CApath: none
> * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS handshake, Server hello (2):
> * TLSv1.2 (IN), TLS handshake, Certificate (11):
> * TLSv1.2 (IN), TLS handshake, Server key exchange (12):
> * TLSv1.2 (IN), TLS handshake, Server finished (14):
> * TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
> * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
> * TLSv1.2 (OUT), TLS handshake, Finished (20):
> * TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
> * TLSv1.2 (IN), TLS handshake, Finished (20):
> * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
> * ALPN, server did not agree to a protocol
> * Server certificate:
> * subject: C=us; ST=il; L=Unknown; O=usfoods; OU=Unknown; CN=Unknown
> * start date: Apr 14 16:52:07 2021 GMT
> * expire date: Apr 12 16:52:07 2031 GMT
> * issuer: C=US; ST=MI; L=Unknown; O=FUD; OU=Unknown; CN=Unknown
> * SSL certificate verify result: self signed certificate (18), continuing
> anyway.
> > PUT /connectors HTTP/1.1
> > Host: localhost:8443
> > User-Agent: curl/7.61.1
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 1251
> > Expect: 100-continue
> >
> < HTTP/1.1 100 Continue
> * We are completely uploaded and fine
> < HTTP/1.1 405 Method Not Allowed
> < Date: Wed, 14 Apr 2021 19:07:01 GMT
> < Content-Length: 58
> < Server: Jetty(9.4.33.v20201020)
> <
> * Connection #0 to host localhost left intact
> *{"error_code":405,"message":"HTTP 405 Method Not Allowed"}*
>
> with the error, the mm2 process still isn't running. i'm not sure how to
> go about troubleshooting this HTTP 405 msg. Maybe the cert can't be self
> signed and need an official one.
>
>
> On Tue, Apr 13, 2021 at 8:41 PM Ning Zhang <ni...@gmail.com> wrote:
>
>> assume your target / destination kafka cluster is SSL enabled. If your
>> MM2 wants to write to such cluster, you may have the following config in
>> your MM2:
>>
>>
>> https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
>>
>> on the broker (even client) side, you may refer to:
>>
>> http://kafka.apache.org/090/documentation.html#security_ssl
>>
>> On 2021/04/09 15:01:26, Men Lim <zu...@gmail.com> wrote:
>> > Hi Ning,
>> >
>> > thanks for the response. This self sign cert stays on the ec2 instance,
>> > specifically for the curl command and I don't have to share it with the
>> > brokers correct?
>> >
>> > thanks,
>> >
>> >
>> >
>> > On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ni...@gmail.com>
>> wrote:
>> >
>> > > Hi Men,
>> > >
>> > > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
>> > > self-signing certs and key for testing purpose:
>> > > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
>> > >
>> > > On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
>> > > > Hi Ryanne,
>> > > >
>> > > > thanks for the reply. My kafka clusters are on AWS, their
>> serverless
>> > > > platform, MSK. I'm stuck with using the default java cacerts
>> unless I
>> > > use
>> > > > their AWS PCA which is pretty pricey.
>> > > >
>> > > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and
>> got
>> > > the
>> > > > following verbose message:
>> > > >
>> > > > curl -s -X POST -H 'Content-Type: application/json' --data
>> > > @connector.json
>> > > > https://localhost:8443/connectors -v --tlsv1.2
>> > > > * Trying 127.0.0.1...
>> > > > * TCP_NODELAY set
>> > > > * Connected to localhost (127.0.0.1) port 8443 (#0)
>> > > > * ALPN, offering h2
>> > > > * ALPN, offering http/1.1
>> > > > * Cipher selection:
>> > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
>> > > > * successfully set certificate verify locations:
>> > > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
>> > > > CApath: none
>> > > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
>> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
>> > > > * TLSv1.2 (IN), TLS header, Unknown (21):
>> > > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
>> > > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
>> > > handshake
>> > > > failure
>> > > >
>> > > > Thanks
>> > > >
>> > > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com>
>> > > wrote:
>> > > >
>> > > > > Yes it's possible. The most common issue in my experience is the
>> > > location
>> > > > > of the trust store and key store being different or absent on some
>> > > hosts.
>> > > > > You need to make sure that these locations are consistent across
>> all
>> > > hosts
>> > > > > in your Connect cluster, or use a ConfigProvider to provide the
>> > > location
>> > > > > dynamically. Otherwise, a task will get scheduled on some host and
>> > > fail to
>> > > > > find these files.
>> > > > >
>> > > > > Ryanne
>> > > > >
>> > > > >
>> > > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
>> > > > >
>> > > > > > Hello. I was wondering if someone can help answer my
>> question. I'm
>> > > > > trying
>> > > > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
>> > > > > distributor
>> > > > > > running in SSL but when I can't get the curl REST api to do so.
>> I saw
>> > > > > that
>> > > > > > kif-208 fixed this but I can't seem to implement it.
>> > > > > >
>> > > > > > in my mm2-dist.prop file I have set:
>> > > > > > ////
>> > > > > > listeners=https://localhost:8443
>> > > > > > security.protocol=SSL
>> > > > > >
>> > > > > >
>> > > > >
>> > >
>> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
>> > > > > > ////
>> > > > > > my connector.json file look like this:
>> > > > > >
>> > > > > > ////
>> > > > > > {
>> > > > > > "name": "mm2-connect-cluster",
>> > > > > > "config":{
>> > > > > > "connector.class":
>> > > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
>> > > > > > "connector.client.config.override.policy": "All",
>> > > > > > "name": "mm2-connect-cluster",
>> > > > > > "topics": "test.*",
>> > > > > > "tasks.max": "1",
>> > > > > > "source.cluster.alias": "source",
>> > > > > > "target.cluster.alias": "target",
>> > > > > > "source.cluster.bootstrap.servers": "source:9094",
>> > > > > > "target.cluster.bootstrap.servers": "target:9094",
>> > > > > > "source->target.enabled": "true",
>> > > > > > "target->source.enabled": "false",
>> > > > > > "offset-syncs.topic.replication.factor": "4",
>> > > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
>> > > > > > __consumer_offsets",
>> > > > > > "groups.blacklist": "console-consumer-.*, connect-.*,
>> __.*",
>> > > > > > "topic.creation.enabled": "true",
>> > > > > > "topic.creation.default.replication.factor": "4",
>> > > > > > "topic.creation.default.partitions": "1"
>> > > > > > "key.converter":
>> > > "org.apache.kafka.connect.json.JsonConverter",
>> > > > > > "value.converter":
>> > > "org.apache.kafka.connect.json.JsonConverter",
>> > > > > > "security.protocol": "SSL",
>> > > > > > "ssl.truststore.password":
>> > > > > >
>> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
>> > > > > > }
>> > > > > > }
>> > > > > > ////
>> > > > > >
>> > > > > > I would then start up the distributor and it launched fine. So
>> I
>> > > try to
>> > > > > > run the CURl command
>> > > > > >
>> > > > > > ////
>> > > > > > curl -s -X POST -H 'Content-Type: application/json' --data
>> > > > > @connector.json
>> > > > > > https://localhost:8443/connectors
>> > > > > > ////
>> > > > > > nada. nothing. no error. no reasons for not starting.
>> > > > > >
>> > > > > > Is it possible to run MM2 with SSL? If so, can someone point
>> me to a
>> > > > > > working example?
>> > > > > >
>> > > > > > thanks.
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: MirrorMaker 2 with SSL
Posted by Men Lim <zu...@gmail.com>.
I read thru the security_ssl page when I started this, it doesn't apply
much to me because I'm running this in AWS MSK, where I can't access the
broker. so my hands are tied there when it come to certificate.
however, this morning, I decided to work on creating a self sign cert for
the CURL command. I was able to get past the ssl handshake error and now
running into another issue.
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
** SSL certificate problem: self signed certificate* Closing connection 0*
so I tried passing the -k flag and got the following message
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=us; ST=il; L=Unknown; O=usfoods; OU=Unknown; CN=Unknown
* start date: Apr 14 16:52:07 2021 GMT
* expire date: Apr 12 16:52:07 2031 GMT
* issuer: C=US; ST=MI; L=Unknown; O=FUD; OU=Unknown; CN=Unknown
* SSL certificate verify result: self signed certificate (18), continuing
anyway.
> PUT /connectors HTTP/1.1
> Host: localhost:8443
> User-Agent: curl/7.61.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 1251
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 405 Method Not Allowed
< Date: Wed, 14 Apr 2021 19:07:01 GMT
< Content-Length: 58
< Server: Jetty(9.4.33.v20201020)
<
* Connection #0 to host localhost left intact
*{"error_code":405,"message":"HTTP 405 Method Not Allowed"}*
with the error, the mm2 process still isn't running. i'm not sure how to
go about troubleshooting this HTTP 405 msg. Maybe the cert can't be self
signed and need an official one.
On Tue, Apr 13, 2021 at 8:41 PM Ning Zhang <ni...@gmail.com> wrote:
> assume your target / destination kafka cluster is SSL enabled. If your MM2
> wants to write to such cluster, you may have the following config in your
> MM2:
>
>
> https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
>
> on the broker (even client) side, you may refer to:
>
> http://kafka.apache.org/090/documentation.html#security_ssl
>
> On 2021/04/09 15:01:26, Men Lim <zu...@gmail.com> wrote:
> > Hi Ning,
> >
> > thanks for the response. This self sign cert stays on the ec2 instance,
> > specifically for the curl command and I don't have to share it with the
> > brokers correct?
> >
> > thanks,
> >
> >
> >
> > On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ni...@gmail.com>
> wrote:
> >
> > > Hi Men,
> > >
> > > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
> > > self-signing certs and key for testing purpose:
> > > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
> > >
> > > On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
> > > > Hi Ryanne,
> > > >
> > > > thanks for the reply. My kafka clusters are on AWS, their serverless
> > > > platform, MSK. I'm stuck with using the default java cacerts unless
> I
> > > use
> > > > their AWS PCA which is pretty pricey.
> > > >
> > > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and
> got
> > > the
> > > > following verbose message:
> > > >
> > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > > @connector.json
> > > > https://localhost:8443/connectors -v --tlsv1.2
> > > > * Trying 127.0.0.1...
> > > > * TCP_NODELAY set
> > > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > > > * ALPN, offering h2
> > > > * ALPN, offering http/1.1
> > > > * Cipher selection:
> > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > > > * successfully set certificate verify locations:
> > > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > > > CApath: none
> > > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > > > * TLSv1.2 (IN), TLS header, Unknown (21):
> > > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
> > > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
> > > handshake
> > > > failure
> > > >
> > > > Thanks
> > > >
> > > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com>
> > > wrote:
> > > >
> > > > > Yes it's possible. The most common issue in my experience is the
> > > location
> > > > > of the trust store and key store being different or absent on some
> > > hosts.
> > > > > You need to make sure that these locations are consistent across
> all
> > > hosts
> > > > > in your Connect cluster, or use a ConfigProvider to provide the
> > > location
> > > > > dynamically. Otherwise, a task will get scheduled on some host and
> > > fail to
> > > > > find these files.
> > > > >
> > > > > Ryanne
> > > > >
> > > > >
> > > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
> > > > >
> > > > > > Hello. I was wondering if someone can help answer my question.
> I'm
> > > > > trying
> > > > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
> > > > > distributor
> > > > > > running in SSL but when I can't get the curl REST api to do so.
> I saw
> > > > > that
> > > > > > kif-208 fixed this but I can't seem to implement it.
> > > > > >
> > > > > > in my mm2-dist.prop file I have set:
> > > > > > ////
> > > > > > listeners=https://localhost:8443
> > > > > > security.protocol=SSL
> > > > > >
> > > > > >
> > > > >
> > >
> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > > > > > ////
> > > > > > my connector.json file look like this:
> > > > > >
> > > > > > ////
> > > > > > {
> > > > > > "name": "mm2-connect-cluster",
> > > > > > "config":{
> > > > > > "connector.class":
> > > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > > > > > "connector.client.config.override.policy": "All",
> > > > > > "name": "mm2-connect-cluster",
> > > > > > "topics": "test.*",
> > > > > > "tasks.max": "1",
> > > > > > "source.cluster.alias": "source",
> > > > > > "target.cluster.alias": "target",
> > > > > > "source.cluster.bootstrap.servers": "source:9094",
> > > > > > "target.cluster.bootstrap.servers": "target:9094",
> > > > > > "source->target.enabled": "true",
> > > > > > "target->source.enabled": "false",
> > > > > > "offset-syncs.topic.replication.factor": "4",
> > > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > > > > > __consumer_offsets",
> > > > > > "groups.blacklist": "console-consumer-.*, connect-.*,
> __.*",
> > > > > > "topic.creation.enabled": "true",
> > > > > > "topic.creation.default.replication.factor": "4",
> > > > > > "topic.creation.default.partitions": "1"
> > > > > > "key.converter":
> > > "org.apache.kafka.connect.json.JsonConverter",
> > > > > > "value.converter":
> > > "org.apache.kafka.connect.json.JsonConverter",
> > > > > > "security.protocol": "SSL",
> > > > > > "ssl.truststore.password":
> > > > > >
> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > > > > > }
> > > > > > }
> > > > > > ////
> > > > > >
> > > > > > I would then start up the distributor and it launched fine. So I
> > > try to
> > > > > > run the CURl command
> > > > > >
> > > > > > ////
> > > > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > > > > @connector.json
> > > > > > https://localhost:8443/connectors
> > > > > > ////
> > > > > > nada. nothing. no error. no reasons for not starting.
> > > > > >
> > > > > > Is it possible to run MM2 with SSL? If so, can someone point me
> to a
> > > > > > working example?
> > > > > >
> > > > > > thanks.
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: MirrorMaker 2 with SSL
Posted by Ning Zhang <ni...@gmail.com>.
assume your target / destination kafka cluster is SSL enabled. If your MM2 wants to write to such cluster, you may have the following config in your MM2:
https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
on the broker (even client) side, you may refer to:
http://kafka.apache.org/090/documentation.html#security_ssl
On 2021/04/09 15:01:26, Men Lim <zu...@gmail.com> wrote:
> Hi Ning,
>
> thanks for the response. This self sign cert stays on the ec2 instance,
> specifically for the curl command and I don't have to share it with the
> brokers correct?
>
> thanks,
>
>
>
> On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ni...@gmail.com> wrote:
>
> > Hi Men,
> >
> > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
> > self-signing certs and key for testing purpose:
> > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
> >
> > On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
> > > Hi Ryanne,
> > >
> > > thanks for the reply. My kafka clusters are on AWS, their serverless
> > > platform, MSK. I'm stuck with using the default java cacerts unless I
> > use
> > > their AWS PCA which is pretty pricey.
> > >
> > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and got
> > the
> > > following verbose message:
> > >
> > > curl -s -X POST -H 'Content-Type: application/json' --data
> > @connector.json
> > > https://localhost:8443/connectors -v --tlsv1.2
> > > * Trying 127.0.0.1...
> > > * TCP_NODELAY set
> > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > > * ALPN, offering h2
> > > * ALPN, offering http/1.1
> > > * Cipher selection:
> > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > > * successfully set certificate verify locations:
> > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > > CApath: none
> > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > > * TLSv1.2 (IN), TLS header, Unknown (21):
> > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
> > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
> > handshake
> > > failure
> > >
> > > Thanks
> > >
> > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com>
> > wrote:
> > >
> > > > Yes it's possible. The most common issue in my experience is the
> > location
> > > > of the trust store and key store being different or absent on some
> > hosts.
> > > > You need to make sure that these locations are consistent across all
> > hosts
> > > > in your Connect cluster, or use a ConfigProvider to provide the
> > location
> > > > dynamically. Otherwise, a task will get scheduled on some host and
> > fail to
> > > > find these files.
> > > >
> > > > Ryanne
> > > >
> > > >
> > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
> > > >
> > > > > Hello. I was wondering if someone can help answer my question. I'm
> > > > trying
> > > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
> > > > distributor
> > > > > running in SSL but when I can't get the curl REST api to do so. I saw
> > > > that
> > > > > kif-208 fixed this but I can't seem to implement it.
> > > > >
> > > > > in my mm2-dist.prop file I have set:
> > > > > ////
> > > > > listeners=https://localhost:8443
> > > > > security.protocol=SSL
> > > > >
> > > > >
> > > >
> > ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > > > > ////
> > > > > my connector.json file look like this:
> > > > >
> > > > > ////
> > > > > {
> > > > > "name": "mm2-connect-cluster",
> > > > > "config":{
> > > > > "connector.class":
> > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > > > > "connector.client.config.override.policy": "All",
> > > > > "name": "mm2-connect-cluster",
> > > > > "topics": "test.*",
> > > > > "tasks.max": "1",
> > > > > "source.cluster.alias": "source",
> > > > > "target.cluster.alias": "target",
> > > > > "source.cluster.bootstrap.servers": "source:9094",
> > > > > "target.cluster.bootstrap.servers": "target:9094",
> > > > > "source->target.enabled": "true",
> > > > > "target->source.enabled": "false",
> > > > > "offset-syncs.topic.replication.factor": "4",
> > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > > > > __consumer_offsets",
> > > > > "groups.blacklist": "console-consumer-.*, connect-.*, __.*",
> > > > > "topic.creation.enabled": "true",
> > > > > "topic.creation.default.replication.factor": "4",
> > > > > "topic.creation.default.partitions": "1"
> > > > > "key.converter":
> > "org.apache.kafka.connect.json.JsonConverter",
> > > > > "value.converter":
> > "org.apache.kafka.connect.json.JsonConverter",
> > > > > "security.protocol": "SSL",
> > > > > "ssl.truststore.password":
> > > > > "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > > > > }
> > > > > }
> > > > > ////
> > > > >
> > > > > I would then start up the distributor and it launched fine. So I
> > try to
> > > > > run the CURl command
> > > > >
> > > > > ////
> > > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > > > @connector.json
> > > > > https://localhost:8443/connectors
> > > > > ////
> > > > > nada. nothing. no error. no reasons for not starting.
> > > > >
> > > > > Is it possible to run MM2 with SSL? If so, can someone point me to a
> > > > > working example?
> > > > >
> > > > > thanks.
> > > > >
> > > >
> > >
> >
>
Re: MirrorMaker 2 with SSL
Posted by Men Lim <zu...@gmail.com>.
Hi Ning,
thanks for the response. This self sign cert stays on the ec2 instance,
specifically for the curl command and I don't have to share it with the
brokers correct?
thanks,
On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ni...@gmail.com> wrote:
> Hi Men,
>
> I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
> self-signing certs and key for testing purpose:
> https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
>
> On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
> > Hi Ryanne,
> >
> > thanks for the reply. My kafka clusters are on AWS, their serverless
> > platform, MSK. I'm stuck with using the default java cacerts unless I
> use
> > their AWS PCA which is pretty pricey.
> >
> > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and got
> the
> > following verbose message:
> >
> > curl -s -X POST -H 'Content-Type: application/json' --data
> @connector.json
> > https://localhost:8443/connectors -v --tlsv1.2
> > * Trying 127.0.0.1...
> > * TCP_NODELAY set
> > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > * ALPN, offering h2
> > * ALPN, offering http/1.1
> > * Cipher selection:
> > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > * successfully set certificate verify locations:
> > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > CApath: none
> > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > * TLSv1.2 (IN), TLS header, Unknown (21):
> > * TLSv1.2 (IN), TLS alert, handshake failure (552):
> > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
> handshake
> > failure
> >
> > Thanks
> >
> > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com>
> wrote:
> >
> > > Yes it's possible. The most common issue in my experience is the
> location
> > > of the trust store and key store being different or absent on some
> hosts.
> > > You need to make sure that these locations are consistent across all
> hosts
> > > in your Connect cluster, or use a ConfigProvider to provide the
> location
> > > dynamically. Otherwise, a task will get scheduled on some host and
> fail to
> > > find these files.
> > >
> > > Ryanne
> > >
> > >
> > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
> > >
> > > > Hello. I was wondering if someone can help answer my question. I'm
> > > trying
> > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
> > > distributor
> > > > running in SSL but when I can't get the curl REST api to do so. I saw
> > > that
> > > > kif-208 fixed this but I can't seem to implement it.
> > > >
> > > > in my mm2-dist.prop file I have set:
> > > > ////
> > > > listeners=https://localhost:8443
> > > > security.protocol=SSL
> > > >
> > > >
> > >
> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > > > ////
> > > > my connector.json file look like this:
> > > >
> > > > ////
> > > > {
> > > > "name": "mm2-connect-cluster",
> > > > "config":{
> > > > "connector.class":
> > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > > > "connector.client.config.override.policy": "All",
> > > > "name": "mm2-connect-cluster",
> > > > "topics": "test.*",
> > > > "tasks.max": "1",
> > > > "source.cluster.alias": "source",
> > > > "target.cluster.alias": "target",
> > > > "source.cluster.bootstrap.servers": "source:9094",
> > > > "target.cluster.bootstrap.servers": "target:9094",
> > > > "source->target.enabled": "true",
> > > > "target->source.enabled": "false",
> > > > "offset-syncs.topic.replication.factor": "4",
> > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > > > __consumer_offsets",
> > > > "groups.blacklist": "console-consumer-.*, connect-.*, __.*",
> > > > "topic.creation.enabled": "true",
> > > > "topic.creation.default.replication.factor": "4",
> > > > "topic.creation.default.partitions": "1"
> > > > "key.converter":
> "org.apache.kafka.connect.json.JsonConverter",
> > > > "value.converter":
> "org.apache.kafka.connect.json.JsonConverter",
> > > > "security.protocol": "SSL",
> > > > "ssl.truststore.password":
> > > > "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > > > }
> > > > }
> > > > ////
> > > >
> > > > I would then start up the distributor and it launched fine. So I
> try to
> > > > run the CURl command
> > > >
> > > > ////
> > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > > @connector.json
> > > > https://localhost:8443/connectors
> > > > ////
> > > > nada. nothing. no error. no reasons for not starting.
> > > >
> > > > Is it possible to run MM2 with SSL? If so, can someone point me to a
> > > > working example?
> > > >
> > > > thanks.
> > > >
> > >
> >
>
Re: MirrorMaker 2 with SSL
Posted by Ning Zhang <ni...@gmail.com>.
Hi Men,
I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of self-signing certs and key for testing purpose: https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
On 2021/04/09 03:14:30, Men Lim <zu...@gmail.com> wrote:
> Hi Ryanne,
>
> thanks for the reply. My kafka clusters are on AWS, their serverless
> platform, MSK. I'm stuck with using the default java cacerts unless I use
> their AWS PCA which is pretty pricey.
>
> I ran the CURL command yesterday with the -v and --tlsv1.2 flag and got the
> following verbose message:
>
> curl -s -X POST -H 'Content-Type: application/json' --data @connector.json
> https://localhost:8443/connectors -v --tlsv1.2
> * Trying 127.0.0.1...
> * TCP_NODELAY set
> * Connected to localhost (127.0.0.1) port 8443 (#0)
> * ALPN, offering h2
> * ALPN, offering http/1.1
> * Cipher selection:
> ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> * successfully set certificate verify locations:
> * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> CApath: none
> * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> * TLSv1.2 (IN), TLS header, Unknown (21):
> * TLSv1.2 (IN), TLS alert, handshake failure (552):
> * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake
> failure
>
> Thanks
>
> On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com> wrote:
>
> > Yes it's possible. The most common issue in my experience is the location
> > of the trust store and key store being different or absent on some hosts.
> > You need to make sure that these locations are consistent across all hosts
> > in your Connect cluster, or use a ConfigProvider to provide the location
> > dynamically. Otherwise, a task will get scheduled on some host and fail to
> > find these files.
> >
> > Ryanne
> >
> >
> > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
> >
> > > Hello. I was wondering if someone can help answer my question. I'm
> > trying
> > > to run MirrorMaker 2 in distributed mode using SSL. I have the
> > distributor
> > > running in SSL but when I can't get the curl REST api to do so. I saw
> > that
> > > kif-208 fixed this but I can't seem to implement it.
> > >
> > > in my mm2-dist.prop file I have set:
> > > ////
> > > listeners=https://localhost:8443
> > > security.protocol=SSL
> > >
> > >
> > ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > > ////
> > > my connector.json file look like this:
> > >
> > > ////
> > > {
> > > "name": "mm2-connect-cluster",
> > > "config":{
> > > "connector.class":
> > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > > "connector.client.config.override.policy": "All",
> > > "name": "mm2-connect-cluster",
> > > "topics": "test.*",
> > > "tasks.max": "1",
> > > "source.cluster.alias": "source",
> > > "target.cluster.alias": "target",
> > > "source.cluster.bootstrap.servers": "source:9094",
> > > "target.cluster.bootstrap.servers": "target:9094",
> > > "source->target.enabled": "true",
> > > "target->source.enabled": "false",
> > > "offset-syncs.topic.replication.factor": "4",
> > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > > __consumer_offsets",
> > > "groups.blacklist": "console-consumer-.*, connect-.*, __.*",
> > > "topic.creation.enabled": "true",
> > > "topic.creation.default.replication.factor": "4",
> > > "topic.creation.default.partitions": "1"
> > > "key.converter": "org.apache.kafka.connect.json.JsonConverter",
> > > "value.converter": "org.apache.kafka.connect.json.JsonConverter",
> > > "security.protocol": "SSL",
> > > "ssl.truststore.password":
> > > "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > > }
> > > }
> > > ////
> > >
> > > I would then start up the distributor and it launched fine. So I try to
> > > run the CURl command
> > >
> > > ////
> > > curl -s -X POST -H 'Content-Type: application/json' --data
> > @connector.json
> > > https://localhost:8443/connectors
> > > ////
> > > nada. nothing. no error. no reasons for not starting.
> > >
> > > Is it possible to run MM2 with SSL? If so, can someone point me to a
> > > working example?
> > >
> > > thanks.
> > >
> >
>
Re: MirrorMaker 2 with SSL
Posted by Men Lim <zu...@gmail.com>.
Hi Ryanne,
thanks for the reply. My kafka clusters are on AWS, their serverless
platform, MSK. I'm stuck with using the default java cacerts unless I use
their AWS PCA which is pretty pricey.
I ran the CURL command yesterday with the -v and --tlsv1.2 flag and got the
following verbose message:
curl -s -X POST -H 'Content-Type: application/json' --data @connector.json
https://localhost:8443/connectors -v --tlsv1.2
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Unknown (21):
* TLSv1.2 (IN), TLS alert, handshake failure (552):
* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake
failure
Thanks
On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ry...@gmail.com> wrote:
> Yes it's possible. The most common issue in my experience is the location
> of the trust store and key store being different or absent on some hosts.
> You need to make sure that these locations are consistent across all hosts
> in your Connect cluster, or use a ConfigProvider to provide the location
> dynamically. Otherwise, a task will get scheduled on some host and fail to
> find these files.
>
> Ryanne
>
>
> On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
>
> > Hello. I was wondering if someone can help answer my question. I'm
> trying
> > to run MirrorMaker 2 in distributed mode using SSL. I have the
> distributor
> > running in SSL but when I can't get the curl REST api to do so. I saw
> that
> > kif-208 fixed this but I can't seem to implement it.
> >
> > in my mm2-dist.prop file I have set:
> > ////
> > listeners=https://localhost:8443
> > security.protocol=SSL
> >
> >
> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > ////
> > my connector.json file look like this:
> >
> > ////
> > {
> > "name": "mm2-connect-cluster",
> > "config":{
> > "connector.class":
> "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > "connector.client.config.override.policy": "All",
> > "name": "mm2-connect-cluster",
> > "topics": "test.*",
> > "tasks.max": "1",
> > "source.cluster.alias": "source",
> > "target.cluster.alias": "target",
> > "source.cluster.bootstrap.servers": "source:9094",
> > "target.cluster.bootstrap.servers": "target:9094",
> > "source->target.enabled": "true",
> > "target->source.enabled": "false",
> > "offset-syncs.topic.replication.factor": "4",
> > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > __consumer_offsets",
> > "groups.blacklist": "console-consumer-.*, connect-.*, __.*",
> > "topic.creation.enabled": "true",
> > "topic.creation.default.replication.factor": "4",
> > "topic.creation.default.partitions": "1"
> > "key.converter": "org.apache.kafka.connect.json.JsonConverter",
> > "value.converter": "org.apache.kafka.connect.json.JsonConverter",
> > "security.protocol": "SSL",
> > "ssl.truststore.password":
> > "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > }
> > }
> > ////
> >
> > I would then start up the distributor and it launched fine. So I try to
> > run the CURl command
> >
> > ////
> > curl -s -X POST -H 'Content-Type: application/json' --data
> @connector.json
> > https://localhost:8443/connectors
> > ////
> > nada. nothing. no error. no reasons for not starting.
> >
> > Is it possible to run MM2 with SSL? If so, can someone point me to a
> > working example?
> >
> > thanks.
> >
>
Re: MirrorMaker 2 with SSL
Posted by Ryanne Dolan <ry...@gmail.com>.
Yes it's possible. The most common issue in my experience is the location
of the trust store and key store being different or absent on some hosts.
You need to make sure that these locations are consistent across all hosts
in your Connect cluster, or use a ConfigProvider to provide the location
dynamically. Otherwise, a task will get scheduled on some host and fail to
find these files.
Ryanne
On Wed, Mar 31, 2021, 8:22 PM Men Lim <zu...@gmail.com> wrote:
> Hello. I was wondering if someone can help answer my question. I'm trying
> to run MirrorMaker 2 in distributed mode using SSL. I have the distributor
> running in SSL but when I can't get the curl REST api to do so. I saw that
> kif-208 fixed this but I can't seem to implement it.
>
> in my mm2-dist.prop file I have set:
> ////
> listeners=https://localhost:8443
> security.protocol=SSL
>
> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> ////
> my connector.json file look like this:
>
> ////
> {
> "name": "mm2-connect-cluster",
> "config":{
> "connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> "connector.client.config.override.policy": "All",
> "name": "mm2-connect-cluster",
> "topics": "test.*",
> "tasks.max": "1",
> "source.cluster.alias": "source",
> "target.cluster.alias": "target",
> "source.cluster.bootstrap.servers": "source:9094",
> "target.cluster.bootstrap.servers": "target:9094",
> "source->target.enabled": "true",
> "target->source.enabled": "false",
> "offset-syncs.topic.replication.factor": "4",
> "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> __consumer_offsets",
> "groups.blacklist": "console-consumer-.*, connect-.*, __.*",
> "topic.creation.enabled": "true",
> "topic.creation.default.replication.factor": "4",
> "topic.creation.default.partitions": "1"
> "key.converter": "org.apache.kafka.connect.json.JsonConverter",
> "value.converter": "org.apache.kafka.connect.json.JsonConverter",
> "security.protocol": "SSL",
> "ssl.truststore.password":
> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> }
> }
> ////
>
> I would then start up the distributor and it launched fine. So I try to
> run the CURl command
>
> ////
> curl -s -X POST -H 'Content-Type: application/json' --data @connector.json
> https://localhost:8443/connectors
> ////
> nada. nothing. no error. no reasons for not starting.
>
> Is it possible to run MM2 with SSL? If so, can someone point me to a
> working example?
>
> thanks.
>