You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Zixuan Liu <no...@gmail.com> on 2023/02/09 04:12:14 UTC
[DISCUSS] Base64-encoded key/cert in the ClusterData
Hi all,
In the ClusterData, we have two types of the key/certificate, one is PEM,
and one is JKS.
I would like to discuss the bae64-encoded key/certificate in PEM format.
The Pulsar can only load the key/certificate by the file path. When
configuring the key/certificate, we must copy the key/certificate files to
each cluster. Sometimes duplicating certificates is a chore, so I want to
add base64-encoded format support.
Based-64 encoded:
```
brokerClientKeyFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-KEY-FILE-BODY"
brokerClientCertificateFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-CERTIFICATE-FILE-BODY"
brokerClientTrustCertsFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-TRUST-CERTS-FILE-BODY"
```
File path:
```
brokerClientKeyFilePath="/path/to/KEY-FILE.pem"
brokerClientCertificateFilePath="/path/to/CERTIFICATE-FILE.pem"
brokerClientTrustCertsFilePath="/path/to/TRUST-CERTS-FILE.pem"
```
Please let me know what you think.
Thanks,
Zixuan
Re: [DISCUSS] Base64-encoded key/cert in the ClusterData
Posted by Zixuan Liu <no...@gmail.com>.
Adding a new configuration looks good.
Thanks,
Zixuan
Zike Yang <zi...@apache.org> 于2023年2月10日周五 15:21写道：
> It looks strange to fill a configuration item named `XXXFilePath` with
> the contents of the key/certs instead of the file path. Maybe adding
> new configs like `brokerClientKey`, `brokerClientCertificate`... would
> be more appropriate.
>
> Thanks,
> Zike Yang
>
>
>
>
> Zike Yang
>
>
> On Fri, Feb 10, 2023 at 2:54 PM Michael Marshall <mm...@apache.org>
> wrote:
> >
> > Is this the same ClusterData object stored in zookeeper? If so, it
> > seems risky to store these certs there because many Pulsar components
> > access ZK.
> >
> > I started work to support retrieving in-memory TLS certificates to the
> > Java Client's ClientConfiguration object [0] but my priorities
> > changed, and I wasn't able to get the PR in a good enough place to
> > complete the feature. It's not the same, but it follows a similar
> > pattern of having a different way to configure TLS certs.
> >
> > Would it make sense to add a pluggable supplier that retrieves and
> > decodes certs? Then, it wouldn't require pulsar code changes for minor
> > nuances in implementation.
> >
> > Thanks,
> > Michael
> >
> > [0] https://github.com/apache/pulsar/pull/18121
> >
> > On Wed, Feb 8, 2023 at 10:12 PM Zixuan Liu <no...@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > In the ClusterData, we have two types of the key/certificate, one is
> PEM,
> > > and one is JKS.
> > >
> > > I would like to discuss the bae64-encoded key/certificate in PEM
> format.
> > >
> > > The Pulsar can only load the key/certificate by the file path. When
> > > configuring the key/certificate, we must copy the key/certificate
> files to
> > > each cluster. Sometimes duplicating certificates is a chore, so I want
> to
> > > add base64-encoded format support.
> > >
> > > Based-64 encoded:
> > > ```
> > >
> brokerClientKeyFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-KEY-FILE-BODY"
> > >
> brokerClientCertificateFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-CERTIFICATE-FILE-BODY"
> > >
> brokerClientTrustCertsFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-TRUST-CERTS-FILE-BODY"
> > > ```
> > >
> > > File path:
> > > ```
> > > brokerClientKeyFilePath="/path/to/KEY-FILE.pem"
> > > brokerClientCertificateFilePath="/path/to/CERTIFICATE-FILE.pem"
> > > brokerClientTrustCertsFilePath="/path/to/TRUST-CERTS-FILE.pem"
> > > ```
> > >
> > > Please let me know what you think.
> > >
> > > Thanks,
> > > Zixuan
>
Re: [DISCUSS] Base64-encoded key/cert in the ClusterData
Posted by Zike Yang <zi...@apache.org>.
It looks strange to fill a configuration item named `XXXFilePath` with
the contents of the key/certs instead of the file path. Maybe adding
new configs like `brokerClientKey`, `brokerClientCertificate`... would
be more appropriate.
Thanks,
Zike Yang
Zike Yang
On Fri, Feb 10, 2023 at 2:54 PM Michael Marshall <mm...@apache.org> wrote:
>
> Is this the same ClusterData object stored in zookeeper? If so, it
> seems risky to store these certs there because many Pulsar components
> access ZK.
>
> I started work to support retrieving in-memory TLS certificates to the
> Java Client's ClientConfiguration object [0] but my priorities
> changed, and I wasn't able to get the PR in a good enough place to
> complete the feature. It's not the same, but it follows a similar
> pattern of having a different way to configure TLS certs.
>
> Would it make sense to add a pluggable supplier that retrieves and
> decodes certs? Then, it wouldn't require pulsar code changes for minor
> nuances in implementation.
>
> Thanks,
> Michael
>
> [0] https://github.com/apache/pulsar/pull/18121
>
> On Wed, Feb 8, 2023 at 10:12 PM Zixuan Liu <no...@gmail.com> wrote:
> >
> > Hi all,
> >
> > In the ClusterData, we have two types of the key/certificate, one is PEM,
> > and one is JKS.
> >
> > I would like to discuss the bae64-encoded key/certificate in PEM format.
> >
> > The Pulsar can only load the key/certificate by the file path. When
> > configuring the key/certificate, we must copy the key/certificate files to
> > each cluster. Sometimes duplicating certificates is a chore, so I want to
> > add base64-encoded format support.
> >
> > Based-64 encoded:
> > ```
> > brokerClientKeyFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-KEY-FILE-BODY"
> > brokerClientCertificateFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-CERTIFICATE-FILE-BODY"
> > brokerClientTrustCertsFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-TRUST-CERTS-FILE-BODY"
> > ```
> >
> > File path:
> > ```
> > brokerClientKeyFilePath="/path/to/KEY-FILE.pem"
> > brokerClientCertificateFilePath="/path/to/CERTIFICATE-FILE.pem"
> > brokerClientTrustCertsFilePath="/path/to/TRUST-CERTS-FILE.pem"
> > ```
> >
> > Please let me know what you think.
> >
> > Thanks,
> > Zixuan
Re: [DISCUSS] Base64-encoded key/cert in the ClusterData
Posted by Zixuan Liu <no...@gmail.com>.
> Is this the same ClusterData object stored in zookeeper?
Sure, it is unsafe, I have the same concern, so I made this issue.
> Would it make sense to add a pluggable supplier that retrieves and
decodes certs? Then, it wouldn't require pulsar code changes for minor
nuances in implementation.
This is a good idea, but there's a little trouble for users, who need to
implement their provider.
Thank you for your idea, I will refer to that.
Thanks,
Zixuan
Michael Marshall <mm...@apache.org> 于2023年2月10日周五 14:54写道：
> Is this the same ClusterData object stored in zookeeper? If so, it
> seems risky to store these certs there because many Pulsar components
> access ZK.
>
> I started work to support retrieving in-memory TLS certificates to the
> Java Client's ClientConfiguration object [0] but my priorities
> changed, and I wasn't able to get the PR in a good enough place to
> complete the feature. It's not the same, but it follows a similar
> pattern of having a different way to configure TLS certs.
>
> Would it make sense to add a pluggable supplier that retrieves and
> decodes certs? Then, it wouldn't require pulsar code changes for minor
> nuances in implementation.
>
> Thanks,
> Michael
>
> [0] https://github.com/apache/pulsar/pull/18121
>
> On Wed, Feb 8, 2023 at 10:12 PM Zixuan Liu <no...@gmail.com> wrote:
> >
> > Hi all,
> >
> > In the ClusterData, we have two types of the key/certificate, one is PEM,
> > and one is JKS.
> >
> > I would like to discuss the bae64-encoded key/certificate in PEM format.
> >
> > The Pulsar can only load the key/certificate by the file path. When
> > configuring the key/certificate, we must copy the key/certificate files
> to
> > each cluster. Sometimes duplicating certificates is a chore, so I want to
> > add base64-encoded format support.
> >
> > Based-64 encoded:
> > ```
> >
> brokerClientKeyFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-KEY-FILE-BODY"
> >
> brokerClientCertificateFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-CERTIFICATE-FILE-BODY"
> >
> brokerClientTrustCertsFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-TRUST-CERTS-FILE-BODY"
> > ```
> >
> > File path:
> > ```
> > brokerClientKeyFilePath="/path/to/KEY-FILE.pem"
> > brokerClientCertificateFilePath="/path/to/CERTIFICATE-FILE.pem"
> > brokerClientTrustCertsFilePath="/path/to/TRUST-CERTS-FILE.pem"
> > ```
> >
> > Please let me know what you think.
> >
> > Thanks,
> > Zixuan
>
Re: [DISCUSS] Base64-encoded key/cert in the ClusterData
Posted by Michael Marshall <mm...@apache.org>.
Is this the same ClusterData object stored in zookeeper? If so, it
seems risky to store these certs there because many Pulsar components
access ZK.
I started work to support retrieving in-memory TLS certificates to the
Java Client's ClientConfiguration object [0] but my priorities
changed, and I wasn't able to get the PR in a good enough place to
complete the feature. It's not the same, but it follows a similar
pattern of having a different way to configure TLS certs.
Would it make sense to add a pluggable supplier that retrieves and
decodes certs? Then, it wouldn't require pulsar code changes for minor
nuances in implementation.
Thanks,
Michael
[0] https://github.com/apache/pulsar/pull/18121
On Wed, Feb 8, 2023 at 10:12 PM Zixuan Liu <no...@gmail.com> wrote:
>
> Hi all,
>
> In the ClusterData, we have two types of the key/certificate, one is PEM,
> and one is JKS.
>
> I would like to discuss the bae64-encoded key/certificate in PEM format.
>
> The Pulsar can only load the key/certificate by the file path. When
> configuring the key/certificate, we must copy the key/certificate files to
> each cluster. Sometimes duplicating certificates is a chore, so I want to
> add base64-encoded format support.
>
> Based-64 encoded:
> ```
> brokerClientKeyFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-KEY-FILE-BODY"
> brokerClientCertificateFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-CERTIFICATE-FILE-BODY"
> brokerClientTrustCertsFilePath="data:application/x-pem-file;base64,BASE64-ENCODED-TRUST-CERTS-FILE-BODY"
> ```
>
> File path:
> ```
> brokerClientKeyFilePath="/path/to/KEY-FILE.pem"
> brokerClientCertificateFilePath="/path/to/CERTIFICATE-FILE.pem"
> brokerClientTrustCertsFilePath="/path/to/TRUST-CERTS-FILE.pem"
> ```
>
> Please let me know what you think.
>
> Thanks,
> Zixuan