You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Ivan Kelly <iv...@apache.org> on 2018/08/07 08:12:44 UTC
PIP 20: Mechanism to revoke TLS authentication
Hi folks,
This is a PIP to add a mechanism to block TLS client certs from
accessing Pulsar if they have been compromised.
This is a relatively small change, but I thought it best to put it to
the community before moving ahead with it, as people may have opinions
on the approach.
The PIP is here:
https://github.com/apache/incubator-pulsar/wiki/PIP-20%3A-Mechanism-to-revoke-TLS-authentication
Cheers,
Ivan
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Sijie Guo <gu...@gmail.com>.
On Mon, Aug 20, 2018 at 5:59 AM Ivan Kelly <iv...@apache.org> wrote:
> > Instead of using zookeeper, can we consider using a managed ledger or a
> system topic for keeping all these revoked keys?
>
> Do we already have an example of a system topic in use?
Functions is using system topics. And all its topics are under `functions`
namespace.
> Is there a
> custom namespace that's restricted to the admin role?
>
Currently I don't think we restrict the namespace to admin role. But we
should probably do so.
- Sijie
>
> -Ivan
>
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Ivan Kelly <iv...@apache.org>.
> Instead of using zookeeper, can we consider using a managed ledger or a
system topic for keeping all these revoked keys?
Do we already have an example of a system topic in use? Is there a
custom namespace that's restricted to the admin role?
-Ivan
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Ivan Kelly <iv...@apache.org>.
That said, it is stored in Json so it wouldn't be much work to sort
one of the values as a list of strings.
-Ivan
On Thu, Sep 6, 2018 at 10:55 AM, Ivan Kelly <iv...@apache.org> wrote:
> PIP updated. The one thing that isn't 100% clean with the dynamic
> configuration keys is that they only support string as value, so the
> list of revoked keys needs some sort of serialization.
>
> -Ivan
>
> On Thu, Sep 6, 2018 at 10:41 AM, Ivan Kelly <iv...@apache.org> wrote:
>> On Tue, Aug 21, 2018 at 3:57 AM, Rajan Dhabalia <rd...@apache.org> wrote:
>>>> I will consider both this and system topics and update the PIP.
>>>
>>> I think we should avoid more complexity by introducing system topic to
>>> store "subject key" and then process it. Instead we can store it to
>>> global/config zk as it requires to store keys with few bytes. I think
>>> dynamic-configuration might be enough for this configuration.
>>
>>
>> I think so too. Given that system topics are uses in the broker now,
>> the amount of plumbing would likely be quite high, so it's better to
>> go with a dynamic-configuration for the subject key
>>
>> -Ivan
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Ivan Kelly <iv...@apache.org>.
PIP updated. The one thing that isn't 100% clean with the dynamic
configuration keys is that they only support string as value, so the
list of revoked keys needs some sort of serialization.
-Ivan
On Thu, Sep 6, 2018 at 10:41 AM, Ivan Kelly <iv...@apache.org> wrote:
> On Tue, Aug 21, 2018 at 3:57 AM, Rajan Dhabalia <rd...@apache.org> wrote:
>>> I will consider both this and system topics and update the PIP.
>>
>> I think we should avoid more complexity by introducing system topic to
>> store "subject key" and then process it. Instead we can store it to
>> global/config zk as it requires to store keys with few bytes. I think
>> dynamic-configuration might be enough for this configuration.
>
>
> I think so too. Given that system topics are uses in the broker now,
> the amount of plumbing would likely be quite high, so it's better to
> go with a dynamic-configuration for the subject key
>
> -Ivan
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Ivan Kelly <iv...@apache.org>.
On Tue, Aug 21, 2018 at 3:57 AM, Rajan Dhabalia <rd...@apache.org> wrote:
>> I will consider both this and system topics and update the PIP.
>
> I think we should avoid more complexity by introducing system topic to
> store "subject key" and then process it. Instead we can store it to
> global/config zk as it requires to store keys with few bytes. I think
> dynamic-configuration might be enough for this configuration.
I think so too. Given that system topics are uses in the broker now,
the amount of plumbing would likely be quite high, so it's better to
go with a dynamic-configuration for the subject key
-Ivan
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Rajan Dhabalia <rd...@apache.org>.
> I will consider both this and system topics and update the PIP.
I think we should avoid more complexity by introducing system topic to
store "subject key" and then process it. Instead we can store it to
global/config zk as it requires to store keys with few bytes. I think
dynamic-configuration might be enough for this configuration.
Thanks,
Rajan
On Mon, Aug 20, 2018 at 6:01 AM, Ivan Kelly <iv...@apache.org> wrote:
> > Here, we would like to configure "subject-key-identifier" at every broker
> in the cluster dynamically. We also want to perform certain actions once
> this configuration-value has been changed.
>
> Ah, i didn't know this existed. Very useful. I will consider both this
> and system topics and update the PIP.
>
>
> -Ivan
>
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Ivan Kelly <iv...@apache.org>.
> Here, we would like to configure "subject-key-identifier" at every broker
in the cluster dynamically. We also want to perform certain actions once
this configuration-value has been changed.
Ah, i didn't know this existed. Very useful. I will consider both this
and system topics and update the PIP.
-Ivan
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Rajan Dhabalia <rd...@apache.org>.
Hi,
Here, we would like to configure "subject-key-identifier" at every broker
in the cluster dynamically. We also want to perform certain actions once
this configuration-value has been changed.
Broker's dynamic configuration is introduced for such usecases where
dynamic-config value gets stored into zk and change will be listened by all
brokers so, it can be applied to entire cluster immediately.
So, I think we should add dynamic configuration
<https://github.com/apache/incubator-pulsar/blob/master/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java#L369>
"*tlsSubjectKeyIdentifier*" at "ServiceConfiguration" and we can also register
appropriate listener
<https://github.com/apache/incubator-pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L1184>to
perform subsequent action with newly changed value of the configuration.
So, I would not add one more REST end-point and it should be part of
dynamic configuration.
You can see "dispatchThrottlingRatePerSubscriptionInMsg
<https://github.com/apache/incubator-pulsar/blob/master/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L1177>"
example for dynamic-configuration.
Thanks,
Rajan
On Tue, Aug 7, 2018 at 1:12 AM, Ivan Kelly <iv...@apache.org> wrote:
> Hi folks,
>
> This is a PIP to add a mechanism to block TLS client certs from
> accessing Pulsar if they have been compromised.
>
> This is a relatively small change, but I thought it best to put it to
> the community before moving ahead with it, as people may have opinions
> on the approach.
>
> The PIP is here:
> https://github.com/apache/incubator-pulsar/wiki/PIP-20%
> 3A-Mechanism-to-revoke-TLS-authentication
>
> Cheers,
> Ivan
>
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Sanjeev Kulkarni <sa...@gmail.com>.
+1 to Sijie's suggestion to keep this inside a managed ledger.
On Mon, Aug 13, 2018 at 3:34 PM Sijie Guo <gu...@gmail.com> wrote:
> Ivan,
>
> Thank you for writing this up. This PIP looks great to me! +1
>
> just one question:
>
> > This will add the subject key identifier to zookeeper under
> /tls/revoked/<subject-key-id>. All brokers and proxies cache the children
> /tls/revoked.
>
> Instead of using zookeeper, can we consider using a managed ledger or a
> system topic for keeping all these revoked keys?
>
> - Sijie
>
> On Tue, Aug 7, 2018 at 1:12 AM Ivan Kelly <iv...@apache.org> wrote:
>
> > Hi folks,
> >
> > This is a PIP to add a mechanism to block TLS client certs from
> > accessing Pulsar if they have been compromised.
> >
> > This is a relatively small change, but I thought it best to put it to
> > the community before moving ahead with it, as people may have opinions
> > on the approach.
> >
> > The PIP is here:
> >
> >
> https://github.com/apache/incubator-pulsar/wiki/PIP-20%3A-Mechanism-to-revoke-TLS-authentication
> >
> > Cheers,
> > Ivan
> >
>
Re: PIP 20: Mechanism to revoke TLS authentication
Posted by Sijie Guo <gu...@gmail.com>.
Ivan,
Thank you for writing this up. This PIP looks great to me! +1
just one question:
> This will add the subject key identifier to zookeeper under
/tls/revoked/<subject-key-id>. All brokers and proxies cache the children
/tls/revoked.
Instead of using zookeeper, can we consider using a managed ledger or a
system topic for keeping all these revoked keys?
- Sijie
On Tue, Aug 7, 2018 at 1:12 AM Ivan Kelly <iv...@apache.org> wrote:
> Hi folks,
>
> This is a PIP to add a mechanism to block TLS client certs from
> accessing Pulsar if they have been compromised.
>
> This is a relatively small change, but I thought it best to put it to
> the community before moving ahead with it, as people may have opinions
> on the approach.
>
> The PIP is here:
>
> https://github.com/apache/incubator-pulsar/wiki/PIP-20%3A-Mechanism-to-revoke-TLS-authentication
>
> Cheers,
> Ivan
>