You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Mickael Maison <mi...@gmail.com> on 2025/03/12 16:05:29 UTC
Re: [DISCUSS] KIP-1119: Add support for SSL hot reload
Hi,
Thanks for the KIP, it seems a useful improvement. I guess this would
supersede KIP-687.
The KIP is a bit light on details.
- Can you provide the description and default value for the new configuration?
- The KIP explicitly mentions brokers, consumer and producers, I
assume it also covers admin clients (and controllers)?
- The KIP states "To prevent redundant reconfigurations, a quiet
period is enforced across watched files". How long is this period? Is
it configurable?
- I'm unsure about the configuration name. Have you considered
alternative like "ssl.auto.reload" or "ssl.stores.auto.reload"?
Thanks,
Mickael
On Thu, Dec 5, 2024 at 6:28 PM Moncef Abboud <mo...@gmail.com> wrote:
>
> Hi Gaurav,
>
> Thank you for your reply.
>
> There is some overlap between the two. However, while KIP-687 focuses
> primarily on brokers, this KIP targets consumers and producers, with
> additional benefits for brokers.
>
> Best,
> Moncef
>
> On Thu, Dec 5, 2024, 10:38 AM Gaurav Narula <ka...@gnarula.com> wrote:
>
> > Hi Moncef,
> >
> > Thank you for the KIP. It seems very similar in spirit to KIP-687 (
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-687%3A+Automatic+Reloading+of+Security+Store)
> > which seems like it was approved but never fully implemented. Can you
> > please confirm if it is the case indeed?
> >
> > Regards,
> > Gaurav
> >
> > > On 2 Dec 2024, at 23:12, Moncef Abboud <mo...@gmail.com>
> > wrote:
> > >
> > > Hi all,
> > >
> > > I hope your week is off to a great start.
> > >
> > > I created a KIP to add support for SSL hot reloading.
> > > https://cwiki.apache.org/confluence/x/eIrREw
> > >
> > > Thank you for your feedback!
> > >
> > >
> > > Moncef
> >
> >
Re: [DISCUSS] KIP-1119: Add support for SSL hot reload
Posted by Moncef Abboud <mo...@gmail.com>.
Gaurav, Colin,
Thank you for your replies.
> It appears that inotify on Linux doesn't emit events for bind mounts.
This is indeed a valid concern and it seems that this a well-known issue..
As Colin suggested, we can work around this by manually polling or by
leveraging a library that uses polling behind the scenes, such as Apache
Commons IO Monitor [0].
Regarding the symlinks, we can add a check to resolve symlinks to their
actual paths (java.nio.file.Files.readSymbolicLink).
What do you think?
Thanks,
Moncef
[0]
https://github.com/apache/commons-io/blob/6ddc0834e63d0924ae1bca939058ac749214f2fb/src/main/java/org/apache/commons/io/monitor/FileAlterationObserver.java#L531
On Tue, Apr 29, 2025 at 3:09 PM Colin McCabe <cm...@apache.org> wrote:
> Gaurav brings up a good point. However, we can work around that bug by
> doing polling every 5 or 10 minutes in addition to using WatchService.
>
> best,
> Colin
>
> On Wed, Apr 23, 2025, at 00:39, Gaurav Narula wrote:
> > Hi Moncef,
> >
> > Have you tried experimenting the behaviour of JDK's WatchService with
> > bind mounts?
> >
> > Reading [0], it appears that inotify on linux doesn't emit events for
> > bind mounts. This may pose a limitation in Kubernetes as that's how
> > config maps are mounted.
> >
> > Another scenario to consider would be symlinks and how WatchService
> > handles events on them.
> >
> > Regards,
> > Gaurav
> >
> > [0]: https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/
> >
> >
> >> On 21 Mar 2025, at 04:40, Moncef Abboud <mo...@gmail.com>
> wrote:
> >>
> >> Hi Mickael,
> >>
> >> Thank you for your feedback.
> >>
> >>> - Can you provide the description and default value for the new
> >> configuration?
> >>
> >> Done. I amended the KIP.
> >>
> >>> The KIP explicitly mentions brokers, consumer and producers, I assume
> it
> >> also covers admin clients (and controllers)?
> >>
> >> Correct. I believe both the controllers and admin clients would benefit
> >> from the change since they use the SslFactory.
> >>
> >>> The KIP states "To prevent redundant reconfigurations, a quiet period
> is
> >> enforced across watched files". How long is this period? Is it
> configurable?
> >>
> >> The period is 30 seconds. My reasoning is that this should be long
> enough
> >> to allow an agent (such as a k8s Vault sidecar) to complete a few round
> >> trips with a remote server, but not too long. Given that each SslFactory
> >> (broker listener, client) has its own SSL properties, I thought it best
> to
> >> have this quiet period be consistent across all factories since there
> will
> >> only be one watcher thread. I chose not to make it configurable (one
> less
> >> thing to worry about), but I’m happy to change that.
> >>
> >>> I'm unsure about the configuration name. Have you considered
> alternative
> >> like "ssl.auto.reload" or "ssl.stores.auto.reload"?
> >> Agreed. I changed it to "ssl.auto.reload".
> >>
> >> Let me know what you think.
> >>
> >> Thanks again,
> >> Moncef
> >>
> >> On Wed, Mar 12, 2025 at 11:08 AM Mickael Maison <
> mickael.maison@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Thanks for the KIP, it seems a useful improvement. I guess this would
> >>> supersede KIP-687.
> >>>
> >>> The KIP is a bit light on details.
> >>> - Can you provide the description and default value for the new
> >>> configuration?
> >>> - The KIP explicitly mentions brokers, consumer and producers, I
> >>> assume it also covers admin clients (and controllers)?
> >>> - The KIP states "To prevent redundant reconfigurations, a quiet
> >>> period is enforced across watched files". How long is this period? Is
> >>> it configurable?
> >>> - I'm unsure about the configuration name. Have you considered
> >>> alternative like "ssl.auto.reload" or "ssl.stores.auto.reload"?
> >>>
> >>> Thanks,
> >>> Mickael
> >>>
> >>> On Thu, Dec 5, 2024 at 6:28 PM Moncef Abboud <
> moncef.abboud95@gmail.com>
> >>> wrote:
> >>>>
> >>>> Hi Gaurav,
> >>>>
> >>>> Thank you for your reply.
> >>>>
> >>>> There is some overlap between the two. However, while KIP-687 focuses
> >>>> primarily on brokers, this KIP targets consumers and producers, with
> >>>> additional benefits for brokers.
> >>>>
> >>>> Best,
> >>>> Moncef
> >>>>
> >>>> On Thu, Dec 5, 2024, 10:38 AM Gaurav Narula <ka...@gnarula.com>
> wrote:
> >>>>
> >>>>> Hi Moncef,
> >>>>>
> >>>>> Thank you for the KIP. It seems very similar in spirit to KIP-687 (
> >>>>>
> >>>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-687%3A+Automatic+Reloading+of+Security+Store
> >>> )
> >>>>> which seems like it was approved but never fully implemented. Can you
> >>>>> please confirm if it is the case indeed?
> >>>>>
> >>>>> Regards,
> >>>>> Gaurav
> >>>>>
> >>>>>> On 2 Dec 2024, at 23:12, Moncef Abboud <mo...@gmail.com>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I hope your week is off to a great start.
> >>>>>>
> >>>>>> I created a KIP to add support for SSL hot reloading.
> >>>>>> https://cwiki.apache.org/confluence/x/eIrREw
> >>>>>>
> >>>>>> Thank you for your feedback!
> >>>>>>
> >>>>>>
> >>>>>> Moncef
> >>>>>
> >>>>>
> >>>
> >>
> >>
> >> --
> >> Moncef ABBOUD
>
--
Moncef ABBOUD
Re: [DISCUSS] KIP-1119: Add support for SSL hot reload
Posted by Colin McCabe <cm...@apache.org>.
Gaurav brings up a good point. However, we can work around that bug by doing polling every 5 or 10 minutes in addition to using WatchService.
best,
Colin
On Wed, Apr 23, 2025, at 00:39, Gaurav Narula wrote:
> Hi Moncef,
>
> Have you tried experimenting the behaviour of JDK's WatchService with
> bind mounts?
>
> Reading [0], it appears that inotify on linux doesn't emit events for
> bind mounts. This may pose a limitation in Kubernetes as that's how
> config maps are mounted.
>
> Another scenario to consider would be symlinks and how WatchService
> handles events on them.
>
> Regards,
> Gaurav
>
> [0]: https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/
>
>
>> On 21 Mar 2025, at 04:40, Moncef Abboud <mo...@gmail.com> wrote:
>>
>> Hi Mickael,
>>
>> Thank you for your feedback.
>>
>>> - Can you provide the description and default value for the new
>> configuration?
>>
>> Done. I amended the KIP.
>>
>>> The KIP explicitly mentions brokers, consumer and producers, I assume it
>> also covers admin clients (and controllers)?
>>
>> Correct. I believe both the controllers and admin clients would benefit
>> from the change since they use the SslFactory.
>>
>>> The KIP states "To prevent redundant reconfigurations, a quiet period is
>> enforced across watched files". How long is this period? Is it configurable?
>>
>> The period is 30 seconds. My reasoning is that this should be long enough
>> to allow an agent (such as a k8s Vault sidecar) to complete a few round
>> trips with a remote server, but not too long. Given that each SslFactory
>> (broker listener, client) has its own SSL properties, I thought it best to
>> have this quiet period be consistent across all factories since there will
>> only be one watcher thread. I chose not to make it configurable (one less
>> thing to worry about), but I’m happy to change that.
>>
>>> I'm unsure about the configuration name. Have you considered alternative
>> like "ssl.auto.reload" or "ssl.stores.auto.reload"?
>> Agreed. I changed it to "ssl.auto.reload".
>>
>> Let me know what you think.
>>
>> Thanks again,
>> Moncef
>>
>> On Wed, Mar 12, 2025 at 11:08 AM Mickael Maison <mi...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks for the KIP, it seems a useful improvement. I guess this would
>>> supersede KIP-687.
>>>
>>> The KIP is a bit light on details.
>>> - Can you provide the description and default value for the new
>>> configuration?
>>> - The KIP explicitly mentions brokers, consumer and producers, I
>>> assume it also covers admin clients (and controllers)?
>>> - The KIP states "To prevent redundant reconfigurations, a quiet
>>> period is enforced across watched files". How long is this period? Is
>>> it configurable?
>>> - I'm unsure about the configuration name. Have you considered
>>> alternative like "ssl.auto.reload" or "ssl.stores.auto.reload"?
>>>
>>> Thanks,
>>> Mickael
>>>
>>> On Thu, Dec 5, 2024 at 6:28 PM Moncef Abboud <mo...@gmail.com>
>>> wrote:
>>>>
>>>> Hi Gaurav,
>>>>
>>>> Thank you for your reply.
>>>>
>>>> There is some overlap between the two. However, while KIP-687 focuses
>>>> primarily on brokers, this KIP targets consumers and producers, with
>>>> additional benefits for brokers.
>>>>
>>>> Best,
>>>> Moncef
>>>>
>>>> On Thu, Dec 5, 2024, 10:38 AM Gaurav Narula <ka...@gnarula.com> wrote:
>>>>
>>>>> Hi Moncef,
>>>>>
>>>>> Thank you for the KIP. It seems very similar in spirit to KIP-687 (
>>>>>
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-687%3A+Automatic+Reloading+of+Security+Store
>>> )
>>>>> which seems like it was approved but never fully implemented. Can you
>>>>> please confirm if it is the case indeed?
>>>>>
>>>>> Regards,
>>>>> Gaurav
>>>>>
>>>>>> On 2 Dec 2024, at 23:12, Moncef Abboud <mo...@gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I hope your week is off to a great start.
>>>>>>
>>>>>> I created a KIP to add support for SSL hot reloading.
>>>>>> https://cwiki.apache.org/confluence/x/eIrREw
>>>>>>
>>>>>> Thank you for your feedback!
>>>>>>
>>>>>>
>>>>>> Moncef
>>>>>
>>>>>
>>>
>>
>>
>> --
>> Moncef ABBOUD
Re: [DISCUSS] KIP-1119: Add support for SSL hot reload
Posted by Gaurav Narula <ka...@gnarula.com>.
Hi Moncef,
Have you tried experimenting the behaviour of JDK's WatchService with bind mounts?
Reading [0], it appears that inotify on linux doesn't emit events for bind mounts. This may pose a limitation in Kubernetes as that's how config maps are mounted.
Another scenario to consider would be symlinks and how WatchService handles events on them.
Regards,
Gaurav
[0]: https://blog.arkey.fr/2019/09/13/watchservice-and-bind-mount/
> On 21 Mar 2025, at 04:40, Moncef Abboud <mo...@gmail.com> wrote:
>
> Hi Mickael,
>
> Thank you for your feedback.
>
>> - Can you provide the description and default value for the new
> configuration?
>
> Done. I amended the KIP.
>
>> The KIP explicitly mentions brokers, consumer and producers, I assume it
> also covers admin clients (and controllers)?
>
> Correct. I believe both the controllers and admin clients would benefit
> from the change since they use the SslFactory.
>
>> The KIP states "To prevent redundant reconfigurations, a quiet period is
> enforced across watched files". How long is this period? Is it configurable?
>
> The period is 30 seconds. My reasoning is that this should be long enough
> to allow an agent (such as a k8s Vault sidecar) to complete a few round
> trips with a remote server, but not too long. Given that each SslFactory
> (broker listener, client) has its own SSL properties, I thought it best to
> have this quiet period be consistent across all factories since there will
> only be one watcher thread. I chose not to make it configurable (one less
> thing to worry about), but I’m happy to change that.
>
>> I'm unsure about the configuration name. Have you considered alternative
> like "ssl.auto.reload" or "ssl.stores.auto.reload"?
> Agreed. I changed it to "ssl.auto.reload".
>
> Let me know what you think.
>
> Thanks again,
> Moncef
>
> On Wed, Mar 12, 2025 at 11:08 AM Mickael Maison <mi...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Thanks for the KIP, it seems a useful improvement. I guess this would
>> supersede KIP-687.
>>
>> The KIP is a bit light on details.
>> - Can you provide the description and default value for the new
>> configuration?
>> - The KIP explicitly mentions brokers, consumer and producers, I
>> assume it also covers admin clients (and controllers)?
>> - The KIP states "To prevent redundant reconfigurations, a quiet
>> period is enforced across watched files". How long is this period? Is
>> it configurable?
>> - I'm unsure about the configuration name. Have you considered
>> alternative like "ssl.auto.reload" or "ssl.stores.auto.reload"?
>>
>> Thanks,
>> Mickael
>>
>> On Thu, Dec 5, 2024 at 6:28 PM Moncef Abboud <mo...@gmail.com>
>> wrote:
>>>
>>> Hi Gaurav,
>>>
>>> Thank you for your reply.
>>>
>>> There is some overlap between the two. However, while KIP-687 focuses
>>> primarily on brokers, this KIP targets consumers and producers, with
>>> additional benefits for brokers.
>>>
>>> Best,
>>> Moncef
>>>
>>> On Thu, Dec 5, 2024, 10:38 AM Gaurav Narula <ka...@gnarula.com> wrote:
>>>
>>>> Hi Moncef,
>>>>
>>>> Thank you for the KIP. It seems very similar in spirit to KIP-687 (
>>>>
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-687%3A+Automatic+Reloading+of+Security+Store
>> )
>>>> which seems like it was approved but never fully implemented. Can you
>>>> please confirm if it is the case indeed?
>>>>
>>>> Regards,
>>>> Gaurav
>>>>
>>>>> On 2 Dec 2024, at 23:12, Moncef Abboud <mo...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> I hope your week is off to a great start.
>>>>>
>>>>> I created a KIP to add support for SSL hot reloading.
>>>>> https://cwiki.apache.org/confluence/x/eIrREw
>>>>>
>>>>> Thank you for your feedback!
>>>>>
>>>>>
>>>>> Moncef
>>>>
>>>>
>>
>
>
> --
> Moncef ABBOUD
Re: [DISCUSS] KIP-1119: Add support for SSL hot reload
Posted by Moncef Abboud <mo...@gmail.com>.
Hi Mickael,
Thank you for your feedback.
>- Can you provide the description and default value for the new
configuration?
Done. I amended the KIP.
>The KIP explicitly mentions brokers, consumer and producers, I assume it
also covers admin clients (and controllers)?
Correct. I believe both the controllers and admin clients would benefit
from the change since they use the SslFactory.
> The KIP states "To prevent redundant reconfigurations, a quiet period is
enforced across watched files". How long is this period? Is it configurable?
The period is 30 seconds. My reasoning is that this should be long enough
to allow an agent (such as a k8s Vault sidecar) to complete a few round
trips with a remote server, but not too long. Given that each SslFactory
(broker listener, client) has its own SSL properties, I thought it best to
have this quiet period be consistent across all factories since there will
only be one watcher thread. I chose not to make it configurable (one less
thing to worry about), but I’m happy to change that.
> I'm unsure about the configuration name. Have you considered alternative
like "ssl.auto.reload" or "ssl.stores.auto.reload"?
Agreed. I changed it to "ssl.auto.reload".
Let me know what you think.
Thanks again,
Moncef
On Wed, Mar 12, 2025 at 11:08 AM Mickael Maison <mi...@gmail.com>
wrote:
> Hi,
>
> Thanks for the KIP, it seems a useful improvement. I guess this would
> supersede KIP-687.
>
> The KIP is a bit light on details.
> - Can you provide the description and default value for the new
> configuration?
> - The KIP explicitly mentions brokers, consumer and producers, I
> assume it also covers admin clients (and controllers)?
> - The KIP states "To prevent redundant reconfigurations, a quiet
> period is enforced across watched files". How long is this period? Is
> it configurable?
> - I'm unsure about the configuration name. Have you considered
> alternative like "ssl.auto.reload" or "ssl.stores.auto.reload"?
>
> Thanks,
> Mickael
>
> On Thu, Dec 5, 2024 at 6:28 PM Moncef Abboud <mo...@gmail.com>
> wrote:
> >
> > Hi Gaurav,
> >
> > Thank you for your reply.
> >
> > There is some overlap between the two. However, while KIP-687 focuses
> > primarily on brokers, this KIP targets consumers and producers, with
> > additional benefits for brokers.
> >
> > Best,
> > Moncef
> >
> > On Thu, Dec 5, 2024, 10:38 AM Gaurav Narula <ka...@gnarula.com> wrote:
> >
> > > Hi Moncef,
> > >
> > > Thank you for the KIP. It seems very similar in spirit to KIP-687 (
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-687%3A+Automatic+Reloading+of+Security+Store
> )
> > > which seems like it was approved but never fully implemented. Can you
> > > please confirm if it is the case indeed?
> > >
> > > Regards,
> > > Gaurav
> > >
> > > > On 2 Dec 2024, at 23:12, Moncef Abboud <mo...@gmail.com>
> > > wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I hope your week is off to a great start.
> > > >
> > > > I created a KIP to add support for SSL hot reloading.
> > > > https://cwiki.apache.org/confluence/x/eIrREw
> > > >
> > > > Thank you for your feedback!
> > > >
> > > >
> > > > Moncef
> > >
> > >
>
--
Moncef ABBOUD