You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@trafficserver.apache.org by Nikhil Marathe <nm...@linkedin.com.INVALID> on 2015/08/05 00:30:09 UTC
TLS Session Ticket: Key Rotation
Hi,
This is Nikhil from Linkedin Engineering.
A Key Rotation feature has been added to TLS session tickets; details:
http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/2084
At present, this feature relies on periodic execution of traffic_line -x to
reload new keys. However traffic_line -x reloads entire configuration, and
so has a much wider impact than needed.
In order to address this concern and to localize the impact of reloading
keys, we would like to propose following approach:
ATS will schedule periodic continuation which checks the session ticket key
file. The reload interval will be records.configurable. If the session
ticket key file has been modified, ATS will reload the keys from the file.
Please comment on the proposal!
Regards,
Nikhil Marathe
Re: TLS Session Ticket: Key Rotation
Posted by Leif Hedstrom <zw...@apache.org>.
> On Aug 4, 2015, at 4:30 PM, Nikhil Marathe <nm...@linkedin.com.INVALID> wrote:
>
> Hi,
>
> This is Nikhil from Linkedin Engineering.
>
> A Key Rotation feature has been added to TLS session tickets; details:
> http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/2084
>
> At present, this feature relies on periodic execution of traffic_line -x to
> reload new keys. However traffic_line -x reloads entire configuration, and
> so has a much wider impact than needed.
Well, that is definitely not true. It will only reload the configuration files that have changed.
>
> In order to address this concern and to localize the impact of reloading
> keys, we would like to propose following approach:
>
> ATS will schedule periodic continuation which checks the session ticket key
> file. The reload interval will be records.configurable. If the session
> ticket key file has been modified, ATS will reload the keys from the file.
My gut feeling tells me that this should be an API. Something where you write a plugin, that when it decides to, tells ATS to reload the specified config file regardless of time stamp. This would slow for a generic pattern that allows programmatic control over config reloads.
I'm not at a computer right now, but there might be APIs in the Mgmt APIs to do this (talking to the admin socket)? If not, it should ;) If so, we could implement this as an external watcher process too?
Cheers,
-- Leif
>
> Please comment on the proposal!
>
> Regards,
> Nikhil Marathe
Re: TLS Session Ticket: Key Rotation
Posted by Leif Hedstrom <zw...@apache.org>.
> On Aug 5, 2015, at 4:10 AM, Bret Palsson <br...@gmail.com> wrote:
>
> The problem with reloading SSL configuration is if there is a problem with
> one of your certs, say a permission issue, ATS will unload all the certs
> from the running process and still accept traffic causing SSL errors.
That sounds like a bug. :) Maybe this is an OpenSSL misfeature? Or is it an ATS problem? In general, the way config reloads in ATS works is that they either completely succeeds or completely fail. Maybe that’s not possible with the certs? It’s a real shame that it’d leave the server in this indeterminate state while reloading. Susan or James, can we fix / improve that?
— leif
Re: TLS Session Ticket: Key Rotation
Posted by Brian Geffon <br...@gmail.com>.
Bump, any last feedback regarding my previous email?
On Wed, Aug 12, 2015 at 1:54 PM, Brian Geffon <br...@gmail.com> wrote:
> I'd like to close the loop on this discussion. In general I believe there
> is a consensus that perhaps ssl_multicert is not the place to deal with
> ticket rotation and that if you're willing to have global session tickets
> (meaning not tied to a specific domain) then the implementation that would
> accomplish this would be trivial compared to the current approach where
> rotation would happen with traffic_line -x on a per domain basis coming
> from ssl_multicert. Which I strongly agree with if this is something that
> most people believe would remain secure and is acceptable...? Additionally,
> in the long run if something more complicated was required we could
> implement it via early ssl hooks and a plugin.
>
> Does this accurately sum things up?
>
> Nikhil / Bret, do you guys think rotating a global ticket file via
> records.config works both from a security and operational standpoint?
>
> Thanks everyone for the great feedback!
> Brian
>
> On Fri, Aug 7, 2015 at 1:10 AM, Bret Palsson <br...@gmail.com> wrote:
>
>> On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org> wrote:
>>
>> >
>> > > On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org> wrote:
>> > >
>> > >
>> > >> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org> wrote:
>> > >>
>> > >>
>> > >>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
>> > shinrich@network-geographics.com> wrote:
>> > >>>
>> > >>> I would argue that the specification of the session ticket key in
>> the
>> > ssl_multicert.config file is inappropriate at least as the primary
>> > mechanism. It seems that for the common case, you don't need to use
>> > different session keys for different domains. You could specify one key
>> > file set in records.config.
>> > >>
>> > >> Yes, I think this is a promising approach.
>> > >
>> > >
>> > > I like that too. I don’t know how easily this can be done as an
>> > overridable configuration, without introducing a lot of additional
>> > complexity (remember, the HttpSM needs to generally be available for
>> you to
>> > use overridable configs).
>> >
>> > You can't override this at the HTTP layer since you already had to deal
>> > with session tickets when you terminated the TLS session.
>> >
>> > > If it can’t be overridable, would it make sense to have an API as well
>> > for this? Such that a plugin can set the session keys, which would then
>> let
>> > you manage the rotation in any way that you seem fit.
>> >
>> > It would be great to have more flexibility in TLS. As I may have implied
>> > before, I think ssl_multicert.config is stretching the limits of what it
>> > can reasonably express :)
>> >
>>
>> I very much agree with this!
>>
>>
>> > J
>>
>>
>>
>>
>> --
>> Bret Palsson | https://cobook.co/bretep
>>
>
>
Re: TLS Session Ticket: Key Rotation
Posted by Thomas Jackson <ja...@gmail.com>.
Reply inline
On Tue, Aug 25, 2015 at 2:58 PM, James Peach <jp...@apache.org> wrote:
>
> > On Aug 25, 2015, at 2:13 PM, Thomas Jackson <ja...@apache.org>
> wrote:
> >
> > So as I read this thread we have a couple points:
> >
> > - Global key instead of per-ssl_multicert line
> > - Fix issue with traffic_line -x not being transactional
> >
> > The one point that seems to have been dropped is how the keys themselves
> > get rotated. I personally don't particuarly like the idea of having an
> > external process rotating a file and then calling traffic_line -x to
> rotate
> > the keys.
>
> If you have more than one server on the same VIP, then you have to
> co-ordinate session ticket keys, in which case having to run "traffic_ctl
> config reload" seems quite reasonable to me.
>
This actually depends. For very small setups (single hosts) this is a
non-issue (as you implied). For some installs (such as ours) traffic is
consistently hashed to reals-- so sharing isn't required. This is I think
where Leif's thought of doing this in plugin space starts to make sense--
because everyone has different requirements on how the keys are rotated.
>
> > I'd actually like it if ATS (core or plugin) could do the
> > rotation on its own-- and I'd like to make that the default. Today the
> > default for tickets creates one in memory, and then uses it until
> > trafficserver is restarted-- which is potentially bad for PFC (since
> uptime
> > should be high).
>
> Yeh I can see there is a case to do better in the default configuration.
> However, we need to balance that against the additional complexity.
>
Agree that we have to balance it, but in addition to complexity we should
probably consider security with the favor being more secure instead of less
secure. IIRC keys are enabled by default-- but we don't rotate the keys
(which is not good). So, I'd be equally fine with tickets being disabled by
default, tickets without rotation isn't really safe :/
>
> > IMO it makes sense to put a feature like this in the core
> > (since Tickets are a core feature), but I can understand how if you
> wanted
> > to do something more complicated (shared keys, etc.) that would make more
> > sense in plugin-space. Maybe we can have some way of shipping a basic
> > implementation (in either the core or a simple plugin) which is enabled
> by
> > default (if tickets are enabled). Thoughts?
>
> Here's a straw person proposal:
>
> traffic_ctl ssl rotate-ticket-key [OPTIONAL-48-BYTES]
>
> This is a trivial line to add to crontab and could be used with shared
> ticket keys and implicit (default) ticket keys.
>
This could work, from the command you posted it seems that we would have
configured ATS with the number of keys and it will do the rotation on its
own, just adding those bytes as the newest key-- right?
> > On Thu, Aug 20, 2015 at 11:27 AM, Bret Palsson <br...@gmail.com> wrote:
> >
> >> I sent a response on Aug 12. Here was what I sent. Are my messages being
> >> moderated? I'm not seeing the email in the archives.
> >>
> >>
> https://mail-archives.apache.org/mod_mbox/trafficserver-dev/201508.mbox/browser
> >>
> >> ---------- Forwarded message ----------
> >> From: Bret Palsson <br...@gmail.com>
> >> Date: Wed, Aug 12, 2015 at 8:57 AM
> >> Subject: Re: TLS Session Ticket: Key Rotation
> >> To: dev@trafficserver.apache.org
> >>
> >>
> >> Brian:
> >>
> >> Thanks for summarizing this thread!
> >>
> >> That would work operationally. I think there still there needs to be a
> safe
> >> way to force a rotation without having to restart traffic_server and
> >> reloading all the configs via traffic_line -x.
> >>
> >> -Bret
> >>
> >>
> >>
> >> On Tue, Aug 11, 2015 at 10:54 PM, Brian Geffon <br...@gmail.com>
> >> wrote:
> >>
> >>> I'd like to close the loop on this discussion. In general I believe
> there
> >>> is a consensus that perhaps ssl_multicert is not the place to deal with
> >>> ticket rotation and that if you're willing to have global session
> tickets
> >>> (meaning not tied to a specific domain) then the implementation that
> >> would
> >>> accomplish this would be trivial compared to the current approach where
> >>> rotation would happen with traffic_line -x on a per domain basis coming
> >>> from ssl_multicert. Which I strongly agree with if this is something
> that
> >>> most people believe would remain secure and is acceptable...?
> >> Additionally,
> >>> in the long run if something more complicated was required we could
> >>> implement it via early ssl hooks and a plugin.
> >>>
> >>> Does this accurately sum things up?
> >>>
> >>> Nikhil / Bret, do you guys think rotating a global ticket file via
> >>> records.config works both from a security and operational standpoint?
> >>>
> >>> Thanks everyone for the great feedback!
> >>> Brian
> >>>
> >>> On Fri, Aug 7, 2015 at 1:10 AM, Bret Palsson <br...@gmail.com> wrote:
> >>>
> >>>> On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org>
> >> wrote:
> >>>>
> >>>>>
> >>>>>> On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org>
> >> wrote:
> >>>>>>
> >>>>>>
> >>>>>>> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org>
> >> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
> >>>>> shinrich@network-geographics.com> wrote:
> >>>>>>>>
> >>>>>>>> I would argue that the specification of the session ticket key in
> >>> the
> >>>>> ssl_multicert.config file is inappropriate at least as the primary
> >>>>> mechanism. It seems that for the common case, you don't need to use
> >>>>> different session keys for different domains. You could specify one
> >>> key
> >>>>> file set in records.config.
> >>>>>>>
> >>>>>>> Yes, I think this is a promising approach.
> >>>>>>
> >>>>>>
> >>>>>> I like that too. I don’t know how easily this can be done as an
> >>>>> overridable configuration, without introducing a lot of additional
> >>>>> complexity (remember, the HttpSM needs to generally be available for
> >>> you
> >>>> to
> >>>>> use overridable configs).
> >>>>>
> >>>>> You can't override this at the HTTP layer since you already had to
> >> deal
> >>>>> with session tickets when you terminated the TLS session.
> >>>>>
> >>>>>> If it can’t be overridable, would it make sense to have an API as
> >>> well
> >>>>> for this? Such that a plugin can set the session keys, which would
> >> then
> >>>> let
> >>>>> you manage the rotation in any way that you seem fit.
> >>>>>
> >>>>> It would be great to have more flexibility in TLS. As I may have
> >>> implied
> >>>>> before, I think ssl_multicert.config is stretching the limits of what
> >>> it
> >>>>> can reasonably express :)
> >>>>>
> >>>>
> >>>> I very much agree with this!
> >>>>
> >>>>
> >>>>> J
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Bret Palsson | https://cobook.co/bretep
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Bret Palsson | https://cobook.co/bretep
> >>
> >>
> >>
> >> --
> >> Bret Palsson | https://cobook.co/bretep
> >>
>
>
Re: TLS Session Ticket: Key Rotation
Posted by James Peach <jp...@apache.org>.
> On Aug 25, 2015, at 2:13 PM, Thomas Jackson <ja...@apache.org> wrote:
>
> So as I read this thread we have a couple points:
>
> - Global key instead of per-ssl_multicert line
> - Fix issue with traffic_line -x not being transactional
>
> The one point that seems to have been dropped is how the keys themselves
> get rotated. I personally don't particuarly like the idea of having an
> external process rotating a file and then calling traffic_line -x to rotate
> the keys.
If you have more than one server on the same VIP, then you have to co-ordinate session ticket keys, in which case having to run "traffic_ctl config reload" seems quite reasonable to me.
> I'd actually like it if ATS (core or plugin) could do the
> rotation on its own-- and I'd like to make that the default. Today the
> default for tickets creates one in memory, and then uses it until
> trafficserver is restarted-- which is potentially bad for PFC (since uptime
> should be high).
Yeh I can see there is a case to do better in the default configuration. However, we need to balance that against the additional complexity.
> IMO it makes sense to put a feature like this in the core
> (since Tickets are a core feature), but I can understand how if you wanted
> to do something more complicated (shared keys, etc.) that would make more
> sense in plugin-space. Maybe we can have some way of shipping a basic
> implementation (in either the core or a simple plugin) which is enabled by
> default (if tickets are enabled). Thoughts?
Here's a straw person proposal:
traffic_ctl ssl rotate-ticket-key [OPTIONAL-48-BYTES]
This is a trivial line to add to crontab and could be used with shared ticket keys and implicit (default) ticket keys.
> On Thu, Aug 20, 2015 at 11:27 AM, Bret Palsson <br...@gmail.com> wrote:
>
>> I sent a response on Aug 12. Here was what I sent. Are my messages being
>> moderated? I'm not seeing the email in the archives.
>>
>> https://mail-archives.apache.org/mod_mbox/trafficserver-dev/201508.mbox/browser
>>
>> ---------- Forwarded message ----------
>> From: Bret Palsson <br...@gmail.com>
>> Date: Wed, Aug 12, 2015 at 8:57 AM
>> Subject: Re: TLS Session Ticket: Key Rotation
>> To: dev@trafficserver.apache.org
>>
>>
>> Brian:
>>
>> Thanks for summarizing this thread!
>>
>> That would work operationally. I think there still there needs to be a safe
>> way to force a rotation without having to restart traffic_server and
>> reloading all the configs via traffic_line -x.
>>
>> -Bret
>>
>>
>>
>> On Tue, Aug 11, 2015 at 10:54 PM, Brian Geffon <br...@gmail.com>
>> wrote:
>>
>>> I'd like to close the loop on this discussion. In general I believe there
>>> is a consensus that perhaps ssl_multicert is not the place to deal with
>>> ticket rotation and that if you're willing to have global session tickets
>>> (meaning not tied to a specific domain) then the implementation that
>> would
>>> accomplish this would be trivial compared to the current approach where
>>> rotation would happen with traffic_line -x on a per domain basis coming
>>> from ssl_multicert. Which I strongly agree with if this is something that
>>> most people believe would remain secure and is acceptable...?
>> Additionally,
>>> in the long run if something more complicated was required we could
>>> implement it via early ssl hooks and a plugin.
>>>
>>> Does this accurately sum things up?
>>>
>>> Nikhil / Bret, do you guys think rotating a global ticket file via
>>> records.config works both from a security and operational standpoint?
>>>
>>> Thanks everyone for the great feedback!
>>> Brian
>>>
>>> On Fri, Aug 7, 2015 at 1:10 AM, Bret Palsson <br...@gmail.com> wrote:
>>>
>>>> On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org>
>> wrote:
>>>>
>>>>>
>>>>>> On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org>
>> wrote:
>>>>>>
>>>>>>
>>>>>>> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org>
>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
>>>>> shinrich@network-geographics.com> wrote:
>>>>>>>>
>>>>>>>> I would argue that the specification of the session ticket key in
>>> the
>>>>> ssl_multicert.config file is inappropriate at least as the primary
>>>>> mechanism. It seems that for the common case, you don't need to use
>>>>> different session keys for different domains. You could specify one
>>> key
>>>>> file set in records.config.
>>>>>>>
>>>>>>> Yes, I think this is a promising approach.
>>>>>>
>>>>>>
>>>>>> I like that too. I don’t know how easily this can be done as an
>>>>> overridable configuration, without introducing a lot of additional
>>>>> complexity (remember, the HttpSM needs to generally be available for
>>> you
>>>> to
>>>>> use overridable configs).
>>>>>
>>>>> You can't override this at the HTTP layer since you already had to
>> deal
>>>>> with session tickets when you terminated the TLS session.
>>>>>
>>>>>> If it can’t be overridable, would it make sense to have an API as
>>> well
>>>>> for this? Such that a plugin can set the session keys, which would
>> then
>>>> let
>>>>> you manage the rotation in any way that you seem fit.
>>>>>
>>>>> It would be great to have more flexibility in TLS. As I may have
>>> implied
>>>>> before, I think ssl_multicert.config is stretching the limits of what
>>> it
>>>>> can reasonably express :)
>>>>>
>>>>
>>>> I very much agree with this!
>>>>
>>>>
>>>>> J
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Bret Palsson | https://cobook.co/bretep
>>>>
>>>
>>
>>
>>
>> --
>> Bret Palsson | https://cobook.co/bretep
>>
>>
>>
>> --
>> Bret Palsson | https://cobook.co/bretep
>>
Re: TLS Session Ticket: Key Rotation
Posted by Thomas Jackson <ja...@apache.org>.
So as I read this thread we have a couple points:
- Global key instead of per-ssl_multicert line
- Fix issue with traffic_line -x not being transactional
The one point that seems to have been dropped is how the keys themselves
get rotated. I personally don't particuarly like the idea of having an
external process rotating a file and then calling traffic_line -x to rotate
the keys. I'd actually like it if ATS (core or plugin) could do the
rotation on its own-- and I'd like to make that the default. Today the
default for tickets creates one in memory, and then uses it until
trafficserver is restarted-- which is potentially bad for PFC (since uptime
should be high). IMO it makes sense to put a feature like this in the core
(since Tickets are a core feature), but I can understand how if you wanted
to do something more complicated (shared keys, etc.) that would make more
sense in plugin-space. Maybe we can have some way of shipping a basic
implementation (in either the core or a simple plugin) which is enabled by
default (if tickets are enabled). Thoughts?
On Thu, Aug 20, 2015 at 11:27 AM, Bret Palsson <br...@gmail.com> wrote:
> I sent a response on Aug 12. Here was what I sent. Are my messages being
> moderated? I'm not seeing the email in the archives.
>
> https://mail-archives.apache.org/mod_mbox/trafficserver-dev/201508.mbox/browser
>
> ---------- Forwarded message ----------
> From: Bret Palsson <br...@gmail.com>
> Date: Wed, Aug 12, 2015 at 8:57 AM
> Subject: Re: TLS Session Ticket: Key Rotation
> To: dev@trafficserver.apache.org
>
>
> Brian:
>
> Thanks for summarizing this thread!
>
> That would work operationally. I think there still there needs to be a safe
> way to force a rotation without having to restart traffic_server and
> reloading all the configs via traffic_line -x.
>
> -Bret
>
>
>
> On Tue, Aug 11, 2015 at 10:54 PM, Brian Geffon <br...@gmail.com>
> wrote:
>
> > I'd like to close the loop on this discussion. In general I believe there
> > is a consensus that perhaps ssl_multicert is not the place to deal with
> > ticket rotation and that if you're willing to have global session tickets
> > (meaning not tied to a specific domain) then the implementation that
> would
> > accomplish this would be trivial compared to the current approach where
> > rotation would happen with traffic_line -x on a per domain basis coming
> > from ssl_multicert. Which I strongly agree with if this is something that
> > most people believe would remain secure and is acceptable...?
> Additionally,
> > in the long run if something more complicated was required we could
> > implement it via early ssl hooks and a plugin.
> >
> > Does this accurately sum things up?
> >
> > Nikhil / Bret, do you guys think rotating a global ticket file via
> > records.config works both from a security and operational standpoint?
> >
> > Thanks everyone for the great feedback!
> > Brian
> >
> > On Fri, Aug 7, 2015 at 1:10 AM, Bret Palsson <br...@gmail.com> wrote:
> >
> > > On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org>
> wrote:
> > >
> > > >
> > > > > On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org>
> wrote:
> > > > >
> > > > >
> > > > >> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org>
> wrote:
> > > > >>
> > > > >>
> > > > >>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
> > > > shinrich@network-geographics.com> wrote:
> > > > >>>
> > > > >>> I would argue that the specification of the session ticket key in
> > the
> > > > ssl_multicert.config file is inappropriate at least as the primary
> > > > mechanism. It seems that for the common case, you don't need to use
> > > > different session keys for different domains. You could specify one
> > key
> > > > file set in records.config.
> > > > >>
> > > > >> Yes, I think this is a promising approach.
> > > > >
> > > > >
> > > > > I like that too. I don’t know how easily this can be done as an
> > > > overridable configuration, without introducing a lot of additional
> > > > complexity (remember, the HttpSM needs to generally be available for
> > you
> > > to
> > > > use overridable configs).
> > > >
> > > > You can't override this at the HTTP layer since you already had to
> deal
> > > > with session tickets when you terminated the TLS session.
> > > >
> > > > > If it can’t be overridable, would it make sense to have an API as
> > well
> > > > for this? Such that a plugin can set the session keys, which would
> then
> > > let
> > > > you manage the rotation in any way that you seem fit.
> > > >
> > > > It would be great to have more flexibility in TLS. As I may have
> > implied
> > > > before, I think ssl_multicert.config is stretching the limits of what
> > it
> > > > can reasonably express :)
> > > >
> > >
> > > I very much agree with this!
> > >
> > >
> > > > J
> > >
> > >
> > >
> > >
> > > --
> > > Bret Palsson | https://cobook.co/bretep
> > >
> >
>
>
>
> --
> Bret Palsson | https://cobook.co/bretep
>
>
>
> --
> Bret Palsson | https://cobook.co/bretep
>
Fwd: TLS Session Ticket: Key Rotation
Posted by Bret Palsson <br...@gmail.com>.
I sent a response on Aug 12. Here was what I sent. Are my messages being
moderated? I'm not seeing the email in the archives.
https://mail-archives.apache.org/mod_mbox/trafficserver-dev/201508.mbox/browser
---------- Forwarded message ----------
From: Bret Palsson <br...@gmail.com>
Date: Wed, Aug 12, 2015 at 8:57 AM
Subject: Re: TLS Session Ticket: Key Rotation
To: dev@trafficserver.apache.org
Brian:
Thanks for summarizing this thread!
That would work operationally. I think there still there needs to be a safe
way to force a rotation without having to restart traffic_server and
reloading all the configs via traffic_line -x.
-Bret
On Tue, Aug 11, 2015 at 10:54 PM, Brian Geffon <br...@gmail.com>
wrote:
> I'd like to close the loop on this discussion. In general I believe there
> is a consensus that perhaps ssl_multicert is not the place to deal with
> ticket rotation and that if you're willing to have global session tickets
> (meaning not tied to a specific domain) then the implementation that would
> accomplish this would be trivial compared to the current approach where
> rotation would happen with traffic_line -x on a per domain basis coming
> from ssl_multicert. Which I strongly agree with if this is something that
> most people believe would remain secure and is acceptable...? Additionally,
> in the long run if something more complicated was required we could
> implement it via early ssl hooks and a plugin.
>
> Does this accurately sum things up?
>
> Nikhil / Bret, do you guys think rotating a global ticket file via
> records.config works both from a security and operational standpoint?
>
> Thanks everyone for the great feedback!
> Brian
>
> On Fri, Aug 7, 2015 at 1:10 AM, Bret Palsson <br...@gmail.com> wrote:
>
> > On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org> wrote:
> >
> > >
> > > > On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org> wrote:
> > > >
> > > >
> > > >> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org> wrote:
> > > >>
> > > >>
> > > >>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
> > > shinrich@network-geographics.com> wrote:
> > > >>>
> > > >>> I would argue that the specification of the session ticket key in
> the
> > > ssl_multicert.config file is inappropriate at least as the primary
> > > mechanism. It seems that for the common case, you don't need to use
> > > different session keys for different domains. You could specify one
> key
> > > file set in records.config.
> > > >>
> > > >> Yes, I think this is a promising approach.
> > > >
> > > >
> > > > I like that too. I don’t know how easily this can be done as an
> > > overridable configuration, without introducing a lot of additional
> > > complexity (remember, the HttpSM needs to generally be available for
> you
> > to
> > > use overridable configs).
> > >
> > > You can't override this at the HTTP layer since you already had to deal
> > > with session tickets when you terminated the TLS session.
> > >
> > > > If it can’t be overridable, would it make sense to have an API as
> well
> > > for this? Such that a plugin can set the session keys, which would then
> > let
> > > you manage the rotation in any way that you seem fit.
> > >
> > > It would be great to have more flexibility in TLS. As I may have
> implied
> > > before, I think ssl_multicert.config is stretching the limits of what
> it
> > > can reasonably express :)
> > >
> >
> > I very much agree with this!
> >
> >
> > > J
> >
> >
> >
> >
> > --
> > Bret Palsson | https://cobook.co/bretep
> >
>
--
Bret Palsson | https://cobook.co/bretep
--
Bret Palsson | https://cobook.co/bretep
Re: TLS Session Ticket: Key Rotation
Posted by Brian Geffon <br...@gmail.com>.
I'd like to close the loop on this discussion. In general I believe there
is a consensus that perhaps ssl_multicert is not the place to deal with
ticket rotation and that if you're willing to have global session tickets
(meaning not tied to a specific domain) then the implementation that would
accomplish this would be trivial compared to the current approach where
rotation would happen with traffic_line -x on a per domain basis coming
from ssl_multicert. Which I strongly agree with if this is something that
most people believe would remain secure and is acceptable...? Additionally,
in the long run if something more complicated was required we could
implement it via early ssl hooks and a plugin.
Does this accurately sum things up?
Nikhil / Bret, do you guys think rotating a global ticket file via
records.config works both from a security and operational standpoint?
Thanks everyone for the great feedback!
Brian
On Fri, Aug 7, 2015 at 1:10 AM, Bret Palsson <br...@gmail.com> wrote:
> On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org> wrote:
>
> >
> > > On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org> wrote:
> > >
> > >
> > >> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org> wrote:
> > >>
> > >>
> > >>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
> > shinrich@network-geographics.com> wrote:
> > >>>
> > >>> I would argue that the specification of the session ticket key in the
> > ssl_multicert.config file is inappropriate at least as the primary
> > mechanism. It seems that for the common case, you don't need to use
> > different session keys for different domains. You could specify one key
> > file set in records.config.
> > >>
> > >> Yes, I think this is a promising approach.
> > >
> > >
> > > I like that too. I don’t know how easily this can be done as an
> > overridable configuration, without introducing a lot of additional
> > complexity (remember, the HttpSM needs to generally be available for you
> to
> > use overridable configs).
> >
> > You can't override this at the HTTP layer since you already had to deal
> > with session tickets when you terminated the TLS session.
> >
> > > If it can’t be overridable, would it make sense to have an API as well
> > for this? Such that a plugin can set the session keys, which would then
> let
> > you manage the rotation in any way that you seem fit.
> >
> > It would be great to have more flexibility in TLS. As I may have implied
> > before, I think ssl_multicert.config is stretching the limits of what it
> > can reasonably express :)
> >
>
> I very much agree with this!
>
>
> > J
>
>
>
>
> --
> Bret Palsson | https://cobook.co/bretep
>
Re: TLS Session Ticket: Key Rotation
Posted by Bret Palsson <br...@gmail.com>.
On Thu, Aug 6, 2015 at 10:08 AM, James Peach <jp...@apache.org> wrote:
>
> > On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org> wrote:
> >
> >
> >> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org> wrote:
> >>
> >>
> >>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <
> shinrich@network-geographics.com> wrote:
> >>>
> >>> I would argue that the specification of the session ticket key in the
> ssl_multicert.config file is inappropriate at least as the primary
> mechanism. It seems that for the common case, you don't need to use
> different session keys for different domains. You could specify one key
> file set in records.config.
> >>
> >> Yes, I think this is a promising approach.
> >
> >
> > I like that too. I don’t know how easily this can be done as an
> overridable configuration, without introducing a lot of additional
> complexity (remember, the HttpSM needs to generally be available for you to
> use overridable configs).
>
> You can't override this at the HTTP layer since you already had to deal
> with session tickets when you terminated the TLS session.
>
> > If it can’t be overridable, would it make sense to have an API as well
> for this? Such that a plugin can set the session keys, which would then let
> you manage the rotation in any way that you seem fit.
>
> It would be great to have more flexibility in TLS. As I may have implied
> before, I think ssl_multicert.config is stretching the limits of what it
> can reasonably express :)
>
I very much agree with this!
> J
--
Bret Palsson | https://cobook.co/bretep
Re: TLS Session Ticket: Key Rotation
Posted by James Peach <jp...@apache.org>.
> On Aug 6, 2015, at 9:56 AM, Leif Hedstrom <zw...@apache.org> wrote:
>
>
>> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org> wrote:
>>
>>
>>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <sh...@network-geographics.com> wrote:
>>>
>>> I would argue that the specification of the session ticket key in the ssl_multicert.config file is inappropriate at least as the primary mechanism. It seems that for the common case, you don't need to use different session keys for different domains. You could specify one key file set in records.config.
>>
>> Yes, I think this is a promising approach.
>
>
> I like that too. I don’t know how easily this can be done as an overridable configuration, without introducing a lot of additional complexity (remember, the HttpSM needs to generally be available for you to use overridable configs).
You can't override this at the HTTP layer since you already had to deal with session tickets when you terminated the TLS session.
> If it can’t be overridable, would it make sense to have an API as well for this? Such that a plugin can set the session keys, which would then let you manage the rotation in any way that you seem fit.
It would be great to have more flexibility in TLS. As I may have implied before, I think ssl_multicert.config is stretching the limits of what it can reasonably express :)
J
Re: TLS Session Ticket: Key Rotation
Posted by Leif Hedstrom <zw...@apache.org>.
> On Aug 5, 2015, at 10:16 AM, James Peach <jp...@apache.org> wrote:
>
>
>> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <sh...@network-geographics.com> wrote:
>>
>> I would argue that the specification of the session ticket key in the ssl_multicert.config file is inappropriate at least as the primary mechanism. It seems that for the common case, you don't need to use different session keys for different domains. You could specify one key file set in records.config.
>
> Yes, I think this is a promising approach.
I like that too. I don’t know how easily this can be done as an overridable configuration, without introducing a lot of additional complexity (remember, the HttpSM needs to generally be available for you to use overridable configs).
If it can’t be overridable, would it make sense to have an API as well for this? Such that a plugin can set the session keys, which would then let you manage the rotation in any way that you seem fit.
— leif
Re: TLS Session Ticket: Key Rotation
Posted by James Peach <jp...@apache.org>.
> On Aug 5, 2015, at 8:22 AM, Susan Hinrichs <sh...@network-geographics.com> wrote:
>
> I would argue that the specification of the session ticket key in the ssl_multicert.config file is inappropriate at least as the primary mechanism. It seems that for the common case, you don't need to use different session keys for different domains. You could specify one key file set in records.config.
Yes, I think this is a promising approach.
> Then you are not reloading ssl_multicert.config, which I agree for a large file is a somewhat dicey proposition.
>
> However, there is a setting to allow errors on the ssl_multicert reload (proxy.config.ssl.server.multicert.exit_on_load_fail) so I don't think you are condemned to the SSL failure case you outline.
I don't remember what happens without looking at the code, but ISTR there was a time when a reload failure would just log an error preserve the existing configuration. Does that still happen?
> There are several bugs open in the ticket area to simply ticket control: TS-3528, TS-3371, TS-3742. I'm curious if you have a use case where you do need to differentiate ticket keys by domains?
>
> I would also lobby for a Plugin interface to update the list of tickets. This would allow people to implement any reload strategy they wish. It would also support people who do not want to store ticket keys through the file system.
>
> On 8/5/2015 5:10 AM, Bret Palsson wrote:
>> The problem with reloading SSL configuration is if there is a problem with
>> one of your certs, say a permission issue, ATS will unload all the certs
>> from the running process and still accept traffic causing SSL errors.
>>
>> Being able to reload just the keys is much safer than trying to reload the
>> world, or configs that have changed. Especially since key rotation will
>> likely be automatic on a schedule.
>>
>> -Bret
>>
>> On Tue, Aug 4, 2015 at 8:56 PM, James Peach <jp...@apache.org> wrote:
>>
>>>> On Aug 4, 2015, at 3:30 PM, Nikhil Marathe <nm...@linkedin.com.INVALID>
>>> wrote:
>>>> Hi,
>>>>
>>>> This is Nikhil from Linkedin Engineering.
>>>>
>>>> A Key Rotation feature has been added to TLS session tickets; details:
>>>> http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/2084
>>>>
>>>> At present, this feature relies on periodic execution of traffic_line -x
>>> to
>>>> reload new keys. However traffic_line -x reloads entire configuration,
>>> and
>>>> so has a much wider impact than needed.
>>>>
>>>> In order to address this concern and to localize the impact of reloading
>>>> keys, we would like to propose following approach:
>>>>
>>>> ATS will schedule periodic continuation which checks the session ticket
>>> key
>>>> file. The reload interval will be records.configurable. If the session
>>>> ticket key file has been modified, ATS will reload the keys from the
>>> file.
>>>
>>> Hi Nikhil,
>>>
>>> At the time we discussed the need for this and my view is still that it is
>>> not necessary. Reloading the SSL configuration should be completely
>>> harmless and it seems very straight forward for the job that populates the
>>> new keys to call traffic_line at the right time.
>>>
>>> cheers,
>>> James
>>
>>
>>
>
Re: TLS Session Ticket: Key Rotation
Posted by Susan Hinrichs <sh...@network-geographics.com>.
I would argue that the specification of the session ticket key in the
ssl_multicert.config file is inappropriate at least as the primary
mechanism. It seems that for the common case, you don't need to use
different session keys for different domains. You could specify one key
file set in records.config. Then you are not reloading
ssl_multicert.config, which I agree for a large file is a somewhat dicey
proposition.
However, there is a setting to allow errors on the ssl_multicert reload
(proxy.config.ssl.server.multicert.exit_on_load_fail) so I don't think
you are condemned to the SSL failure case you outline.
There are several bugs open in the ticket area to simply ticket control:
TS-3528, TS-3371, TS-3742. I'm curious if you have a use case where you
do need to differentiate ticket keys by domains?
I would also lobby for a Plugin interface to update the list of
tickets. This would allow people to implement any reload strategy they
wish. It would also support people who do not want to store ticket keys
through the file system.
On 8/5/2015 5:10 AM, Bret Palsson wrote:
> The problem with reloading SSL configuration is if there is a problem with
> one of your certs, say a permission issue, ATS will unload all the certs
> from the running process and still accept traffic causing SSL errors.
>
> Being able to reload just the keys is much safer than trying to reload the
> world, or configs that have changed. Especially since key rotation will
> likely be automatic on a schedule.
>
> -Bret
>
> On Tue, Aug 4, 2015 at 8:56 PM, James Peach <jp...@apache.org> wrote:
>
>>> On Aug 4, 2015, at 3:30 PM, Nikhil Marathe <nm...@linkedin.com.INVALID>
>> wrote:
>>> Hi,
>>>
>>> This is Nikhil from Linkedin Engineering.
>>>
>>> A Key Rotation feature has been added to TLS session tickets; details:
>>> http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/2084
>>>
>>> At present, this feature relies on periodic execution of traffic_line -x
>> to
>>> reload new keys. However traffic_line -x reloads entire configuration,
>> and
>>> so has a much wider impact than needed.
>>>
>>> In order to address this concern and to localize the impact of reloading
>>> keys, we would like to propose following approach:
>>>
>>> ATS will schedule periodic continuation which checks the session ticket
>> key
>>> file. The reload interval will be records.configurable. If the session
>>> ticket key file has been modified, ATS will reload the keys from the
>> file.
>>
>> Hi Nikhil,
>>
>> At the time we discussed the need for this and my view is still that it is
>> not necessary. Reloading the SSL configuration should be completely
>> harmless and it seems very straight forward for the job that populates the
>> new keys to call traffic_line at the right time.
>>
>> cheers,
>> James
>
>
>
Re: TLS Session Ticket: Key Rotation
Posted by Bret Palsson <br...@gmail.com>.
The problem with reloading SSL configuration is if there is a problem with
one of your certs, say a permission issue, ATS will unload all the certs
from the running process and still accept traffic causing SSL errors.
Being able to reload just the keys is much safer than trying to reload the
world, or configs that have changed. Especially since key rotation will
likely be automatic on a schedule.
-Bret
On Tue, Aug 4, 2015 at 8:56 PM, James Peach <jp...@apache.org> wrote:
>
> > On Aug 4, 2015, at 3:30 PM, Nikhil Marathe <nm...@linkedin.com.INVALID>
> wrote:
> >
> > Hi,
> >
> > This is Nikhil from Linkedin Engineering.
> >
> > A Key Rotation feature has been added to TLS session tickets; details:
> > http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/2084
> >
> > At present, this feature relies on periodic execution of traffic_line -x
> to
> > reload new keys. However traffic_line -x reloads entire configuration,
> and
> > so has a much wider impact than needed.
> >
> > In order to address this concern and to localize the impact of reloading
> > keys, we would like to propose following approach:
> >
> > ATS will schedule periodic continuation which checks the session ticket
> key
> > file. The reload interval will be records.configurable. If the session
> > ticket key file has been modified, ATS will reload the keys from the
> file.
>
> Hi Nikhil,
>
> At the time we discussed the need for this and my view is still that it is
> not necessary. Reloading the SSL configuration should be completely
> harmless and it seems very straight forward for the job that populates the
> new keys to call traffic_line at the right time.
>
> cheers,
> James
--
Bret Palsson | https://cobook.co/bretep
Re: TLS Session Ticket: Key Rotation
Posted by James Peach <jp...@apache.org>.
> On Aug 4, 2015, at 3:30 PM, Nikhil Marathe <nm...@linkedin.com.INVALID> wrote:
>
> Hi,
>
> This is Nikhil from Linkedin Engineering.
>
> A Key Rotation feature has been added to TLS session tickets; details:
> http://comments.gmane.org/gmane.comp.apache.trafficserver.devel/2084
>
> At present, this feature relies on periodic execution of traffic_line -x to
> reload new keys. However traffic_line -x reloads entire configuration, and
> so has a much wider impact than needed.
>
> In order to address this concern and to localize the impact of reloading
> keys, we would like to propose following approach:
>
> ATS will schedule periodic continuation which checks the session ticket key
> file. The reload interval will be records.configurable. If the session
> ticket key file has been modified, ATS will reload the keys from the file.
Hi Nikhil,
At the time we discussed the need for this and my view is still that it is not necessary. Reloading the SSL configuration should be completely harmless and it seems very straight forward for the job that populates the new keys to call traffic_line at the right time.
cheers,
James