Shall we move SASL based CRAM-MD5 authentication out of libmesos?

[Till Toenshoff <to...@me.com>]

Dear fellow Apache Mesos developers,

as you know, Apache Mesos supports authentication on various levels - among those is the RPC-style authentication allowing frameworks and agents to authenticate against the master. Even though  this authentication interface has been modularized a long time ago, we still kept the default, SASL based challenge-response authentication mechanism, message digest 5 (aka CRAM-MD5)  authentication within libmesos. 

Modularizing of our RPC authentication has enabled you to add new authentication mechanisms. User have chosen authentication fitting their company security landscape - e.g. ticket based things like Kerberos or Mesosphere’s use of JWT. It has also come to my attention that there are users out there using directory backed (e.g.LDAP) variants - or even combinations of those like LDAP backed Kerberos.

CRAM-MD5, while still being regarded as secure, is not very convenient or flexible and therefor in my experience it is not chosen in production environments.  This in turn means that all those builds of libmesos drag SASL in as a dependency while in fact not making any use of it - and that is what I would like to have fixed (since ages). It would benefit in reducing loading times of libmesos dependent runnable and it would also reduce provisioning complexity.

To fix that, we would have SASL CRAM-MD5 be available as a module exclusively, provisioned only when really needed. That in turn means that users of this mechanism would need to additionally provide the “modules” or “modules_dir” flag to master, agent and/or framework - that would a breaking change for those that rely on the fact that CRAM-MD5 works out of the box.

I could imagine ways where the user would not even have to provide that “modules*” flag and we internal generate that data for him as a convenience function - it is another option.

Given that CRAM-MD5 is the only RPC authentication mechanism Mesos is bundling right now and given that our tests do rely on authentication to be available and tested, we will not be able to remove the dependency against SASL entirely for building and testing.

This first step would only ease the deployment.

But, there is a drawback - Windows builds currently do not support modules. So either we get modules support for Windows up and running OR we would need to let Windows be an exception to this plan for now. 

What do you think? Is it worth following this path - or do you have other suggestions?

Feel free to comment here or on https://issues.apache.org/jira/browse/MESOS-9042 <https://issues.apache.org/jira/browse/MESOS-9042>.

Thanks for your feedback,
Till

Re: Shall we move SASL based CRAM-MD5 authentication out oflibmesos?

[Andrew Schwartzmeyer <an...@schwartzmeyer.com>]

> But, there is a drawback - Windows builds currently do not support 
> modules. So either we get modules support for Windows up and running OR 
> we would need to let Windows be an exception to this plan for now.

The drawback on top of this is that this is the _only_ authentication 
mechanism on Windows currently; just saying it's vital to make sure it 
keeps working on Windows, regardless of how we do that.

On 07/01/2018 7:45 am, Till Toenshoff wrote:
> Dear fellow Apache Mesos developers,
> 
> as you know, Apache Mesos supports authentication on various levels -
> among those is the RPC-style authentication allowing frameworks and
> agents to authenticate against the master. Even though  this
> authentication interface has been modularized a long time ago, we
> still kept the default, SASL based challenge-response authentication
> mechanism, message digest 5 (aka CRAM-MD5)  authentication within
> libmesos.
> 
> Modularizing of our RPC authentication has enabled you to add new
> authentication mechanisms. User have chosen authentication fitting
> their company security landscape - e.g. ticket based things like
> Kerberos or Mesosphere’s use of JWT. It has also come to my attention
> that there are users out there using directory backed (e.g.LDAP)
> variants - or even combinations of those like LDAP backed Kerberos.
> 
> CRAM-MD5, while still being regarded as secure, is not very convenient
> or flexible and therefor in my experience it is not chosen in
> production environments.  This in turn means that all those builds of
> libmesos drag SASL in as a dependency while in fact not making any use
> of it - and that is what I would like to have fixed (since ages). It
> would benefit in reducing loading times of libmesos dependent runnable
> and it would also reduce provisioning complexity.
> 
> To fix that, we would have SASL CRAM-MD5 be available as a module
> exclusively, provisioned only when really needed. That in turn means
> that users of this mechanism would need to additionally provide the
> “modules” or “modules_dir” flag to master, agent and/or framework -
> that would a breaking change for those that rely on the fact that
> CRAM-MD5 works out of the box.
> 
> I could imagine ways where the user would not even have to provide
> that “modules*” flag and we internal generate that data for him as a
> convenience function - it is another option.
> 
> Given that CRAM-MD5 is the only RPC authentication mechanism Mesos is
> bundling right now and given that our tests do rely on authentication
> to be available and tested, we will not be able to remove the
> dependency against SASL entirely for building and testing.
> 
> This first step would only ease the deployment.
> 
> But, there is a drawback - Windows builds currently do not support
> modules. So either we get modules support for Windows up and running
> OR we would need to let Windows be an exception to this plan for now.
> 
> What do you think? Is it worth following this path - or do you have
> other suggestions?
> 
> Feel free to comment here or on
> https://issues.apache.org/jira/browse/MESOS-9042
> <https://issues.apache.org/jira/browse/MESOS-9042>.
> 
> Thanks for your feedback,
> Till

Re: Shall we move SASL based CRAM-MD5 authentication out of libmesos?

[James Peach <jp...@apache.org>]

> On Jul 2, 2018, at 12:45 AM, Till Toenshoff <to...@me.com> wrote:
> 
> Dear fellow Apache Mesos developers,
> 
> as you know, Apache Mesos supports authentication on various levels - among those is the RPC-style authentication allowing frameworks and agents to authenticate against the master. Even though  this authentication interface has been modularized a long time ago, we still kept the default, SASL based challenge-response authentication mechanism, message digest 5 (aka CRAM-MD5)  authentication within libmesos. 
> 
> Modularizing of our RPC authentication has enabled you to add new authentication mechanisms. User have chosen authentication fitting their company security landscape - e.g. ticket based things like Kerberos or Mesosphere’s use of JWT. It has also come to my attention that there are users out there using directory backed (e.g.LDAP) variants - or even combinations of those like LDAP backed Kerberos.
> 
> CRAM-MD5, while still being regarded as secure, is not very convenient or flexible and therefor in my experience it is not chosen in production environments.  This in turn means that all those builds of libmesos drag SASL in as a dependency while in fact not making any use of it - and that is what I would like to have fixed (since ages). It would benefit in reducing loading times of libmesos dependent runnable and it would also reduce provisioning complexity.
> 
> To fix that, we would have SASL CRAM-MD5 be available as a module exclusively, provisioned only when really needed. That in turn means that users of this mechanism would need to additionally provide the “modules” or “modules_dir” flag to master, agent and/or framework - that would a breaking change for those that rely on the fact that CRAM-MD5 works out of the box.
> 
> I could imagine ways where the user would not even have to provide that “modules*” flag and we internal generate that data for him as a convenience function - it is another option.
> 
> Given that CRAM-MD5 is the only RPC authentication mechanism Mesos is bundling right now and given that our tests do rely on authentication to be available and tested, we will not be able to remove the dependency against SASL entirely for building and testing.
> 
> This first step would only ease the deployment.
> 
> But, there is a drawback - Windows builds currently do not support modules. So either we get modules support for Windows up and running OR we would need to let Windows be an exception to this plan for now. 
> 
> What do you think? Is it worth following this path - or do you have other suggestions?

I’m +1 for simplifying libmesos dependencies in principle. In this case, CRAM-MD5 is the only auth mechanism available in the main Mesos tree, so we should consider the upgrade path carefully. For my usage, I can updae our puppet configs, but we need to make sure that either the old configuration continues to work, or that it hard-fails so that operators can switch to the new config.

J