[DISCUSS] Kerberos First

[Nick Allen <ni...@nickallen.org>]

I hate dealing with Kerberos.  It is a pain to setup, it is a pain to work
with, it has its own learning curve, *but it is absolutely necessary*.  Due
to the sensitive nature of Metron's use case, most of our users should be
using Kerberos as part of a defense-in-depth strategy to protect sensitive
data.

For the upcoming 0.4.0 release, we have all put in a tremendous amount of
work to make integrating Metron into a Kerberized cluster as simple as
possible.  I hated every minute of it, but the end result speaks for
itself.  Really great work, everyone.

*Proposal*

I am proposing that as a community we choose to support Kerberos first.  We
should not treat Kerberos as an optional add-on, but as an absolutely
necessary component; no different than Kafka or Storm.

Should we choose to take this approach as a community, I would assume at
least the following by-products.  I am sure there are others, but this is a
start.

   - All PRs should be tested against a Kerberized environment.
   - All docs should be written assuming a Kerberized environment.
   - All development environments should spin-up
   ​pre-
   Kerberized.
   - If a feature does not support Kerberos
   ​,​
   then it should be
   ​ marked​
   experimental and
   ​ not suitable for
   production
   ​ use​
   .


*Benefits*

Metron needs to be secure by default.

Production deployments of Metron should use Kerberos, so we should
developing and testing features against the same target.

Dealing with the pains of Kerberos day-to-day will drive us to think of
ways to make it easier to use and less of a pain for our users.


[image: Inline image 1]

Re: [DISCUSS] Kerberos First

[Nick Allen <ni...@nickallen.org>]

> Do you expect to automate FullDev spin-up with a KDC, realm and
principals, and automate the kinit process for all?

Yes, ideally it would spin-up with everything it needs, including the KDC.
If there are technical limitations than we can make compromises where we
see fit.


> If we automate that, how do we avoid interfering with local policy in
labs that already have a KDC?  In particular, many enterprise environments
use ActiveDirectory as their KDC...

We would only need to fully automate the development environments to make
our lives easier.  Nothing would change about the deployment process to a
cluster for an enterprise SOC. With that in mind, I don't see an issue with
it accidentally reaching out to a random ActiveDirectory host, but maybe I
am misunderstanding your point.

I am most interested right now to see if the community sees enough value in
this approach to outweigh the costs.  Is Kerberos really that important?
But maybe we cannot even begin to understand the costs, unless we dig into
the technical details.

Many thanks



On Mon, May 1, 2017 at 2:31 PM, Matt Foley <ma...@apache.org> wrote:

> Hi Nick,
>
> I support this direction, but would like to better understand the extent
> of the proposal.
>
>
>
> Do you expect to automate FullDev spin-up with a KDC, realm and
> principals, and automate the kinit process for all?
>
> Currently there is a quite well documented manual process, but I don’t
> think it’s reasonable to step thru that for every dev test.
>
>
>
> If we automate that, how do we avoid interfering with local policy in labs
> that already have a KDC?  In particular, many enterprise environments use
> ActiveDirectory as their KDC, while we will presumably use MIT for the
> built-in.  The three choices Ambari provides (existing MIT KDC, existing AD
> KDC, or manage principals and keytabs manually) would be good to apply at
> the higher level, with a fourth choice of “instantiate local MIT KDC”.
>
>
>
> Can Kerberos run satisfactorily in QuickDev?  If so, need for KDC applies,
> and it would still be good to allow QuickDev to come up without Kerberos as
> a simplified environment when that’s desirable.
>
>
>
> Thanks,
>
> --Matt
>
>
>
> From: Nick Allen <ni...@nickallen.org>
> Reply-To: "dev@metron.apache.org" <de...@metron.apache.org>
> Date: Monday, May 1, 2017 at 7:57 AM
> To: dev <de...@metron.incubator.apache.org>
> Subject: [DISCUSS] Kerberos First
>
>
>
>
>
> I hate dealing with Kerberos.  It is a pain to setup, it is a pain to work
> with, it has its own learning curve, but it is absolutely necessary.  Due
> to the sensitive nature of Metron's use case, most of our users should be
> using Kerberos as part of a defense-in-depth strategy to protect sensitive
> data.
>
>
>
> For the upcoming 0.4.0 release, we have all put in a tremendous amount of
> work to make integrating Metron into a Kerberized cluster as simple as
> possible.  I hated every minute of it, but the end result speaks for
> itself.  Really great work, everyone.
>
>
>
> Proposal
>
>
>
> I am proposing that as a community we choose to support Kerberos first.
> We should not treat Kerberos as an optional add-on, but as an absolutely
> necessary component; no different than Kafka or Storm.
>
>
>
> Should we choose to take this approach as a community, I would assume at
> least the following by-products.  I am sure there are others, but this is a
> start.
> All PRs should be tested against a Kerberized environment.
> All docs should be written assuming a Kerberized environment.
> All development environments should spin-up pre-Kerberized.
> If a feature does not support Kerberos,​ then it should be marked​
> experimental and not suitable for production use​.
>
>
> Benefits
>
>
>
> Metron needs to be secure by default.
>
>
>
> Production deployments of Metron should use Kerberos, so we should
> developing and testing features against the same target.
>
>
>
> Dealing with the pains of Kerberos day-to-day will drive us to think of
> ways to make it easier to use and less of a pain for our users.
>
>
>
>
>
>
>
>
>
>
>
>

Re: [DISCUSS] Kerberos First

[Matt Foley <ma...@apache.org>]

Hi Nick,

I support this direction, but would like to better understand the extent of the proposal.

 

Do you expect to automate FullDev spin-up with a KDC, realm and principals, and automate the kinit process for all?

Currently there is a quite well documented manual process, but I don’t think it’s reasonable to step thru that for every dev test.

 

If we automate that, how do we avoid interfering with local policy in labs that already have a KDC?  In particular, many enterprise environments use ActiveDirectory as their KDC, while we will presumably use MIT for the built-in.  The three choices Ambari provides (existing MIT KDC, existing AD KDC, or manage principals and keytabs manually) would be good to apply at the higher level, with a fourth choice of “instantiate local MIT KDC”.

 

Can Kerberos run satisfactorily in QuickDev?  If so, need for KDC applies, and it would still be good to allow QuickDev to come up without Kerberos as a simplified environment when that’s desirable.

 

Thanks,

--Matt

 

From: Nick Allen <ni...@nickallen.org>
Reply-To: "dev@metron.apache.org" <de...@metron.apache.org>
Date: Monday, May 1, 2017 at 7:57 AM
To: dev <de...@metron.incubator.apache.org>
Subject: [DISCUSS] Kerberos First

 

 

I hate dealing with Kerberos.  It is a pain to setup, it is a pain to work with, it has its own learning curve, but it is absolutely necessary.  Due to the sensitive nature of Metron's use case, most of our users should be using Kerberos as part of a defense-in-depth strategy to protect sensitive data.

 

For the upcoming 0.4.0 release, we have all put in a tremendous amount of work to make integrating Metron into a Kerberized cluster as simple as possible.  I hated every minute of it, but the end result speaks for itself.  Really great work, everyone. 

 

Proposal

 

I am proposing that as a community we choose to support Kerberos first.  We should not treat Kerberos as an optional add-on, but as an absolutely necessary component; no different than Kafka or Storm.

 

Should we choose to take this approach as a community, I would assume at least the following by-products.  I am sure there are others, but this is a start.
All PRs should be tested against a Kerberized environment.
All docs should be written assuming a Kerberized environment.
All development environments should spin-up pre-Kerberized.
If a feature does not support Kerberos,​ then it should be marked​ experimental and not suitable for production use​.
 

Benefits

 

Metron needs to be secure by default.

 

Production deployments of Metron should use Kerberos, so we should developing and testing features against the same target.

 

Dealing with the pains of Kerberos day-to-day will drive us to think of ways to make it easier to use and less of a pain for our users.