You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@slider.apache.org by Billie Rinaldi <bi...@gmail.com> on 2014/07/23 22:03:24 UTC
password handling
Accumulo requires a couple of sensitive pieces of information to be
provided when it is starting up (including the Accumulo root user's
password). Those are currently provided in the appConfig.json file.
Hadoop common is adding a CredentialProvider (HADOOP-10607) that allows you
to enter passwords into a Java KeyStore (the storage mechanism is
pluggable) and to retrieve them. This is in branch-2 and trunk, though I'm
not sure which branch-2 release it will make it into. I'd like to add a
capability into Slider to prompt the user for needed passwords and store
them using CredentialProvider so they can be retrieved by an app later.
We could add a new "credentials" section to the appConfig.json, with
entries that map CredentialProvider URLs to a list of credentials to store
in that CP.
+ "credentials": {
+ "jceks://hdfs/slider/accumulo.jceks": ["root", "instance.secret"]
+ },
One thing I'm not sure about is where to do the CredentialProvider
creation. The best place I have thought of so far is in
SliderClient.actionCreate, between building and starting a cluster.
Re: password handling
Posted by Steve Loughran <st...@hortonworks.com>.
ignore that, just seen SLIDER-263 & -264; will link the hadoop JIRA to them
On 30 July 2014 11:22, Steve Loughran <st...@hortonworks.com> wrote:
> could we just jump to using the 2.6+ credential provider? HADOOP-10607
> lacks any concrete examples so far?
>
> at the very least, we can create a JIRA
>
>
> On 24 July 2014 14:22, Billie Rinaldi <bi...@gmail.com> wrote:
>
>> To be clear, ["root", "instance.secret"] is a list of usernames / keys by
>> which to store and retrieve password / secret info. Accumulo's
>> instance.secret is the name of another shared secret it needs. The
>> default
>> implementation of CredentialProvider stores the credentials by key in a
>> no-password Java KeyStore, with file permissions set to 700. It's
>> somewhat
>> better than storing them in plaintext in a similarly locked-down file, and
>> opens the door to better implementations. In my current prototype the
>> credential store is created during actionCreate by prompting the user for
>> passwords. If the keys already exist in the specified store it skips over
>> that step. Then the app itself has to know to read those passwords from
>> the store instead of from a configuration file at start up.
>>
>>
>> On Thu, Jul 24, 2014 at 3:24 AM, Jon Maron <jm...@hortonworks.com>
>> wrote:
>>
>> >
>> > On Jul 23, 2014, at 4:03 PM, Billie Rinaldi <bi...@gmail.com>
>> > wrote:
>> >
>> > > Accumulo requires a couple of sensitive pieces of information to be
>> > > provided when it is starting up (including the Accumulo root user's
>> > > password). Those are currently provided in the appConfig.json file.
>> > > Hadoop common is adding a CredentialProvider (HADOOP-10607) that
>> allows
>> > you
>> > > to enter passwords into a Java KeyStore (the storage mechanism is
>> > > pluggable) and to retrieve them. This is in branch-2 and trunk,
>> though
>> > I'm
>> > > not sure which branch-2 release it will make it into. I'd like to
>> add a
>> > > capability into Slider to prompt the user for needed passwords and
>> store
>> > > them using CredentialProvider so they can be retrieved by an app
>> later.
>> > >
>> > > We could add a new "credentials" section to the appConfig.json, with
>> > > entries that map CredentialProvider URLs to a list of credentials to
>> > store
>> > > in that CP.
>> > > + "credentials": {
>> > > + "jceks://hdfs/slider/accumulo.jceks": ["root", "instance.secret"]
>> > > + },
>> >
>> > In that instance you once again have the credential/password or secret
>> in
>> > the clear or available in an app resource (I suppose you could encrypt
>> it).
>> > Does the CredentialProvider have a capability to return credentials
>> based
>> > on a key? It would seem that in that instance the expectation would be
>> > that the credential store would be pre-configured as an administrative
>> > pre-deployment step, allowing for the retrieval of the credential during
>> > app start up (e.g. based on service selection Ambari prompts use with
>> > config panel for credential store creation or for the URI of an existing
>> > store)? I imagine in that instance the configuration would look like
>> > (assuming the need for multiple credentials):
>> >
>> > + "credentials": {
>> > + "jceks://hdfs/slider/accumulo.jceks": ["root", "key-to-credential"]
>> > + },
>> >
>> >
>> > >
>> > > One thing I'm not sure about is where to do the CredentialProvider
>> > > creation. The best place I have thought of so far is in
>> > > SliderClient.actionCreate, between building and starting a cluster.
>> >
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>> >
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: password handling
Posted by Steve Loughran <st...@hortonworks.com>.
could we just jump to using the 2.6+ credential provider? HADOOP-10607
lacks any concrete examples so far?
at the very least, we can create a JIRA
On 24 July 2014 14:22, Billie Rinaldi <bi...@gmail.com> wrote:
> To be clear, ["root", "instance.secret"] is a list of usernames / keys by
> which to store and retrieve password / secret info. Accumulo's
> instance.secret is the name of another shared secret it needs. The default
> implementation of CredentialProvider stores the credentials by key in a
> no-password Java KeyStore, with file permissions set to 700. It's somewhat
> better than storing them in plaintext in a similarly locked-down file, and
> opens the door to better implementations. In my current prototype the
> credential store is created during actionCreate by prompting the user for
> passwords. If the keys already exist in the specified store it skips over
> that step. Then the app itself has to know to read those passwords from
> the store instead of from a configuration file at start up.
>
>
> On Thu, Jul 24, 2014 at 3:24 AM, Jon Maron <jm...@hortonworks.com> wrote:
>
> >
> > On Jul 23, 2014, at 4:03 PM, Billie Rinaldi <bi...@gmail.com>
> > wrote:
> >
> > > Accumulo requires a couple of sensitive pieces of information to be
> > > provided when it is starting up (including the Accumulo root user's
> > > password). Those are currently provided in the appConfig.json file.
> > > Hadoop common is adding a CredentialProvider (HADOOP-10607) that allows
> > you
> > > to enter passwords into a Java KeyStore (the storage mechanism is
> > > pluggable) and to retrieve them. This is in branch-2 and trunk, though
> > I'm
> > > not sure which branch-2 release it will make it into. I'd like to add
> a
> > > capability into Slider to prompt the user for needed passwords and
> store
> > > them using CredentialProvider so they can be retrieved by an app later.
> > >
> > > We could add a new "credentials" section to the appConfig.json, with
> > > entries that map CredentialProvider URLs to a list of credentials to
> > store
> > > in that CP.
> > > + "credentials": {
> > > + "jceks://hdfs/slider/accumulo.jceks": ["root", "instance.secret"]
> > > + },
> >
> > In that instance you once again have the credential/password or secret in
> > the clear or available in an app resource (I suppose you could encrypt
> it).
> > Does the CredentialProvider have a capability to return credentials
> based
> > on a key? It would seem that in that instance the expectation would be
> > that the credential store would be pre-configured as an administrative
> > pre-deployment step, allowing for the retrieval of the credential during
> > app start up (e.g. based on service selection Ambari prompts use with
> > config panel for credential store creation or for the URI of an existing
> > store)? I imagine in that instance the configuration would look like
> > (assuming the need for multiple credentials):
> >
> > + "credentials": {
> > + "jceks://hdfs/slider/accumulo.jceks": ["root", "key-to-credential"]
> > + },
> >
> >
> > >
> > > One thing I'm not sure about is where to do the CredentialProvider
> > > creation. The best place I have thought of so far is in
> > > SliderClient.actionCreate, between building and starting a cluster.
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: password handling
Posted by Billie Rinaldi <bi...@gmail.com>.
To be clear, ["root", "instance.secret"] is a list of usernames / keys by
which to store and retrieve password / secret info. Accumulo's
instance.secret is the name of another shared secret it needs. The default
implementation of CredentialProvider stores the credentials by key in a
no-password Java KeyStore, with file permissions set to 700. It's somewhat
better than storing them in plaintext in a similarly locked-down file, and
opens the door to better implementations. In my current prototype the
credential store is created during actionCreate by prompting the user for
passwords. If the keys already exist in the specified store it skips over
that step. Then the app itself has to know to read those passwords from
the store instead of from a configuration file at start up.
On Thu, Jul 24, 2014 at 3:24 AM, Jon Maron <jm...@hortonworks.com> wrote:
>
> On Jul 23, 2014, at 4:03 PM, Billie Rinaldi <bi...@gmail.com>
> wrote:
>
> > Accumulo requires a couple of sensitive pieces of information to be
> > provided when it is starting up (including the Accumulo root user's
> > password). Those are currently provided in the appConfig.json file.
> > Hadoop common is adding a CredentialProvider (HADOOP-10607) that allows
> you
> > to enter passwords into a Java KeyStore (the storage mechanism is
> > pluggable) and to retrieve them. This is in branch-2 and trunk, though
> I'm
> > not sure which branch-2 release it will make it into. I'd like to add a
> > capability into Slider to prompt the user for needed passwords and store
> > them using CredentialProvider so they can be retrieved by an app later.
> >
> > We could add a new "credentials" section to the appConfig.json, with
> > entries that map CredentialProvider URLs to a list of credentials to
> store
> > in that CP.
> > + "credentials": {
> > + "jceks://hdfs/slider/accumulo.jceks": ["root", "instance.secret"]
> > + },
>
> In that instance you once again have the credential/password or secret in
> the clear or available in an app resource (I suppose you could encrypt it).
> Does the CredentialProvider have a capability to return credentials based
> on a key? It would seem that in that instance the expectation would be
> that the credential store would be pre-configured as an administrative
> pre-deployment step, allowing for the retrieval of the credential during
> app start up (e.g. based on service selection Ambari prompts use with
> config panel for credential store creation or for the URI of an existing
> store)? I imagine in that instance the configuration would look like
> (assuming the need for multiple credentials):
>
> + "credentials": {
> + "jceks://hdfs/slider/accumulo.jceks": ["root", "key-to-credential"]
> + },
>
>
> >
> > One thing I'm not sure about is where to do the CredentialProvider
> > creation. The best place I have thought of so far is in
> > SliderClient.actionCreate, between building and starting a cluster.
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
Re: password handling
Posted by Jon Maron <jm...@hortonworks.com>.
On Jul 23, 2014, at 4:03 PM, Billie Rinaldi <bi...@gmail.com> wrote:
> Accumulo requires a couple of sensitive pieces of information to be
> provided when it is starting up (including the Accumulo root user's
> password). Those are currently provided in the appConfig.json file.
> Hadoop common is adding a CredentialProvider (HADOOP-10607) that allows you
> to enter passwords into a Java KeyStore (the storage mechanism is
> pluggable) and to retrieve them. This is in branch-2 and trunk, though I'm
> not sure which branch-2 release it will make it into. I'd like to add a
> capability into Slider to prompt the user for needed passwords and store
> them using CredentialProvider so they can be retrieved by an app later.
>
> We could add a new "credentials" section to the appConfig.json, with
> entries that map CredentialProvider URLs to a list of credentials to store
> in that CP.
> + "credentials": {
> + "jceks://hdfs/slider/accumulo.jceks": ["root", "instance.secret"]
> + },
In that instance you once again have the credential/password or secret in the clear or available in an app resource (I suppose you could encrypt it). Does the CredentialProvider have a capability to return credentials based on a key? It would seem that in that instance the expectation would be that the credential store would be pre-configured as an administrative pre-deployment step, allowing for the retrieval of the credential during app start up (e.g. based on service selection Ambari prompts use with config panel for credential store creation or for the URI of an existing store)? I imagine in that instance the configuration would look like (assuming the need for multiple credentials):
+ "credentials": {
+ "jceks://hdfs/slider/accumulo.jceks": ["root", "key-to-credential"]
+ },
>
> One thing I'm not sure about is where to do the CredentialProvider
> creation. The best place I have thought of so far is in
> SliderClient.actionCreate, between building and starting a cluster.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.