You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Ashish Singh <as...@cloudera.com> on 2016/08/24 19:03:20 UTC
[DISCUSS] KIP-76: Enable getting password from executable rather than
passing as plaintext in config files
Hey Guys,
I’ve just posted KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
<https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Enable+getting+password+from+executable+rather+than+passing+as+plaintext+in+config+files>
.
The proposal is to enable getting passwords from executable. This is an ask
from very security conscious users.
Full details are here:
KIP:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Enable+getting+password+from+executable+rather+than+passing+as+plaintext+in+config+files
JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
POC: https://github.com/apache/kafka/pull/1770
Thanks
--
Regards,
Ashish
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by Gwen Shapira <gw...@confluent.io>.
Thanks Damian. I chatted with Ashish offline and he will check the
possibility an API (similar to Hadoop's credential store), but warned
that since it is a more complex change, he may not get to it
immediately.
Gwen
On Sat, Aug 27, 2016 at 6:59 AM, Damian Guy <da...@gmail.com> wrote:
> I'm in agreement with Gwen. An API would be a better solution. Running
> executables from Kafka is dangerous.
> On Sat, 27 Aug 2016 at 12:02, Ismael Juma <is...@juma.me.uk> wrote:
>
>> Hi Matthias,
>>
>> Improve Kafka Streams Join Semantics is not mentioned on the KIP page and
>> that is probably the source of confusion:
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/
>> Kafka+Improvement+Proposals
>>
>> Ismael
>>
>> On Thu, Aug 25, 2016 at 10:44 PM, Matthias J. Sax <ma...@confluent.io>
>> wrote:
>>
>> > I guess this should be KIP-77 ?
>> >
>> > KIP-76 is "Improve Kafka Streams Join Semantics"
>> >
>> > See http://search-hadoop.com/m/uyzND19SmQJ1yfCQ42/v=plain
>> >
>> > -Matthias
>> >
>> > On 08/25/2016 10:13 PM, Ashish Singh wrote:
>> > > Hey Gwen,
>> > >
>> > > You’re right that if someone can alter the executable then they can do
>> > > things in the context of the thing running the script, like kafka. But
>> if
>> > > you were kafka admin user (or root), you could also do lots of things
>> to
>> > > lots of other different files owned by the user, so it’s not really
>> that
>> > > much different than the current state of things.
>> > >
>> > > You’re right to wonder about the real security gains here. In some
>> sense,
>> > > they aren’t many, because if you know where to look and what to do, you
>> > can
>> > > coax the password out of that executable. What this approach really
>> does
>> > is
>> > > make it *nontrivial* for an attacker to get the password. And people
>> tend
>> > > to flip out when they see passwords sitting in the clear on a disk,
>> > because
>> > > we’ve all been rightly trained that cleartext passwords are bad.
>> > >
>> > > This approach when combined with some strong security practices, like
>> the
>> > > ones mentioned below makes the system reasonably secure. This approach
>> is
>> > > probably the simplest way for folks to strengthen their Kafka security.
>> > > There are other more complicated ways, like Hadoop’s credential store,
>> > > which depends on external systems. If the community feels that this
>> does
>> > > not help, we can definitely move towards more complicated mechanisms.
>> > > However, this has sufficed for our needs so far and others have
>> expressed
>> > > their satisfaction on the JIRA.
>> > >
>> > > - Executable decrypts a file that stores encrypted passwords.
>> > > - The secret to decrypt that file is passed in via environment,
>> which
>> > is
>> > > generally a bit harder to find than files on disk.
>> > > - The perms also protect the executable.
>> > > - The file sits on an ephemeral disk that’s mounted to memory, so
>> > > stealing a physical disk won’t result in getting even the encrypted
>> > > password.
>> > >
>> > > On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <gw...@confluent.io>
>> wrote:
>> > >
>> > > Hi Ashish,
>> > >>
>> > >> I appreciate the need to integrate our authentication with other
>> > >> systems that store passwords.
>> > >> I am not sure that doing so by running a binary is the best solution.
>> > >>
>> > >> First, it does not add security: As you said, a file is just "sitting
>> > >> there" the same way an executable is just "sitting there" - we still
>> > >> rely on file system privileges for security.
>> > >> Second, the idea that Kafka will run arbitrary filesystem executables
>> > >> is pretty terrifying. Reading a string from a file is harmless, but an
>> > >> incorrectly privileged executable can be replaced with "rm -rf /" or
>> > >> anything really. Kafka sometimes runs from privileged account, so this
>> > >> is a serious risk.
>> > >>
>> > >> I looked at the Hadoop credential store you helpfully linked to in the
>> > >> KIP, and it seems like the Hadoop proposal includes a well thought out
>> > >> API to integrate with external systems. Since we took this approach in
>> > >> the past, I'm wondering why not follow the same and use an API to
>> > >> integrate with credential stores rather than arbitrary executables.
>> > >>
>> > >> Gwen
>> > >>
>> > >> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com>
>> > >> wrote:
>> > >>> Hey Guys,
>> > >>>
>> > >>> I’ve just posted KIP-76: Enable getting password from executable
>> rather
>> > >>> than passing as plaintext in config files
>> > >>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En
>> > >> able+getting+password+from+executable+rather+than+passing+
>> > >> as+plaintext+in+config+files>
>> > >>> .
>> > >>>
>> > >>> The proposal is to enable getting passwords from executable. This is
>> an
>> > >> ask
>> > >>> from very security conscious users.
>> > >>>
>> > >>> Full details are here:
>> > >>>
>> > >>> KIP:
>> > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena
>> > >> ble+getting+password+from+executable+rather+than+passing+as+
>> > >> plaintext+in+config+files
>> > >>> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
>> > >>> POC: https://github.com/apache/kafka/pull/1770
>> > >>>
>> > >>> Thanks
>> > >>>
>> > >>> --
>> > >>>
>> > >>> Regards,
>> > >>> Ashish
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Gwen Shapira
>> > >> Product Manager | Confluent
>> > >> 650.450.2760 | @gwenshap
>> > >> Follow us: Twitter | blog
>> > >>
>> > >
>> > >
>> >
>> >
>>
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by Damian Guy <da...@gmail.com>.
I'm in agreement with Gwen. An API would be a better solution. Running
executables from Kafka is dangerous.
On Sat, 27 Aug 2016 at 12:02, Ismael Juma <is...@juma.me.uk> wrote:
> Hi Matthias,
>
> Improve Kafka Streams Join Semantics is not mentioned on the KIP page and
> that is probably the source of confusion:
>
> https://cwiki.apache.org/confluence/display/KAFKA/
> Kafka+Improvement+Proposals
>
> Ismael
>
> On Thu, Aug 25, 2016 at 10:44 PM, Matthias J. Sax <ma...@confluent.io>
> wrote:
>
> > I guess this should be KIP-77 ?
> >
> > KIP-76 is "Improve Kafka Streams Join Semantics"
> >
> > See http://search-hadoop.com/m/uyzND19SmQJ1yfCQ42/v=plain
> >
> > -Matthias
> >
> > On 08/25/2016 10:13 PM, Ashish Singh wrote:
> > > Hey Gwen,
> > >
> > > You’re right that if someone can alter the executable then they can do
> > > things in the context of the thing running the script, like kafka. But
> if
> > > you were kafka admin user (or root), you could also do lots of things
> to
> > > lots of other different files owned by the user, so it’s not really
> that
> > > much different than the current state of things.
> > >
> > > You’re right to wonder about the real security gains here. In some
> sense,
> > > they aren’t many, because if you know where to look and what to do, you
> > can
> > > coax the password out of that executable. What this approach really
> does
> > is
> > > make it *nontrivial* for an attacker to get the password. And people
> tend
> > > to flip out when they see passwords sitting in the clear on a disk,
> > because
> > > we’ve all been rightly trained that cleartext passwords are bad.
> > >
> > > This approach when combined with some strong security practices, like
> the
> > > ones mentioned below makes the system reasonably secure. This approach
> is
> > > probably the simplest way for folks to strengthen their Kafka security.
> > > There are other more complicated ways, like Hadoop’s credential store,
> > > which depends on external systems. If the community feels that this
> does
> > > not help, we can definitely move towards more complicated mechanisms.
> > > However, this has sufficed for our needs so far and others have
> expressed
> > > their satisfaction on the JIRA.
> > >
> > > - Executable decrypts a file that stores encrypted passwords.
> > > - The secret to decrypt that file is passed in via environment,
> which
> > is
> > > generally a bit harder to find than files on disk.
> > > - The perms also protect the executable.
> > > - The file sits on an ephemeral disk that’s mounted to memory, so
> > > stealing a physical disk won’t result in getting even the encrypted
> > > password.
> > >
> > > On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <gw...@confluent.io>
> wrote:
> > >
> > > Hi Ashish,
> > >>
> > >> I appreciate the need to integrate our authentication with other
> > >> systems that store passwords.
> > >> I am not sure that doing so by running a binary is the best solution.
> > >>
> > >> First, it does not add security: As you said, a file is just "sitting
> > >> there" the same way an executable is just "sitting there" - we still
> > >> rely on file system privileges for security.
> > >> Second, the idea that Kafka will run arbitrary filesystem executables
> > >> is pretty terrifying. Reading a string from a file is harmless, but an
> > >> incorrectly privileged executable can be replaced with "rm -rf /" or
> > >> anything really. Kafka sometimes runs from privileged account, so this
> > >> is a serious risk.
> > >>
> > >> I looked at the Hadoop credential store you helpfully linked to in the
> > >> KIP, and it seems like the Hadoop proposal includes a well thought out
> > >> API to integrate with external systems. Since we took this approach in
> > >> the past, I'm wondering why not follow the same and use an API to
> > >> integrate with credential stores rather than arbitrary executables.
> > >>
> > >> Gwen
> > >>
> > >> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com>
> > >> wrote:
> > >>> Hey Guys,
> > >>>
> > >>> I’ve just posted KIP-76: Enable getting password from executable
> rather
> > >>> than passing as plaintext in config files
> > >>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En
> > >> able+getting+password+from+executable+rather+than+passing+
> > >> as+plaintext+in+config+files>
> > >>> .
> > >>>
> > >>> The proposal is to enable getting passwords from executable. This is
> an
> > >> ask
> > >>> from very security conscious users.
> > >>>
> > >>> Full details are here:
> > >>>
> > >>> KIP:
> > >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena
> > >> ble+getting+password+from+executable+rather+than+passing+as+
> > >> plaintext+in+config+files
> > >>> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
> > >>> POC: https://github.com/apache/kafka/pull/1770
> > >>>
> > >>> Thanks
> > >>>
> > >>> --
> > >>>
> > >>> Regards,
> > >>> Ashish
> > >>
> > >>
> > >>
> > >> --
> > >> Gwen Shapira
> > >> Product Manager | Confluent
> > >> 650.450.2760 | @gwenshap
> > >> Follow us: Twitter | blog
> > >>
> > >
> > >
> >
> >
>
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by "Matthias J. Sax" <ma...@confluent.io>.
Thanks Ismael,
I created the child page but forgot the add it to the table... I
re-numbered "Join Semantics" to KIP-77.
Sorry for the confusion.
-Matthias
On 08/27/2016 01:01 PM, Ismael Juma wrote:
> Hi Matthias,
>
> Improve Kafka Streams Join Semantics is not mentioned on the KIP page and
> that is probably the source of confusion:
>
> https://cwiki.apache.org/confluence/display/KAFKA/
> Kafka+Improvement+Proposals
>
> Ismael
>
> On Thu, Aug 25, 2016 at 10:44 PM, Matthias J. Sax <ma...@confluent.io>
> wrote:
>
>> I guess this should be KIP-77 ?
>>
>> KIP-76 is "Improve Kafka Streams Join Semantics"
>>
>> See http://search-hadoop.com/m/uyzND19SmQJ1yfCQ42/v=plain
>>
>> -Matthias
>>
>> On 08/25/2016 10:13 PM, Ashish Singh wrote:
>>> Hey Gwen,
>>>
>>> You’re right that if someone can alter the executable then they can do
>>> things in the context of the thing running the script, like kafka. But if
>>> you were kafka admin user (or root), you could also do lots of things to
>>> lots of other different files owned by the user, so it’s not really that
>>> much different than the current state of things.
>>>
>>> You’re right to wonder about the real security gains here. In some sense,
>>> they aren’t many, because if you know where to look and what to do, you
>> can
>>> coax the password out of that executable. What this approach really does
>> is
>>> make it *nontrivial* for an attacker to get the password. And people tend
>>> to flip out when they see passwords sitting in the clear on a disk,
>> because
>>> we’ve all been rightly trained that cleartext passwords are bad.
>>>
>>> This approach when combined with some strong security practices, like the
>>> ones mentioned below makes the system reasonably secure. This approach is
>>> probably the simplest way for folks to strengthen their Kafka security.
>>> There are other more complicated ways, like Hadoop’s credential store,
>>> which depends on external systems. If the community feels that this does
>>> not help, we can definitely move towards more complicated mechanisms.
>>> However, this has sufficed for our needs so far and others have expressed
>>> their satisfaction on the JIRA.
>>>
>>> - Executable decrypts a file that stores encrypted passwords.
>>> - The secret to decrypt that file is passed in via environment, which
>> is
>>> generally a bit harder to find than files on disk.
>>> - The perms also protect the executable.
>>> - The file sits on an ephemeral disk that’s mounted to memory, so
>>> stealing a physical disk won’t result in getting even the encrypted
>>> password.
>>>
>>> On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <gw...@confluent.io> wrote:
>>>
>>> Hi Ashish,
>>>>
>>>> I appreciate the need to integrate our authentication with other
>>>> systems that store passwords.
>>>> I am not sure that doing so by running a binary is the best solution.
>>>>
>>>> First, it does not add security: As you said, a file is just "sitting
>>>> there" the same way an executable is just "sitting there" - we still
>>>> rely on file system privileges for security.
>>>> Second, the idea that Kafka will run arbitrary filesystem executables
>>>> is pretty terrifying. Reading a string from a file is harmless, but an
>>>> incorrectly privileged executable can be replaced with "rm -rf /" or
>>>> anything really. Kafka sometimes runs from privileged account, so this
>>>> is a serious risk.
>>>>
>>>> I looked at the Hadoop credential store you helpfully linked to in the
>>>> KIP, and it seems like the Hadoop proposal includes a well thought out
>>>> API to integrate with external systems. Since we took this approach in
>>>> the past, I'm wondering why not follow the same and use an API to
>>>> integrate with credential stores rather than arbitrary executables.
>>>>
>>>> Gwen
>>>>
>>>> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com>
>>>> wrote:
>>>>> Hey Guys,
>>>>>
>>>>> I’ve just posted KIP-76: Enable getting password from executable rather
>>>>> than passing as plaintext in config files
>>>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En
>>>> able+getting+password+from+executable+rather+than+passing+
>>>> as+plaintext+in+config+files>
>>>>> .
>>>>>
>>>>> The proposal is to enable getting passwords from executable. This is an
>>>> ask
>>>>> from very security conscious users.
>>>>>
>>>>> Full details are here:
>>>>>
>>>>> KIP:
>>>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena
>>>> ble+getting+password+from+executable+rather+than+passing+as+
>>>> plaintext+in+config+files
>>>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
>>>>> POC: https://github.com/apache/kafka/pull/1770
>>>>>
>>>>> Thanks
>>>>>
>>>>> --
>>>>>
>>>>> Regards,
>>>>> Ashish
>>>>
>>>>
>>>>
>>>> --
>>>> Gwen Shapira
>>>> Product Manager | Confluent
>>>> 650.450.2760 | @gwenshap
>>>> Follow us: Twitter | blog
>>>>
>>>
>>>
>>
>>
>
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by Ismael Juma <is...@juma.me.uk>.
Hi Matthias,
Improve Kafka Streams Join Semantics is not mentioned on the KIP page and
that is probably the source of confusion:
https://cwiki.apache.org/confluence/display/KAFKA/
Kafka+Improvement+Proposals
Ismael
On Thu, Aug 25, 2016 at 10:44 PM, Matthias J. Sax <ma...@confluent.io>
wrote:
> I guess this should be KIP-77 ?
>
> KIP-76 is "Improve Kafka Streams Join Semantics"
>
> See http://search-hadoop.com/m/uyzND19SmQJ1yfCQ42/v=plain
>
> -Matthias
>
> On 08/25/2016 10:13 PM, Ashish Singh wrote:
> > Hey Gwen,
> >
> > You’re right that if someone can alter the executable then they can do
> > things in the context of the thing running the script, like kafka. But if
> > you were kafka admin user (or root), you could also do lots of things to
> > lots of other different files owned by the user, so it’s not really that
> > much different than the current state of things.
> >
> > You’re right to wonder about the real security gains here. In some sense,
> > they aren’t many, because if you know where to look and what to do, you
> can
> > coax the password out of that executable. What this approach really does
> is
> > make it *nontrivial* for an attacker to get the password. And people tend
> > to flip out when they see passwords sitting in the clear on a disk,
> because
> > we’ve all been rightly trained that cleartext passwords are bad.
> >
> > This approach when combined with some strong security practices, like the
> > ones mentioned below makes the system reasonably secure. This approach is
> > probably the simplest way for folks to strengthen their Kafka security.
> > There are other more complicated ways, like Hadoop’s credential store,
> > which depends on external systems. If the community feels that this does
> > not help, we can definitely move towards more complicated mechanisms.
> > However, this has sufficed for our needs so far and others have expressed
> > their satisfaction on the JIRA.
> >
> > - Executable decrypts a file that stores encrypted passwords.
> > - The secret to decrypt that file is passed in via environment, which
> is
> > generally a bit harder to find than files on disk.
> > - The perms also protect the executable.
> > - The file sits on an ephemeral disk that’s mounted to memory, so
> > stealing a physical disk won’t result in getting even the encrypted
> > password.
> >
> > On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <gw...@confluent.io> wrote:
> >
> > Hi Ashish,
> >>
> >> I appreciate the need to integrate our authentication with other
> >> systems that store passwords.
> >> I am not sure that doing so by running a binary is the best solution.
> >>
> >> First, it does not add security: As you said, a file is just "sitting
> >> there" the same way an executable is just "sitting there" - we still
> >> rely on file system privileges for security.
> >> Second, the idea that Kafka will run arbitrary filesystem executables
> >> is pretty terrifying. Reading a string from a file is harmless, but an
> >> incorrectly privileged executable can be replaced with "rm -rf /" or
> >> anything really. Kafka sometimes runs from privileged account, so this
> >> is a serious risk.
> >>
> >> I looked at the Hadoop credential store you helpfully linked to in the
> >> KIP, and it seems like the Hadoop proposal includes a well thought out
> >> API to integrate with external systems. Since we took this approach in
> >> the past, I'm wondering why not follow the same and use an API to
> >> integrate with credential stores rather than arbitrary executables.
> >>
> >> Gwen
> >>
> >> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com>
> >> wrote:
> >>> Hey Guys,
> >>>
> >>> I’ve just posted KIP-76: Enable getting password from executable rather
> >>> than passing as plaintext in config files
> >>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En
> >> able+getting+password+from+executable+rather+than+passing+
> >> as+plaintext+in+config+files>
> >>> .
> >>>
> >>> The proposal is to enable getting passwords from executable. This is an
> >> ask
> >>> from very security conscious users.
> >>>
> >>> Full details are here:
> >>>
> >>> KIP:
> >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena
> >> ble+getting+password+from+executable+rather+than+passing+as+
> >> plaintext+in+config+files
> >>> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
> >>> POC: https://github.com/apache/kafka/pull/1770
> >>>
> >>> Thanks
> >>>
> >>> --
> >>>
> >>> Regards,
> >>> Ashish
> >>
> >>
> >>
> >> --
> >> Gwen Shapira
> >> Product Manager | Confluent
> >> 650.450.2760 | @gwenshap
> >> Follow us: Twitter | blog
> >>
> >
> >
>
>
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by "Matthias J. Sax" <ma...@confluent.io>.
I guess this should be KIP-77 ?
KIP-76 is "Improve Kafka Streams Join Semantics"
See http://search-hadoop.com/m/uyzND19SmQJ1yfCQ42/v=plain
-Matthias
On 08/25/2016 10:13 PM, Ashish Singh wrote:
> Hey Gwen,
>
> You’re right that if someone can alter the executable then they can do
> things in the context of the thing running the script, like kafka. But if
> you were kafka admin user (or root), you could also do lots of things to
> lots of other different files owned by the user, so it’s not really that
> much different than the current state of things.
>
> You’re right to wonder about the real security gains here. In some sense,
> they aren’t many, because if you know where to look and what to do, you can
> coax the password out of that executable. What this approach really does is
> make it *nontrivial* for an attacker to get the password. And people tend
> to flip out when they see passwords sitting in the clear on a disk, because
> we’ve all been rightly trained that cleartext passwords are bad.
>
> This approach when combined with some strong security practices, like the
> ones mentioned below makes the system reasonably secure. This approach is
> probably the simplest way for folks to strengthen their Kafka security.
> There are other more complicated ways, like Hadoop’s credential store,
> which depends on external systems. If the community feels that this does
> not help, we can definitely move towards more complicated mechanisms.
> However, this has sufficed for our needs so far and others have expressed
> their satisfaction on the JIRA.
>
> - Executable decrypts a file that stores encrypted passwords.
> - The secret to decrypt that file is passed in via environment, which is
> generally a bit harder to find than files on disk.
> - The perms also protect the executable.
> - The file sits on an ephemeral disk that’s mounted to memory, so
> stealing a physical disk won’t result in getting even the encrypted
> password.
>
> On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <gw...@confluent.io> wrote:
>
> Hi Ashish,
>>
>> I appreciate the need to integrate our authentication with other
>> systems that store passwords.
>> I am not sure that doing so by running a binary is the best solution.
>>
>> First, it does not add security: As you said, a file is just "sitting
>> there" the same way an executable is just "sitting there" - we still
>> rely on file system privileges for security.
>> Second, the idea that Kafka will run arbitrary filesystem executables
>> is pretty terrifying. Reading a string from a file is harmless, but an
>> incorrectly privileged executable can be replaced with "rm -rf /" or
>> anything really. Kafka sometimes runs from privileged account, so this
>> is a serious risk.
>>
>> I looked at the Hadoop credential store you helpfully linked to in the
>> KIP, and it seems like the Hadoop proposal includes a well thought out
>> API to integrate with external systems. Since we took this approach in
>> the past, I'm wondering why not follow the same and use an API to
>> integrate with credential stores rather than arbitrary executables.
>>
>> Gwen
>>
>> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com>
>> wrote:
>>> Hey Guys,
>>>
>>> I’ve just posted KIP-76: Enable getting password from executable rather
>>> than passing as plaintext in config files
>>> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En
>> able+getting+password+from+executable+rather+than+passing+
>> as+plaintext+in+config+files>
>>> .
>>>
>>> The proposal is to enable getting passwords from executable. This is an
>> ask
>>> from very security conscious users.
>>>
>>> Full details are here:
>>>
>>> KIP:
>>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena
>> ble+getting+password+from+executable+rather+than+passing+as+
>> plaintext+in+config+files
>>> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
>>> POC: https://github.com/apache/kafka/pull/1770
>>>
>>> Thanks
>>>
>>> --
>>>
>>> Regards,
>>> Ashish
>>
>>
>>
>> --
>> Gwen Shapira
>> Product Manager | Confluent
>> 650.450.2760 | @gwenshap
>> Follow us: Twitter | blog
>>
>
>
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by Ashish Singh <as...@cloudera.com>.
Hey Gwen,
You’re right that if someone can alter the executable then they can do
things in the context of the thing running the script, like kafka. But if
you were kafka admin user (or root), you could also do lots of things to
lots of other different files owned by the user, so it’s not really that
much different than the current state of things.
You’re right to wonder about the real security gains here. In some sense,
they aren’t many, because if you know where to look and what to do, you can
coax the password out of that executable. What this approach really does is
make it *nontrivial* for an attacker to get the password. And people tend
to flip out when they see passwords sitting in the clear on a disk, because
we’ve all been rightly trained that cleartext passwords are bad.
This approach when combined with some strong security practices, like the
ones mentioned below makes the system reasonably secure. This approach is
probably the simplest way for folks to strengthen their Kafka security.
There are other more complicated ways, like Hadoop’s credential store,
which depends on external systems. If the community feels that this does
not help, we can definitely move towards more complicated mechanisms.
However, this has sufficed for our needs so far and others have expressed
their satisfaction on the JIRA.
- Executable decrypts a file that stores encrypted passwords.
- The secret to decrypt that file is passed in via environment, which is
generally a bit harder to find than files on disk.
- The perms also protect the executable.
- The file sits on an ephemeral disk that’s mounted to memory, so
stealing a physical disk won’t result in getting even the encrypted
password.
On Thu, Aug 25, 2016 at 9:07 AM, Gwen Shapira <gw...@confluent.io> wrote:
Hi Ashish,
>
> I appreciate the need to integrate our authentication with other
> systems that store passwords.
> I am not sure that doing so by running a binary is the best solution.
>
> First, it does not add security: As you said, a file is just "sitting
> there" the same way an executable is just "sitting there" - we still
> rely on file system privileges for security.
> Second, the idea that Kafka will run arbitrary filesystem executables
> is pretty terrifying. Reading a string from a file is harmless, but an
> incorrectly privileged executable can be replaced with "rm -rf /" or
> anything really. Kafka sometimes runs from privileged account, so this
> is a serious risk.
>
> I looked at the Hadoop credential store you helpfully linked to in the
> KIP, and it seems like the Hadoop proposal includes a well thought out
> API to integrate with external systems. Since we took this approach in
> the past, I'm wondering why not follow the same and use an API to
> integrate with credential stores rather than arbitrary executables.
>
> Gwen
>
> On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com>
> wrote:
> > Hey Guys,
> >
> > I’ve just posted KIP-76: Enable getting password from executable rather
> > than passing as plaintext in config files
> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+En
> able+getting+password+from+executable+rather+than+passing+
> as+plaintext+in+config+files>
> > .
> >
> > The proposal is to enable getting passwords from executable. This is an
> ask
> > from very security conscious users.
> >
> > Full details are here:
> >
> > KIP:
> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Ena
> ble+getting+password+from+executable+rather+than+passing+as+
> plaintext+in+config+files
> > JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
> > POC: https://github.com/apache/kafka/pull/1770
> >
> > Thanks
> >
> > --
> >
> > Regards,
> > Ashish
>
>
>
> --
> Gwen Shapira
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter | blog
>
--
Regards,
Ashish
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by Gwen Shapira <gw...@confluent.io>.
Hi Ashish,
I appreciate the need to integrate our authentication with other
systems that store passwords.
I am not sure that doing so by running a binary is the best solution.
First, it does not add security: As you said, a file is just "sitting
there" the same way an executable is just "sitting there" - we still
rely on file system privileges for security.
Second, the idea that Kafka will run arbitrary filesystem executables
is pretty terrifying. Reading a string from a file is harmless, but an
incorrectly privileged executable can be replaced with "rm -rf /" or
anything really. Kafka sometimes runs from privileged account, so this
is a serious risk.
I looked at the Hadoop credential store you helpfully linked to in the
KIP, and it seems like the Hadoop proposal includes a well thought out
API to integrate with external systems. Since we took this approach in
the past, I'm wondering why not follow the same and use an API to
integrate with credential stores rather than arbitrary executables.
Gwen
On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com> wrote:
> Hey Guys,
>
> I’ve just posted KIP-76: Enable getting password from executable rather
> than passing as plaintext in config files
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Enable+getting+password+from+executable+rather+than+passing+as+plaintext+in+config+files>
> .
>
> The proposal is to enable getting passwords from executable. This is an ask
> from very security conscious users.
>
> Full details are here:
>
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-76+Enable+getting+password+from+executable+rather+than+passing+as+plaintext+in+config+files
> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
> POC: https://github.com/apache/kafka/pull/1770
>
> Thanks
>
> --
>
> Regards,
> Ashish
--
Gwen Shapira
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter | blog
Re: [DISCUSS] KIP-76: Enable getting password from executable rather
than passing as plaintext in config files
Posted by BigData dev <bi...@gmail.com>.
+1 (non-binding)
Thanks,
Bharat
On Wed, Aug 24, 2016 at 12:03 PM, Ashish Singh <as...@cloudera.com> wrote:
> Hey Guys,
>
> I’ve just posted KIP-76: Enable getting password from executable rather
> than passing as plaintext in config files
> <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 76+Enable+getting+password+from+executable+rather+than+
> passing+as+plaintext+in+config+files>
> .
>
> The proposal is to enable getting passwords from executable. This is an ask
> from very security conscious users.
>
> Full details are here:
>
> KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 76+Enable+getting+password+from+executable+rather+than+
> passing+as+plaintext+in+config+files
> JIRA: https://issues.apache.org/jira/browse/KAFKA-2629
> POC: https://github.com/apache/kafka/pull/1770
>
> Thanks
>
> --
>
> Regards,
> Ashish
>