You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Wahrmann, Helmut" <he...@rsa.com> on 2018/02/12 09:16:52 UTC
Store password for config safely?
Hi,
Do we have a way of storing a password safely, i.e. not in clear text?
When e.g. an Elasticsearch cluster is protected by X-Pack Security, I need to specify a userid / password when connecting.
The userid / password could be specified in the config, but then the password would be available in readable form.
Do we have other sinks or sources, where we are dealing with passwords and were a suitable method exists?
best regards,
Helmut
Re: Store password for config safely?
Posted by Ferenc Szabo <sz...@apache.org>.
Hi Helmut,
here is the documentation part of the PR:
https://github.com/szaboferee/flume/blob/fa13593baa06c9d770a21fc970110e0c9abf2ef8/flume-ng-doc/sphinx/FlumeUserGuide.rst#configuration-filters
it is basically a variable substitution where the value comes from an
external source.
the PR contains 3 implementations to have a few to start with:
- Environment variable ( there was already another method to use env vars,
however, I figured it would be nice to have it in the new format to be
consistent)
- External Process where you can call a command that returns the value
- Hadoop credential store API where you can configure credential providers
documented here:
https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html#Provider_Types
feel free to review it and give feedback if You like.
Best regards,
Ferenc
On Wed, Feb 14, 2018 at 10:45 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:
> That sounds good. Need to have a closer look tough, how it can be used.
>
> Best regards,
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Mittwoch, 14. Februar 2018 00:32
> To: dev@flume.apache.org
> Subject: Re: Store password for config safely?
>
> I think Ferenc has been looking at something related to this, or perhaps
> is trying to get an existing patch merged (FLUME-2442 <
> https://issues.apache.org/jira/browse/FLUME-2442>, PR 197 <
> https://github.com/apache/flume/pull/197>). I haven't been following that
> work closely so I don't know if it's exactly what you're looking for, but
> maybe he can chime in here.
>
> Mike
>
> On Mon, Feb 12, 2018 at 1:16 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi,
> >
> > Do we have a way of storing a password safely, i.e. not in clear text?
> > When e.g. an Elasticsearch cluster is protected by X-Pack Security, I
> > need to specify a userid / password when connecting.
> > The userid / password could be specified in the config, but then the
> > password would be available in readable form.
> >
> > Do we have other sinks or sources, where we are dealing with passwords
> > and were a suitable method exists?
> >
> > best regards,
> >
> > Helmut
> >
>
RE: Store password for config safely?
Posted by "Wahrmann, Helmut" <he...@rsa.com>.
That sounds good. Need to have a closer look tough, how it can be used.
Best regards,
Helmut
-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org]
Sent: Mittwoch, 14. Februar 2018 00:32
To: dev@flume.apache.org
Subject: Re: Store password for config safely?
I think Ferenc has been looking at something related to this, or perhaps is trying to get an existing patch merged (FLUME-2442 <https://issues.apache.org/jira/browse/FLUME-2442>, PR 197 <https://github.com/apache/flume/pull/197>). I haven't been following that work closely so I don't know if it's exactly what you're looking for, but maybe he can chime in here.
Mike
On Mon, Feb 12, 2018 at 1:16 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:
> Hi,
>
> Do we have a way of storing a password safely, i.e. not in clear text?
> When e.g. an Elasticsearch cluster is protected by X-Pack Security, I
> need to specify a userid / password when connecting.
> The userid / password could be specified in the config, but then the
> password would be available in readable form.
>
> Do we have other sinks or sources, where we are dealing with passwords
> and were a suitable method exists?
>
> best regards,
>
> Helmut
>
Re: Store password for config safely?
Posted by Mike Percy <mp...@apache.org>.
I think Ferenc has been looking at something related to this, or perhaps is
trying to get an existing patch merged (FLUME-2442
<https://issues.apache.org/jira/browse/FLUME-2442>, PR 197
<https://github.com/apache/flume/pull/197>). I haven't been following that
work closely so I don't know if it's exactly what you're looking for, but
maybe he can chime in here.
Mike
On Mon, Feb 12, 2018 at 1:16 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:
> Hi,
>
> Do we have a way of storing a password safely, i.e. not in clear text?
> When e.g. an Elasticsearch cluster is protected by X-Pack Security, I need
> to specify a userid / password when connecting.
> The userid / password could be specified in the config, but then the
> password would be available in readable form.
>
> Do we have other sinks or sources, where we are dealing with passwords and
> were a suitable method exists?
>
> best regards,
>
> Helmut
>