[jira] [Commented] (FLUME-2442) Need an alternative to providing clear text passwords in flume config

["Roshan Naik (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097673#comment-14097673 ] 

Roshan Naik commented on FLUME-2442:
------------------------------------

There is an initiative in Hadoop to create an API to address clear text passwords that some projects seem to be adopting.

https://issues.apache.org/jira/browse/HADOOP-10607
https://issues.apache.org/jira/browse/HADOOP-10904

To summarize my understanding of it:
- Its an api for accessing passwords which are stored elsewhere
- It will have several "providers" which will determine where the password actually gets stored
- A simple provider will basically put the password in a local/hdfs file .. the api has an option to use a default "none" password to decrypt the local/hdfs file. this  functionality is very similar to what we do currently when we put the keys in a local keystore. 
- A more sophisticated "provider" will user kerberos keytab to acquire the password from a remote "credential store service" 
- The api interface will support a few different providers and each provider may require some unique configuration settings.. but the API interface would remain the same.

*Benefits of going with this API for flume:*
- Give flume users the  option to use more sophisticated secure providers as they become available.
- with kerberos based provider we completely move away from clear text passwords

*Concerns that I have:*
- Initially only the simple local store provider will be available. So it will not offer any new level of security than the current local keystore based approach we have in say file channel and jms source. A minor benefit is that it has the option of using a default "none" password which means no explicit cleartext password is required in config file (although it has hard code in some java source file)
- Once the kerberos based provider becomes available, level of security does not appear to be much stronger than the local keystore approach. Since, in both case the security is governed by the file permission of keytab or local keystore.  so once the hacker gets access to the keytab or keystore file, the security has been compromised.



> Need an alternative to providing clear text passwords in flume config
> ---------------------------------------------------------------------
>
>                 Key: FLUME-2442
>                 URL: https://issues.apache.org/jira/browse/FLUME-2442
>             Project: Flume
>          Issue Type: Bug
>          Components: Sinks+Sources
>    Affects Versions: v1.5.0.1
>            Reporter: Roshan Naik
>            Assignee: Roshan Naik
>              Labels: Security
>
> For some sources and sinks, currently, passwords to keystores/other are specified in clear text in the flume config file.   Since flume config files are often easily accessible to a broader audience (like in source control for instance), the visibility of these passwords can be too much and risky for institutions where security is too critical (like banks) 
> To help address this visibility issue it would be desirable to do the following two things :
> 1) Store the password in a separate file and provide the path of that password file in the flume config. this will enable the flume config to be shared with a wider audience and reduce risk. the password file will need to be very tightly guarded. Some components like file channel & JMS source already do this. 
> 2) As an additional measure, obfuscate the password in the external password file. A simple command line tool can be used to generate the obfuscated password file. Flume source/sink configuration will read the password file and de-obfuscate the password before using it to access the keystore. This obfuscation step IMO is nice but unclear to me if it is essential.
> The following Sources and Sinks appear to use inline cleartext passwords:
> - Avro Source
> - Avro sink
> - HTTP(S) source 
> JDBC channel also uses inline passwords but i am not aware of anybody who uses JDBC channel. So it may not be an issue.



--
This message was sent by Atlassian JIRA
(v6.2#6252)