[jira] [Created] (HADOOP-15663) ABFS: Simplify configuration

["Thomas Marquardt (JIRA)" <ji...@apache.org>]

Thomas Marquardt created HADOOP-15663:
-----------------------------------------

             Summary: ABFS: Simplify configuration
                 Key: HADOOP-15663
                 URL: https://issues.apache.org/jira/browse/HADOOP-15663
             Project: Hadoop Common
          Issue Type: Sub-task
          Components: fs/azure
            Reporter: Thomas Marquardt


Configuration for WASB and ABFS is too complex.  The current approach is to use four files for test configuration. 

Both WASB and ABFS have basic test configuration which is committed to the repo (azure-test.xml and azure-bfs-test.xml).  Currently these contain the fs.AbstractFileSystem.[scheme].impl configuration, but otherwise are empty except for an include reference to a file containing the endpoint credentials. 

Both WASB and ABFS have endpoint credential configuration files (azure-auth-keys.xml and azure-bfs-auth-keys.xml).  These have been added to .gitignore to prevent them from accidentally being submitted in a patch, which would leak the developers storage account credentials.  These files contain account names, storage account keys, and service endpoints.

There is some overlap of the configuration for WASB and ABFS, where they use the same property name but use different values.  

1) Let's reduce the number of test configuration files to one, if possible.

2) Let's simplify the account name, key, and endpoint configuration for WASB and ABFS if possible, but still support the legacy way of doing it, which is very error prone.

3) Let's improve error handling, so that typos or misconfiguration are not so difficult to troubleshoot.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org