[jira] [Created] (DRILL-4893) Define in Drill-override.conf a common DFS root directory

["Paul Rogers (JIRA)" <ji...@apache.org>]

Paul Rogers created DRILL-4893:
----------------------------------

             Summary: Define in Drill-override.conf a common DFS root directory
                 Key: DRILL-4893
                 URL: https://issues.apache.org/jira/browse/DRILL-4893
             Project: Apache Drill
          Issue Type: Improvement
            Reporter: Paul Rogers
            Priority: Minor


Drill-on-YARN and Dynamic UDFs are two Drill features that write files to the distributed file system (DFS) such as HDFS. These files are configuration-style files (Drill software for DoY, UDF jars for Dynamic UDFs.) Likely other Drill features will also use DFS for storage, perhaps for logs, metadata, etc.

At present, each feature defines its own DFS storage path. An administrator who wants to define a DFS location must know of, and modify, each of these locations.

The proposal here is to define a single Drill root directory property. Then, define all other DFS locations relative to the root. This allows the admin to relocate all Drill files by changing a single property. (Of course, this change should be made at install time, not after Drill has written files, unless the files themselves are moved.)

For example (using made-up property names):

{code}
drill.dfs: “/user/drill”
drill.yarn.dfs: “${drill.dfs}/doy”
drill.exec.udf.directory.base: “${drill.dfs}/udf”
drill.exec.udf.directory.staging: “${drill.exec.udf.directory.base}/staging”
{code}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)