[jira] [Updated] (HADOOP-12490) Add default resources to Configuration which are not inside the class path

["Matthias Jacob (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-12490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matthias Jacob updated HADOOP-12490:
------------------------------------
    Attachment: HADOOP-12490.001.patch

Added a patch for this, please review.

> Add default resources to Configuration which are not inside the class path
> --------------------------------------------------------------------------
>
>                 Key: HADOOP-12490
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12490
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>            Reporter: Matthias Jacob
>         Attachments: HADOOP-12490.001.patch
>
>
> *Problem*
> It is currently not possible to add default resources to {{org.apache.hadoop.conf.Configuration}} that are not included in the class path.
> We need to add resources to {{org.apache.hadoop.conf.Configuration}} that are not included in the class path. Currently, the only way to do this is to use {{addResource(URL)}} or {{addResource(Path)}} which both are instance methods. However, a library we work with creates Configuration instances internally that we cannot access. So we need to add them as default resources in order to make them available to all subsequent Configuration instances (and those existing instances where {{loadDefaults}} equals {{true}}, of course).
> *Solution*
> The proposed solution is to add the following static methods:
> {code}
>   public static synchronized void addDefaultResource(URL url);
>   public static synchronized void addDefaultResource(Path file);
> {code}
> Please see the attached patch for details.
> *Example*
> In order to initialize the Configuration correctly, what we need to do currently is as follows:
> {code}
> // Set resource paths for the class loader
> File ffile = new File("/path/to/conf/dir");
> URL url = ffile.toURI().toURL();
> URLClassLoader systemClassLoader = (URLClassLoader) getClass().getClassLoader();
> Class<URLClassLoader> classLoaderClass = URLClassLoader.class; 
> Method method = classLoaderClass.getDeclaredMethod("addURL", new Class[]{URL.class}); 
> method.setAccessible(true); 
> method.invoke(systemClassLoader, new Object[]{url}); 
> //-----------------------------
> // Read cluster config
> for (File f : ffile.listFiles()) {
>       if(f.getName().toLowerCase().endsWith(".xml")) {
>             Configuration.addDefaultResource(f.getName());
>       }
> }
> {code}
> As you can see, we currently need to manipulate the class path through reflection which is kind of dirty.
> With the proposed solution, the same (adding default resources from outside the class path) can be achieved doing the following:
> {code}
> // Set resource paths for the class loader
> File ffile = new File("/path/to/conf/dir");
> //-----------------------------
> // Read cluster config
> for (File f : ffile.listFiles()) {
>       if(f.getName().toLowerCase().endsWith(".xml")) {
>             Configuration.addDefaultResource(new Path(f.getAbsolutePath()));
>       }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)