[jira] [Created] (SLIDER-515) Ensure Slider clients uploads all jars needed by AppMaster and construct own class path

["Sumit Mohanty (JIRA)" <ji...@apache.org>]

Sumit Mohanty created SLIDER-515:
------------------------------------

             Summary: Ensure Slider clients uploads all jars needed by AppMaster and construct own class path 
                 Key: SLIDER-515
                 URL: https://issues.apache.org/jira/browse/SLIDER-515
             Project: Slider
          Issue Type: Bug
          Components: appmaster, client
    Affects Versions: Slider 0.50
            Reporter: Sumit Mohanty
            Priority: Critical
             Fix For: Slider 0.60


Slider needs to ensure that all dependencies (jars) are automatically uploaded and available to the AppMaster.

Proposal:
Slider client will automatically upload all the required jars for the AM. By default, it will pick the jars from <Slider Install Dir>/lib folder. _This is what is done today but it only uploads a subset of jars._

The location to upload from, can be influenced by setting an environment variable (SLIDER_LIBS) to an alternate path to libraries. This is needed for code embedding slider client - e.g. Slider App View.

Also, to support embedded client, pass in the path to the libraries as a system property to java.

Slider AM startup will be associated with a classpath that will include:
* path to Slider config (log4j), check if this is being uploaded as well
* path to hadoop conf dir
* path to the uploaded library files 

The downside is duplicate jars - one per application instance and longer startup. These two are not critical for Slider apps. In the next releases, Slider may switch to using libraries uploaded to HDFS in a version aware manner. In this case, slider can be configured with a HDFS path that points to the versioned libraries to be used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)