[jira] [Commented] (SLIDER-1192) Slider Agent tarball should be uploaded to HDFS to avoid client side dependency

["Sergey Shelukhin (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLIDER-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864488#comment-15864488 ] 

Sergey Shelukhin commented on SLIDER-1192:
------------------------------------------

ping

> Slider Agent tarball should be uploaded to HDFS to avoid client side dependency
> -------------------------------------------------------------------------------
>
>                 Key: SLIDER-1192
>                 URL: https://issues.apache.org/jira/browse/SLIDER-1192
>             Project: Slider
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Gour Saha
>            Priority: Blocker
>             Fix For: Slider 1.0.0
>
>
> Trying to use Slider client from a Hive process that is shipped with slider client JAR
> {noformat} 
> Caused by: java.io.FileNotFoundException: File /grid/5/sershe/tez-autobuild/dist/hive/lib/slider-agent.tar.gz does not exist
>    at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:606)
>    at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819)
>    at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596)
>    at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
>    at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
>    at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1965)
>    at org.apache.slider.providers.ProviderUtils.addAgentTar(ProviderUtils.java:128)
>    at org.apache.slider.providers.agent.AgentClientProvider.prepareAMAndConfigForLaunch(AgentClientProvider.java:270){noformat}
> I can see the gz file in the slider lib directory. However, using slider client one should not even be required to have slider on the box (Hive already has the jars); nor is there a good way to find where it is, if installed (given that jar is shipped and we are not aware of the slider installation).
> The gz file should be included with client jars (perhaps as a resource?)
> Update: setting slider.libdir and/or SLIDER_HOME actually doesn't work at all.
> The code in addAgentTar basically looks for it in the directory of the client jar, no option to override it at all.
> That makes slider client absolutely unusable



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)