[jira] [Commented] (FLINK-13127) "--yarnship" doesn't support resource classloading

["Yang Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-13127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16880157#comment-16880157 ] 

Yang Wang commented on FLINK-13127:
-----------------------------------

Hi [~davidmoravek],

You are correct. The shipped files and jars are included in CLASSPATH of job manager and task manager. However, the container workDir is not in CLASSPATH. So you could not use java.lang.ClassLoader#getResource to read/write your file. This feature should be supported in the future and maybe you could open your file by the following codes now.
{code:java}
Map<String, String> env = System.getenv(); 
final String workingDirectory = env.get(ApplicationConstants.Environment.PWD.key());
File myFile = new File(workingDirectory, "myFileName");
{code}

> "--yarnship" doesn't support resource classloading
> --------------------------------------------------
>
>                 Key: FLINK-13127
>                 URL: https://issues.apache.org/jira/browse/FLINK-13127
>             Project: Flink
>          Issue Type: Bug
>          Components: Deployment / YARN
>    Affects Versions: 1.8.1
>            Reporter: David Moravek
>            Priority: Major
>
> Currently yarnship works as follows:
>  * user specifies directory to ship with the job
>  * yarn ships it with the container
>  * org.apache.flink.yarn.AbstractYarnClusterDescriptor#uploadAndRegisterFiles traverses directory recursively and adds each file to the classpath
> This works well for shipping jars, but doesn't work correctly with shipping resources that we want to load using java.lang.ClassLoader#getResource method.
> In order to make resource classloading work, we need to register it's directory instead of the file itself (java classpath expects directories or archives).
> CLASSPATH="shipped/custom.conf:${CLASSPATH}" needs to become CLASSPATH="shipped:${CLASSPATH}"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)