Java classpath separator

[Reed Umbrasas <ry...@microsoft.com.INVALID>]

Hi,

I was looking at how the JavaLaunchCommandBuilder.java (https://github.com/apache/reef/blob/master/lang/java/reef-common/src/main/java/org/apache/reef/runtime/common/launch/JavaLaunchCommandBuilder.java#L178) is implemented. It uses "File.pathSeparatorChar" to separate paths in Java classpath. So it's separated with semi-colons in Windows and colons in Linux. This works fine when both the client and the remote cluster are running the same OS. However, when starting a REEF job from Windows, classpath gets separated with semi-colons which fails when trying to run the job on a Linux cluster.

Has this issue been reported/addressed before? Are there any recommendations on how to handle it? (Happy to write code, just wanted to hear people's thoughts)

Thank you!

Re: Java classpath separator

[Markus Weimer <ma...@weimo.de>]

On Tue, Jan 23, 2018 at 12:15 PM, Reed Umbrasas <
ryumbra@microsoft.com.invalid> wrote:

> I was looking at how the JavaLaunchCommandBuilder.java (
> https://github.com/apache/reef/blob/master/lang/java/
> reef-common/src/main/java/org/apache/reef/runtime/common/launch/
> JavaLaunchCommandBuilder.java#L178) is implemented. It uses
> "File.pathSeparatorChar" to separate paths in Java classpath. So it's
> separated with semi-colons in Windows and colons in Linux. This works fine
> when both the client and the remote cluster are running the same OS.
> However, when starting a REEF job from Windows, classpath gets separated
> with semi-colons which fails when trying to run the job on a Linux cluster.
>
> Has this issue been reported/addressed before? Are there any
> recommendations on how to handle it? (Happy to write code, just wanted to
> hear people's thoughts)
>

I know *of* the issue, but I don't think this has been addressed. In fact,
I believe it was one of the reasons for having an HDInsight runtime in
addition to a YARN runtime. IIRC, the HDI runtime hard codes the separator.

Is the same possible in your use case? If not, we might want to make this
configurable.

Markus