You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Douglas Service <ds...@gmail.com> on 2016/10/05 04:00:00 UTC
HelloREEF runs on Windows but not on Linux
************************************************************
************************************************************
************************************************************
*************************************************
When I run HelloREEF on Linux following the instructions on the developer
WIKI, I get the following errors which looks like it is having a problem
finding java.
dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
target/reef-examples-0.16.0-SNAPSHOT-shaded.jar org.apache.reef.examples.
hello.HelloREEF
Powered by
___________ ______ ______ _______
/ ______ / / ___/ / ___/ / ____/
/ _____/ / /__ / /__ / /___
/ /\ \ / ___/ / ___/ / ____/
/ / \ \ / /__ / /__ / /
/__/ \__\ /_____/ /_____/ /__/
Oct 05, 2016 3:02:53 AM org.apache.reef.util.REEFVersion logVersion
INFO: REEF Version: 0.16.0-SNAPSHOT
Oct 05, 2016 3:02:53 AM org.apache.reef.runtime.local.process.RunnableProcess
run
SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
-Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
reef/local/*:reef/global/*:/code/reef/lang/java/reef-
examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar, -Dproc_reef,
org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
Exception:java.io.IOException: Cannot run program "null/bin/java" (in
directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/driver"):
error=2, No such file or directory
Oct 05, 2016 3:03:03 AM org.apache.reef.client.DriverLauncher run
WARNING: The Job timed out.
Oct 05, 2016 3:03:03 AM org.apache.reef.examples.hello.HelloREEF main
INFO: REEF job completed: FORCE_CLOSED
dougse@reefdevlx:/code/reef$ java -version
openjdk version "1.8.0_91"
OpenJDK Runtime Environment (build 1.8.0_91-8u91-b14-3ubuntu1~16.04.1-b14)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
dougse@reefdevlx:/code/reef$
However, java is installed and here is the version. The only configuration
for REEF on Linux I can find in the developer documentation is setting
$REEF_HOME which I did. Am I missing a configuration step?
dougse@reefdevlx:/code/reef$ java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
************************************************************
************************************************************
************************************************************
*************************************************
When I run HelloREEF on windows it runs and I get the following output.
Which appears to be fine
C:\code\reef [master ≡]> java -cp lang/java/reef-examples/target
/reef-examples-0.16.0-SNAPSHOT-shaded.jar org.apache.reef.examples.hello
.HelloREEF
Powered by
___________ ______ ______ _______
/ ______ / / ___/ / ___/ / ____/
/ _____/ / /__ / /__ / /___
/ /\ \ / ___/ / ___/ / ____/
/ / \ \ / /__ / /__ / /
/__/ \__\ /_____/ /_____/ /__/
Oct 04, 2016 8:04:39 PM org.apache.reef.util.REEFVersion logVersion
INFO: REEF Version: 0.16.0-SNAPSHOT
Oct 04, 2016 8:04:40 PM org.apache.reef.client.DriverLauncher$RunningJobHandler
onNext
INFO: The Job HelloREEF is running.
Oct 04, 2016 8:04:42 PM
org.apache.reef.client.DriverLauncher$CompletedJobHandler
onNext
INFO: The Job HelloREEF is done.
Oct 04, 2016 8:04:42 PM org.apache.reef.examples.hello.HelloREEF main
INFO: REEF job completed: COMPLETED
However, when I look at the output
at C:\code\reef\REEF_LOCAL_RUNTIME\HelloREEF-1475636679071\driver, I see
the following error which looks like a socket error. Is this the expected
output for this example? If not what am I missing? My powershell
environment is setup as per the developer WIKI.
Oct 04, 2016 8:04:39 PM org.apache.reef.runtime.common.REEFLauncher main^M
INFO: Entering REEFLauncher.main().^M
Oct 04, 2016 8:04:39 PM org.apache.reef.util.REEFVersion logVersion^M
INFO: REEF Version: 0.16.0-SNAPSHOT^M
SLF4J: Class path contains multiple SLF4J bindings.^M
SLF4J: Found binding in
[jar:file:/C:/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636679071/driver/reef/global/reef-examples-0.16.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]^M
SLF4J: Found binding in
[jar:file:/C:/code/reef/lang/java/reef-examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar!/org/slf4j/impl/StaticLoggerBinder.class]^M
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation.^M
SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]^M
Oct 04, 2016 8:04:40 PM
org.apache.reef.examples.hello.HelloDriver$EvaluatorAllocatedHandler
onNext^M
INFO: Submitting HelloREEF task to AllocatedEvaluator:
AllocatedEvaluator{ID='Node-2-1475636680531'}^M
Oct 04, 2016 8:04:40 PM
org.apache.reef.runtime.local.driver.ResourceManager sendRuntimeStatus^M
INFO: Allocated: 1, Outstanding requests: Optional:{0}^M
Oct 04, 2016 8:04:40 PM
org.apache.reef.runtime.local.driver.ResourceManager sendRuntimeStatus^M
INFO: Allocated: 1, Outstanding requests: Optional:{0}^M
Oct 04, 2016 8:04:40 PM
org.apache.reef.examples.hello.HelloDriver$StartHandler onNext^M
INFO: Requested Evaluator.^M
Oct 04, 2016 8:04:40 PM
org.apache.reef.runtime.common.driver.evaluator.AllocatedEvaluatorImpl
makeRootServiceConfiguration^M
INFO: No service configuration given and no ConfigurationProviders set.^M
Oct 04, 2016 8:04:40 PM
org.apache.reef.runtime.local.process.ReefRunnableProcessObserver
onResourceStatus^M
INFO: Sending resource status:
org.apache.reef.runtime.common.driver.resourcemanager.ResourceStatusEventImpl@282c0956
^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.common.driver.task.TaskRepresenter onTaskInit^M
WARNING: Received a INIT message for task with id HelloREEFTask which we
have seen before. Ignoring the second message^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.common.driver.defaults.DefaultTaskRunningHandler
onNext^M
INFO: Received TaskRuntime: RunningTask{taskId='HelloREEFTask'}^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.common.driver.defaults.DefaultTaskCompletionHandler
onNext^M
INFO: Received CompletedTask: CompletedTask{ID='HelloREEFTask'} :: CLOSING
context:
EvaluatorContext{contextIdentifier='RootContext_Node-2-1475636680531',
evaluatorIdentifier='Node-2-1475636680531', p
arentID=Optional.empty}^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.common.driver.defaults.DefaultEvaluatorCompletionHandler
onNext^M
INFO: Received CompletedEvaluator:
CompletedEvaluator{id='Node-2-1475636680531'}^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.local.driver.ContainerManager release^M
INFO: Releasing Container with containerId
[ProcessContainer{containedID='Node-2-1475636680531', nodeID='Node-2',
errorHandlerRID='socket://172.25.0.5:18813',
folder=C:\code\reef\REEF_LOCAL_RUNTIME\He
lloREEF-1475636679071\Node-2-1475636680531', rack=/default-rack}]^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.local.driver.ProcessContainer close^M
WARNING: Force-closing a container that is still running:
ProcessContainer{containedID='Node-2-1475636680531', nodeID='Node-2',
errorHandlerRID='socket://172.25.0.5:18813', folder=C:\code\reef\REEF_LO
CAL_RUNTIME\HelloREEF-1475636679071\Node-2-1475636680531',
rack=/default-rack}^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.wake.remote.transport.netty.NettyChannelHandler
exceptionCaught^M
INFO: Unexpected exception from downstream. channel: [id: 0x910c4204, /
172.25.0.5:16175 => /172.25.0.5:18813] local: /172.25.0.5:18813 remote: /
172.25.0.5:16175^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.wake.remote.transport.netty.NettyChannelHandler
exceptionCaught^M
WARNING: Unexpected exception from downstream.^M
java.io.IOException: An existing connection was forcibly closed by the
remote host^M
at sun.nio.ch.SocketDispatcher.read0(Native Method)^M
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)^M
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)^M
at sun.nio.ch.IOUtil.read(IOUtil.java:192)^M
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)^M
at
io.netty.buffer.UnpooledUnsafeDirectByteBuf.setBytes(UnpooledUnsafeDirectByteBuf.java:446)^M
at
io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:879)^M
at
io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:225)^M
at
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:114)^M
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)^M
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)^M
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)^M
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)^M
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)^M
at java.lang.Thread.run(Thread.java:745)^M
^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.wake.remote.transport.netty.AbstractNettyEventListener
exceptionCaught^M
WARNING: ExceptionEvent: local: /172.25.0.5:18813 remote: /172.25.0.5:16175
:: java.io.IOException: An existing connection was forcibly closed by the
remote host^M
Oct 04, 2016 8:04:41 PM
org.apache.reef.runtime.local.process.ReefRunnableProcessObserver
onResourceStatus^M
INFO: Sending resource status:
org.apache.reef.runtime.common.driver.resourcemanager.ResourceStatusEventImpl@966eb0e
^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.local.process.RunnableProcess cancel^M
WARNING: The child process survived Process.destroy()^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.local.process.RunnableProcess cancel^M
WARNING: Attempting to kill the process via the kill command line^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.local.driver.ResourceManager sendRuntimeStatus^M
INFO: Allocated: 0, Outstanding requests: Optional:{0}^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.common.driver.idle.DriverIdleManager
onPotentiallyIdle^M
INFO: All components indicated idle. Initiating Driver shutdown.^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.common.driver.DriverStatusManager onComplete^M
INFO: Clean shutdown of the Driver.^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.local.driver.ContainerManager release^M
INFO: Ignoring release request for unknown containerID
[Node-2-1475636680531]^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.local.driver.ResourceManager sendRuntimeStatus^M
INFO: Allocated: 0, Outstanding requests: Optional:{0}^M
Oct 04, 2016 8:04:42 PM
org.apache.reef.runtime.common.driver.DriverRuntimeStopHandler onNext^M
INFO: Driver shutdown complete^M
Oct 04, 2016 8:04:42 PM org.apache.reef.runtime.common.REEFLauncher main^M
INFO: Exiting REEFLauncher.main()^M
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m;
support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=128m; support was removed in 8.0
Re: HelloREEF runs on Windows but not on Linux
Posted by Markus Weimer <ma...@weimo.de>.
On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych
<se...@gmail.com> wrote:
> That is, *by default* REEF client assumes that
> 1) you have JAVA_HOME environment variable set, and
> 2) your JAVA_HOME is the same as the one on the remote host.
The first assumption should be true, right? The second only makes
sense when local and remote host are part of the same cluster with a
similar installation.
Markus
Re: HelloREEF runs on Windows but not on Linux
Posted by Douglas Service <ds...@gmail.com>.
Okay I will give it a go and get back to you. Thanks
On Thu, Nov 10, 2016 at 4:32 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> hi Douglas,
>
> The fix is in the master now - please go ahead and give it a try.
>
> Cheers,
> Sergiy.
>
> On Mon, Nov 7, 2016 at 6:00 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
> > OK I think I'll create a PR for this. Here goes: https://github.com/
> > apache/reef/pull/1179
> >
> > Cheers,
> > Sergiy.
> >
> > On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych <
> > sergiy.matusevych@gmail.com> wrote:
> >
> >> Hi Douglas,
> >>
> >> I think I know what's going on. The key is this line:
> >>
> >> public final class JavaLaunchCommandBuilder implements
> >> LaunchCommandBuilder {
> >> // ...
> >> private static final String DEFAULT_JAVA_PATH =
> >> System.getenv("JAVA_HOME") + "/bin/" + "java";
> >>
> >> That is, *by default* REEF client assumes that
> >> 1) you have JAVA_HOME environment variable set, and
> >> 2) your JAVA_HOME is the same as the one on the remote host.
> >>
> >> I am not sure this is the best default value. I would rather set
> >> DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the system
> >> PATH - on both client and in the cluster.
> >>
> >> I've actually made that fix to debug REEF applications on my PC, but I
> >> can transform it into a pull request in no time.
> >>
> >> Markus, what do you think? I would love to file a JIRA and a PR for
> this.
> >>
> >> Cheers,
> >> Sergiy.
> >>
> >> On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com>
> >> wrote:
> >>
> >> > When I run HelloREEF on Linux following the instructions on the
> >> developer
> >> > WIKI, I get the following errors which looks like it is having a
> problem
> >> > finding java.
> >> >
> >> >
> >> > dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
> >> > target/reef-examples-0.16.0-SNAPSHOT-shaded.jar
> >> org.apache.reef.examples.
> >> > hello.HelloREEF
> >>
> >> [...]
> >>
> >> > SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
> >> > -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
> >> > reef/local/*:reef/global/*:/code/reef/lang/java/reef-
> >> > examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar,
> -Dproc_reef,
> >> > org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
> >> > Exception:java.io.IOException: Cannot run program "null/bin/java" (in
> >> > directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/
> >> driver"):
> >> > error=2, No such file or directory
> >>
> >>
> >
>
Re: HelloREEF runs on Windows but not on Linux
Posted by Sergiy Matusevych <se...@gmail.com>.
Hi Doug,
We've discussed the PATH vs. JAVA_HOME issue with Markus, and came to the
conclusion that we should follow the Spark behavior, i.e. rely on the
JAVA_HOME variable. I think we already have it in the documentation, but I
will double check.
P.S. I believe on Ubuntu you can also set JAVA_HOME=/usr to use the default
JVM, since the only way we use JAVA_HOME in REEF is to specify the path to
java binary when starting a REEF process, like this:
"{{JAVA_HOME}}/bin/java ..."
Cheers,
Sergiy.
On Fri, Nov 11, 2016 at 6:09 PM, Douglas Service <ds...@gmail.com> wrote:
> Either way we should probably put a note about JAVA_HOME on the tutorials
> page on the REEF Wiki in case the user still has problems. I can add the
> information to the Wiki. I can also update the code based on what we
> decide.
>
> Doug
>
> On Fri, Nov 11, 2016 at 6:02 PM, Douglas Service <ds...@gmail.com>
> wrote:
>
> > Both the Java and CLR versions of HelloReef run fine on windows.
> >
> > However on Ubuntu I had to set the JAVA_HOME to
> > /usr/lib/jvm/java-8-oracle/jre to get the HelloREEF to run correctly. I
> do
> > not know about other Linux distributions, but on Ubuntu, the
> > update-alternatives functionality, which is enabled by default even only
> > one version of java is installed, puts links in /usr/bin for java, jar,
> jdb
> > etc. This causes all of the java programs to appear in the user's usual
> > path because /usr/bin is always included in the user's path by default
> > unless they specifically remove it. Plus they always get the version of
> > java controlled by "update-alternatives --config java". If this is the
> case
> > on other Linux distributions, we should use JAVA_HOME if it is set,
> > otherwise we should just rely on the user's path when we invoke java.
> >
> > Doug
> >
> >
> > On Thu, Nov 10, 2016 at 4:32 PM, Sergiy Matusevych <
> > sergiy.matusevych@gmail.com> wrote:
> >
> >> hi Douglas,
> >>
> >> The fix is in the master now - please go ahead and give it a try.
> >>
> >> Cheers,
> >> Sergiy.
> >>
> >> On Mon, Nov 7, 2016 at 6:00 PM, Sergiy Matusevych <
> >> sergiy.matusevych@gmail.com> wrote:
> >>
> >> > OK I think I'll create a PR for this. Here goes: https://github.com/
> >> > apache/reef/pull/1179
> >> >
> >> > Cheers,
> >> > Sergiy.
> >> >
> >> > On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych <
> >> > sergiy.matusevych@gmail.com> wrote:
> >> >
> >> >> Hi Douglas,
> >> >>
> >> >> I think I know what's going on. The key is this line:
> >> >>
> >> >> public final class JavaLaunchCommandBuilder implements
> >> >> LaunchCommandBuilder {
> >> >> // ...
> >> >> private static final String DEFAULT_JAVA_PATH =
> >> >> System.getenv("JAVA_HOME") + "/bin/" + "java";
> >> >>
> >> >> That is, *by default* REEF client assumes that
> >> >> 1) you have JAVA_HOME environment variable set, and
> >> >> 2) your JAVA_HOME is the same as the one on the remote host.
> >> >>
> >> >> I am not sure this is the best default value. I would rather set
> >> >> DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the
> >> system
> >> >> PATH - on both client and in the cluster.
> >> >>
> >> >> I've actually made that fix to debug REEF applications on my PC, but
> I
> >> >> can transform it into a pull request in no time.
> >> >>
> >> >> Markus, what do you think? I would love to file a JIRA and a PR for
> >> this.
> >> >>
> >> >> Cheers,
> >> >> Sergiy.
> >> >>
> >> >> On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > When I run HelloREEF on Linux following the instructions on the
> >> >> developer
> >> >> > WIKI, I get the following errors which looks like it is having a
> >> problem
> >> >> > finding java.
> >> >> >
> >> >> >
> >> >> > dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
> >> >> > target/reef-examples-0.16.0-SNAPSHOT-shaded.jar
> >> >> org.apache.reef.examples.
> >> >> > hello.HelloREEF
> >> >>
> >> >> [...]
> >> >>
> >> >> > SEVERE: Unable to spawn process "driver" wth command
> [null/bin/java,
> >> >> > -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
> >> >> > reef/local/*:reef/global/*:/code/reef/lang/java/reef-
> >> >> > examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar,
> >> -Dproc_reef,
> >> >> > org.apache.reef.runtime.common.REEFLauncher,
> reef/local/driver.conf]
> >> >> > Exception:java.io.IOException: Cannot run program "null/bin/java"
> >> (in
> >> >> > directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/
> >> >> driver"):
> >> >> > error=2, No such file or directory
> >> >>
> >> >>
> >> >
> >>
> >
> >
>
Re: HelloREEF runs on Windows but not on Linux
Posted by Douglas Service <ds...@gmail.com>.
Either way we should probably put a note about JAVA_HOME on the tutorials
page on the REEF Wiki in case the user still has problems. I can add the
information to the Wiki. I can also update the code based on what we decide.
Doug
On Fri, Nov 11, 2016 at 6:02 PM, Douglas Service <ds...@gmail.com> wrote:
> Both the Java and CLR versions of HelloReef run fine on windows.
>
> However on Ubuntu I had to set the JAVA_HOME to
> /usr/lib/jvm/java-8-oracle/jre to get the HelloREEF to run correctly. I do
> not know about other Linux distributions, but on Ubuntu, the
> update-alternatives functionality, which is enabled by default even only
> one version of java is installed, puts links in /usr/bin for java, jar, jdb
> etc. This causes all of the java programs to appear in the user's usual
> path because /usr/bin is always included in the user's path by default
> unless they specifically remove it. Plus they always get the version of
> java controlled by "update-alternatives --config java". If this is the case
> on other Linux distributions, we should use JAVA_HOME if it is set,
> otherwise we should just rely on the user's path when we invoke java.
>
> Doug
>
>
> On Thu, Nov 10, 2016 at 4:32 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
>> hi Douglas,
>>
>> The fix is in the master now - please go ahead and give it a try.
>>
>> Cheers,
>> Sergiy.
>>
>> On Mon, Nov 7, 2016 at 6:00 PM, Sergiy Matusevych <
>> sergiy.matusevych@gmail.com> wrote:
>>
>> > OK I think I'll create a PR for this. Here goes: https://github.com/
>> > apache/reef/pull/1179
>> >
>> > Cheers,
>> > Sergiy.
>> >
>> > On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych <
>> > sergiy.matusevych@gmail.com> wrote:
>> >
>> >> Hi Douglas,
>> >>
>> >> I think I know what's going on. The key is this line:
>> >>
>> >> public final class JavaLaunchCommandBuilder implements
>> >> LaunchCommandBuilder {
>> >> // ...
>> >> private static final String DEFAULT_JAVA_PATH =
>> >> System.getenv("JAVA_HOME") + "/bin/" + "java";
>> >>
>> >> That is, *by default* REEF client assumes that
>> >> 1) you have JAVA_HOME environment variable set, and
>> >> 2) your JAVA_HOME is the same as the one on the remote host.
>> >>
>> >> I am not sure this is the best default value. I would rather set
>> >> DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the
>> system
>> >> PATH - on both client and in the cluster.
>> >>
>> >> I've actually made that fix to debug REEF applications on my PC, but I
>> >> can transform it into a pull request in no time.
>> >>
>> >> Markus, what do you think? I would love to file a JIRA and a PR for
>> this.
>> >>
>> >> Cheers,
>> >> Sergiy.
>> >>
>> >> On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com>
>> >> wrote:
>> >>
>> >> > When I run HelloREEF on Linux following the instructions on the
>> >> developer
>> >> > WIKI, I get the following errors which looks like it is having a
>> problem
>> >> > finding java.
>> >> >
>> >> >
>> >> > dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
>> >> > target/reef-examples-0.16.0-SNAPSHOT-shaded.jar
>> >> org.apache.reef.examples.
>> >> > hello.HelloREEF
>> >>
>> >> [...]
>> >>
>> >> > SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
>> >> > -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
>> >> > reef/local/*:reef/global/*:/code/reef/lang/java/reef-
>> >> > examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar,
>> -Dproc_reef,
>> >> > org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
>> >> > Exception:java.io.IOException: Cannot run program "null/bin/java"
>> (in
>> >> > directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/
>> >> driver"):
>> >> > error=2, No such file or directory
>> >>
>> >>
>> >
>>
>
>
Re: HelloREEF runs on Windows but not on Linux
Posted by Douglas Service <ds...@gmail.com>.
Both the Java and CLR versions of HelloReef run fine on windows.
However on Ubuntu I had to set the JAVA_HOME to
/usr/lib/jvm/java-8-oracle/jre to get the HelloREEF to run correctly. I do
not know about other Linux distributions, but on Ubuntu, the
update-alternatives functionality, which is enabled by default even only
one version of java is installed, puts links in /usr/bin for java, jar, jdb
etc. This causes all of the java programs to appear in the user's usual
path because /usr/bin is always included in the user's path by default
unless they specifically remove it. Plus they always get the version of
java controlled by "update-alternatives --config java". If this is the case
on other Linux distributions, we should use JAVA_HOME if it is set,
otherwise we should just rely on the user's path when we invoke java.
Doug
On Thu, Nov 10, 2016 at 4:32 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> hi Douglas,
>
> The fix is in the master now - please go ahead and give it a try.
>
> Cheers,
> Sergiy.
>
> On Mon, Nov 7, 2016 at 6:00 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
> > OK I think I'll create a PR for this. Here goes: https://github.com/
> > apache/reef/pull/1179
> >
> > Cheers,
> > Sergiy.
> >
> > On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych <
> > sergiy.matusevych@gmail.com> wrote:
> >
> >> Hi Douglas,
> >>
> >> I think I know what's going on. The key is this line:
> >>
> >> public final class JavaLaunchCommandBuilder implements
> >> LaunchCommandBuilder {
> >> // ...
> >> private static final String DEFAULT_JAVA_PATH =
> >> System.getenv("JAVA_HOME") + "/bin/" + "java";
> >>
> >> That is, *by default* REEF client assumes that
> >> 1) you have JAVA_HOME environment variable set, and
> >> 2) your JAVA_HOME is the same as the one on the remote host.
> >>
> >> I am not sure this is the best default value. I would rather set
> >> DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the system
> >> PATH - on both client and in the cluster.
> >>
> >> I've actually made that fix to debug REEF applications on my PC, but I
> >> can transform it into a pull request in no time.
> >>
> >> Markus, what do you think? I would love to file a JIRA and a PR for
> this.
> >>
> >> Cheers,
> >> Sergiy.
> >>
> >> On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com>
> >> wrote:
> >>
> >> > When I run HelloREEF on Linux following the instructions on the
> >> developer
> >> > WIKI, I get the following errors which looks like it is having a
> problem
> >> > finding java.
> >> >
> >> >
> >> > dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
> >> > target/reef-examples-0.16.0-SNAPSHOT-shaded.jar
> >> org.apache.reef.examples.
> >> > hello.HelloREEF
> >>
> >> [...]
> >>
> >> > SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
> >> > -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
> >> > reef/local/*:reef/global/*:/code/reef/lang/java/reef-
> >> > examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar,
> -Dproc_reef,
> >> > org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
> >> > Exception:java.io.IOException: Cannot run program "null/bin/java" (in
> >> > directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/
> >> driver"):
> >> > error=2, No such file or directory
> >>
> >>
> >
>
Re: HelloREEF runs on Windows but not on Linux
Posted by Sergiy Matusevych <se...@gmail.com>.
hi Douglas,
The fix is in the master now - please go ahead and give it a try.
Cheers,
Sergiy.
On Mon, Nov 7, 2016 at 6:00 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> OK I think I'll create a PR for this. Here goes: https://github.com/
> apache/reef/pull/1179
>
> Cheers,
> Sergiy.
>
> On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych <
> sergiy.matusevych@gmail.com> wrote:
>
>> Hi Douglas,
>>
>> I think I know what's going on. The key is this line:
>>
>> public final class JavaLaunchCommandBuilder implements
>> LaunchCommandBuilder {
>> // ...
>> private static final String DEFAULT_JAVA_PATH =
>> System.getenv("JAVA_HOME") + "/bin/" + "java";
>>
>> That is, *by default* REEF client assumes that
>> 1) you have JAVA_HOME environment variable set, and
>> 2) your JAVA_HOME is the same as the one on the remote host.
>>
>> I am not sure this is the best default value. I would rather set
>> DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the system
>> PATH - on both client and in the cluster.
>>
>> I've actually made that fix to debug REEF applications on my PC, but I
>> can transform it into a pull request in no time.
>>
>> Markus, what do you think? I would love to file a JIRA and a PR for this.
>>
>> Cheers,
>> Sergiy.
>>
>> On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com>
>> wrote:
>>
>> > When I run HelloREEF on Linux following the instructions on the
>> developer
>> > WIKI, I get the following errors which looks like it is having a problem
>> > finding java.
>> >
>> >
>> > dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
>> > target/reef-examples-0.16.0-SNAPSHOT-shaded.jar
>> org.apache.reef.examples.
>> > hello.HelloREEF
>>
>> [...]
>>
>> > SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
>> > -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
>> > reef/local/*:reef/global/*:/code/reef/lang/java/reef-
>> > examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar, -Dproc_reef,
>> > org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
>> > Exception:java.io.IOException: Cannot run program "null/bin/java" (in
>> > directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/
>> driver"):
>> > error=2, No such file or directory
>>
>>
>
Re: HelloREEF runs on Windows but not on Linux
Posted by Sergiy Matusevych <se...@gmail.com>.
OK I think I'll create a PR for this. Here goes:
https://github.com/apache/reef/pull/1179
Cheers,
Sergiy.
On Mon, Nov 7, 2016 at 5:36 PM, Sergiy Matusevych <
sergiy.matusevych@gmail.com> wrote:
> Hi Douglas,
>
> I think I know what's going on. The key is this line:
>
> public final class JavaLaunchCommandBuilder implements
> LaunchCommandBuilder {
> // ...
> private static final String DEFAULT_JAVA_PATH =
> System.getenv("JAVA_HOME") + "/bin/" + "java";
>
> That is, *by default* REEF client assumes that
> 1) you have JAVA_HOME environment variable set, and
> 2) your JAVA_HOME is the same as the one on the remote host.
>
> I am not sure this is the best default value. I would rather set
> DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the system
> PATH - on both client and in the cluster.
>
> I've actually made that fix to debug REEF applications on my PC, but I can
> transform it into a pull request in no time.
>
> Markus, what do you think? I would love to file a JIRA and a PR for this.
>
> Cheers,
> Sergiy.
>
> On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com> wrote:
>
> > When I run HelloREEF on Linux following the instructions on the developer
> > WIKI, I get the following errors which looks like it is having a problem
> > finding java.
> >
> >
> > dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
> > target/reef-examples-0.16.0-SNAPSHOT-shaded.jar
> org.apache.reef.examples.
> > hello.HelloREEF
>
> [...]
>
> > SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
> > -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
> > reef/local/*:reef/global/*:/code/reef/lang/java/reef-
> > examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar, -Dproc_reef,
> > org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
> > Exception:java.io.IOException: Cannot run program "null/bin/java" (in
> > directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-
> 1475636573040/driver"):
> > error=2, No such file or directory
>
>
Re: HelloREEF runs on Windows but not on Linux
Posted by Sergiy Matusevych <se...@gmail.com>.
Hi Douglas,
I think I know what's going on. The key is this line:
public final class JavaLaunchCommandBuilder implements LaunchCommandBuilder
{
// ...
private static final String DEFAULT_JAVA_PATH =
System.getenv("JAVA_HOME") + "/bin/" + "java";
That is, *by default* REEF client assumes that
1) you have JAVA_HOME environment variable set, and
2) your JAVA_HOME is the same as the one on the remote host.
I am not sure this is the best default value. I would rather set
DEFAULT_JAVA_PATH = "java"; and assume that java binary is in the system
PATH - on both client and in the cluster.
I've actually made that fix to debug REEF applications on my PC, but I can
transform it into a pull request in no time.
Markus, what do you think? I would love to file a JIRA and a PR for this.
Cheers,
Sergiy.
On Tue, Oct 4, 2016 at 9:00 PM, Douglas Service <ds...@gmail.com> wrote:
> When I run HelloREEF on Linux following the instructions on the developer
> WIKI, I get the following errors which looks like it is having a problem
> finding java.
>
>
> dougse@reefdevlx:/code/reef$ java -cp lang/java/reef-examples/
> target/reef-examples-0.16.0-SNAPSHOT-shaded.jar org.apache.reef.examples.
> hello.HelloREEF
[...]
> SEVERE: Unable to spawn process "driver" wth command [null/bin/java,
> -Xmx512m, -XX:PermSize=128m, -XX:MaxPermSize=128m, -classpath,
> reef/local/*:reef/global/*:/code/reef/lang/java/reef-
> examples/target/reef-examples-0.16.0-SNAPSHOT-shaded.jar, -Dproc_reef,
> org.apache.reef.runtime.common.REEFLauncher, reef/local/driver.conf]
> Exception:java.io.IOException: Cannot run program "null/bin/java" (in
> directory "/code/reef/REEF_LOCAL_RUNTIME/HelloREEF-1475636573040/driver"):
> error=2, No such file or directory