You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Sandesh Hegde <sa...@datatorrent.com> on 2016/02/10 21:46:52 UTC
Apex on Mapr
Hello Team,
We tried launching Apex demo app ( pi demo ) on the Mapr cluster using
dtcli but it is failing.
Initial error was "dtadmin" user not found. Then after creating the user
"dtadmin" on all the machines, it is still failing.
Here is the error
Diagnostics: Exception from container-launch.
Container id: container_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:304)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:354)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:87)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Shell output: main : command provided 1
This is probably setting up the YARN on MapR, any help is appreciated.
Thanks
Sandesh
Re: Apex on Mapr
Posted by Sasha Parfenov <sa...@datatorrent.com>.
Turns out the problem was related to unix account configuration on MapR
cluster. The user running Gateway was dtadmin (uid: 20266), the users
created across the rest of the cluster were dtadmin (uid: 10505). When
performing installation, Gateway configured DFS path /user/dtadmin to be
owned by dtadmin(20266), and then when app master came up on one of the
other nodes with dtadmin(10505) and tried to create application directory
under the same path, there was an exception due to permissions. So
container was failed, as can be seen from original error. The log below
came from RM. It was necessary to track down AM log on individual node to
see the details of why failure happened.
On a side node, for MapR installs, same user which runs Gateway process
needs to be created on all the cluster nodes. This is not necessary for
other distros.
Thanks,
Sasha
On Wed, Feb 10, 2016 at 1:02 PM, Vlad Rozov <v....@datatorrent.com> wrote:
> I saw similar errors when JAVA_HOME was not properly defined. Please check
> yarn logs.
>
> Thank you,
> Vlad
>
> Отправлено с iPhone
>
> On Feb 10, 2016, at 12:46, Sandesh Hegde <sa...@datatorrent.com> wrote:
>
> Hello Team,
>
> We tried launching Apex demo app ( pi demo ) on the Mapr cluster using
> dtcli but it is failing.
>
> Initial error was "dtadmin" user not found. Then after creating the user
> "dtadmin" on all the machines, it is still failing.
>
> Here is the error
>
> Diagnostics: Exception from container-launch.
> Container id: container_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
> at org.apache.hadoop.util.Shell.run(Shell.java:456)
> at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
> at
> org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:304)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:354)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:87)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Shell output: main : command provided 1
>
> This is probably setting up the YARN on MapR, any help is appreciated.
>
> Thanks
> Sandesh
>
>
--
*Sasha Parfenov*
Software Engineer, DataTorrent Inc.
3200 Patrick Henry Drive, 2nd floor
Santa Clara, CA 95054
sasha@datatorrent.com
https://www.datatorrent.com/
Re: Apex on Mapr
Posted by Vlad Rozov <v....@datatorrent.com>.
I saw similar errors when JAVA_HOME was not properly defined. Please check yarn logs.
Thank you,
Vlad
���������� � iPhone
> On Feb 10, 2016, at 12:46, Sandesh Hegde <sa...@datatorrent.com> wrote:
>
> Hello Team,
>
> We tried launching Apex demo app ( pi demo ) on the Mapr cluster using dtcli but it is failing.
>
> Initial error was "dtadmin" user not found. Then after creating the user "dtadmin" on all the machines, it is still failing.
>
> Here is the error
>
> Diagnostics: Exception from container-launch.
> Container id: container_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
> at org.apache.hadoop.util.Shell.run(Shell.java:456)
> at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
> at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:304)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:354)
> at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:87)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Shell output: main : command provided 1
>
> This is probably setting up the YARN on MapR, any help is appreciated.
>
> Thanks
> Sandesh
>