You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Thomas Jungblut (JIRA)" <ji...@apache.org> on 2012/11/12 12:29:17 UTC
[jira] [Created] (HAMA-670) Hama YARN
Thomas Jungblut created HAMA-670:
------------------------------------
Summary: Hama YARN
Key: HAMA-670
URL: https://issues.apache.org/jira/browse/HAMA-670
Project: Hama
Issue Type: Umbrella
Components: bsp core, build , yarn
Affects Versions: 0.6.0
Reporter: Thomas Jungblut
Fix For: 0.7.0
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-670) Hama YARN
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13498920#comment-13498920 ]
Thomas Jungblut commented on HAMA-670:
--------------------------------------
A bit more detailed answer, in addition to my "there is no cluster status" on the mailing list:
The ClusterStatus exists to get a notion of how many tasks slots are available and how many tasks are running, besides the info about the grooms.
So in YARN, you don't have these informations, why? Because you should not care about this in the client. So our BSPClient should not be aware of the cluster in YARN. Which sadly makes HAMA-672 useless because you have to override the job submission mechanism, because we require the number of tasks before submitting to the cluster.
In the end, this functionality must be added to the ApplicationMaster, because it can decide how much container need to launch (although he should also be not aware of the resources in the cluster).
Looks like the whole task will be a bit more ugly than I initially thought it would be..
> Hama YARN
> ---------
>
> Key: HAMA-670
> URL: https://issues.apache.org/jira/browse/HAMA-670
> Project: Hama
> Issue Type: Umbrella
> Components: bsp core, build , yarn
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
> Fix For: 0.7.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-670) Hama YARN
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13496113#comment-13496113 ]
Edward J. Yoon commented on HAMA-670:
-------------------------------------
This is my first attempt w/ CDH4. What should be returned from BSPJobClient.getClusterStatus() method?
{code}
edward@udanax:~/workspace/hama-trunk$ yarn jar
yarn/target/hama-yarn-0.6.0-SNAPSHOT.jar
org.apache.hama.bsp.YarnSerializePrinting
12/11/13 19:19:28 INFO bsp.YARNBSPJob: Connecting to ResourceManager
at /0.0.0.0:8032
12/11/13 19:19:29 INFO bsp.YARNBSPJob: Starting job...
Exception in thread "main" java.lang.NullPointerException
at org.apache.hama.bsp.BSPJobClient.getClusterStatus(BSPJobClient.java:773)
at org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:307)
at org.apache.hama.bsp.YARNBSPJob.submit(YARNBSPJob.java:80)
at org.apache.hama.bsp.YARNBSPJob.waitForCompletion(YARNBSPJob.java:95)
at org.apache.hama.bsp.YarnSerializePrinting.main(YarnSerializePrinting.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
edward@udanax:~/workspace/hama-trunk$ vi
yarn/src/main/java/org/apache/hama/bsp/YarnSerializePrinting.java
edward@udanax:~/workspace/hama-trunk$ hadoop version
Hadoop 2.0.0-cdh4.1.2
Subversion file:///var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.1.2-Packaging-Hadoop-2012-11-01_17-01-07/hadoop-2.0.0+552-1.cdh4.1.2.p0.27~precise/src/hadoop-common-project/hadoop-common
-r f0b53c81cbf56f5955e403b49fcd27afd5f082de
Compiled by jenkins on Thu Nov 1 17:31:39 PDT 2012
>From source with checksum c5d56e606a3aa6dd5399cee3b2b8054f
{code}
> Hama YARN
> ---------
>
> Key: HAMA-670
> URL: https://issues.apache.org/jira/browse/HAMA-670
> Project: Hama
> Issue Type: Umbrella
> Components: bsp core, build , yarn
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
> Fix For: 0.7.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-670) Hama YARN
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-670:
--------------------------------
Fix Version/s: (was: 0.7.0)
0.6.1
> Hama YARN
> ---------
>
> Key: HAMA-670
> URL: https://issues.apache.org/jira/browse/HAMA-670
> Project: Hama
> Issue Type: Umbrella
> Components: bsp core, build , yarn
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
> Fix For: 0.6.1
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-670) Hama YARN
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507150#comment-13507150 ]
Edward J. Yoon commented on HAMA-670:
-------------------------------------
Let's do this soon and integrate Hama w/ bigtop TRUNK.
> Hama YARN
> ---------
>
> Key: HAMA-670
> URL: https://issues.apache.org/jira/browse/HAMA-670
> Project: Hama
> Issue Type: Umbrella
> Components: bsp core, build , yarn
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
> Fix For: 0.6.1
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-670) Hama YARN
Posted by "Thomas Jungblut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500424#comment-13500424 ]
Thomas Jungblut commented on HAMA-670:
--------------------------------------
Should be fixed in r1411326, it now defaults to 1 task if there is nothing mentioned or cluster status available.
Still I need a maven expert for HAMA-671 so we can start with a clean build system.
> Hama YARN
> ---------
>
> Key: HAMA-670
> URL: https://issues.apache.org/jira/browse/HAMA-670
> Project: Hama
> Issue Type: Umbrella
> Components: bsp core, build , yarn
> Affects Versions: 0.6.0
> Reporter: Thomas Jungblut
> Fix For: 0.7.0
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira