You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by Mohammad Islam <mi...@yahoo.com> on 2013/11/01 02:48:52 UTC
Re: Review Request 14575: Giraph Application Master: move to new and stable
YARN API
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14575/
-----------------------------------------------------------
(Updated Nov. 1, 2013, 1:48 a.m.)
Review request for giraph.
Changes
-------
Updated with review comments.
Tested with this two commands:
1. mvn clean verify
2. mvn -Phadoop_yarn -Dhadoop.version=2.2.0 clean verify
Bugs: GIRAPH-737
https://issues.apache.org/jira/browse/GIRAPH-737
Repository: giraph-git
Description
-------
WIP patch. Please give your early comments.
Key points:
* Giraph AM using new API and asynchronous/handler model.
* Adding Kerberos support.
Copied from the JIRA:
Giraph was the early adopter of Hadoop YARN AM! Eli successfully wrote a Giraph AM based on Hadoop 2.0.x_alpha. However, in last few months, Yarn significantly overhauled its APIs and associated coding patterns. The new beta version is 2.1.x and I was told by Yarn-dev that current APIs will not change much.
In the above circumstances, we need to substantially overhaul Giraph AM as well to accommodate with the new Yarn API. Moreover, in newer YARN API, supporting kerberos security in AM becomes easier and more transparent.
Potential impact:
The upcoming Girpah AM will not work with earlier alpha Hadoop versions such as 2.0.3. I'm not sure if anyone is using Giraph AM in production. However, the more prevalent way of Giraph processing (MR-based) should continue to work.
Diffs (updated)
-----
README fd9ec20
findbugs-exclude.xml 21aa4ef
giraph-core/src/main/java/org/apache/giraph/comm/netty/SaslNettyServer.java 00a802f
giraph-core/src/main/java/org/apache/giraph/comm/netty/handler/SaslServerHandler.java 922f373
giraph-core/src/main/java/org/apache/giraph/yarn/GiraphApplicationMaster.java c2b88a0
giraph-core/src/main/java/org/apache/giraph/yarn/GiraphYarnClient.java 341db0e
giraph-core/src/main/java/org/apache/giraph/yarn/YarnUtils.java aa042e8
pom.xml f2981ff
Diff: https://reviews.apache.org/r/14575/diff/
Testing
-------
Thanks,
Mohammad Islam