You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Eugene Koontz (JIRA)" <ji...@apache.org> on 2013/01/20 00:56:12 UTC
[jira] [Commented] (GIRAPH-485) Hadoop Shims, smart detect version
on job start
[ https://issues.apache.org/jira/browse/GIRAPH-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558128#comment-13558128 ]
Eugene Koontz commented on GIRAPH-485:
--------------------------------------
Sounds interesting; let's look into it.
> Hadoop Shims, smart detect version on job start
> -----------------------------------------------
>
> Key: GIRAPH-485
> URL: https://issues.apache.org/jira/browse/GIRAPH-485
> Project: Giraph
> Issue Type: Wish
> Reporter: Nitay Joffe
>
> We should be able to (in theory at least, haven't looked deeply into it yet) use hive style shims to get rid of our munge whatnots and compile against all of the hadoops at once. This would mean having a common interface and a module for each hadoop we support. Then, when the user launches a giraph job we can detect the hadoop we're running in and load the appropriate module.
> This will require some exploration and research to figure out.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (GIRAPH-485) Hadoop Shims, smart detect
version on job start
Posted by Eli Reisman <ap...@gmail.com>.
This is a great idea. It will require some serious contemplation to get
something good. I am working on series of patches moving towards a "pure
YARN" implementation. If I don't go the route of a maven profile and/or
munge flags then this will be essential. It would be great to get
everyone's ideas on this as I can try to follow a path that does not
conflict with the ideas presented here. I'll definitely be keeping up on
this discussion as it moves along.
On Sat, Jan 19, 2013 at 3:56 PM, Eugene Koontz (JIRA) <ji...@apache.org>wrote:
>
> [
> https://issues.apache.org/jira/browse/GIRAPH-485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558128#comment-13558128]
>
> Eugene Koontz commented on GIRAPH-485:
> --------------------------------------
>
> Sounds interesting; let's look into it.
>
> > Hadoop Shims, smart detect version on job start
> > -----------------------------------------------
> >
> > Key: GIRAPH-485
> > URL: https://issues.apache.org/jira/browse/GIRAPH-485
> > Project: Giraph
> > Issue Type: Wish
> > Reporter: Nitay Joffe
> >
> > We should be able to (in theory at least, haven't looked deeply into it
> yet) use hive style shims to get rid of our munge whatnots and compile
> against all of the hadoops at once. This would mean having a common
> interface and a module for each hadoop we support. Then, when the user
> launches a giraph job we can detect the hadoop we're running in and load
> the appropriate module.
> > This will require some exploration and research to figure out.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>