You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Joep Rottinghuis (JIRA)" <ji...@apache.org> on 2011/02/15 00:01:57 UTC
[jira] Commented: (PIG-1855) Split hadoop-20.jar into separate
components
[ https://issues.apache.org/jira/browse/PIG-1855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994545#comment-12994545 ]
Joep Rottinghuis commented on PIG-1855:
---------------------------------------
Better would be to either point to a URL where your own Hadoop jar is located (can be public maven URL, or your own).
Remaining issue to solve is to have all supportings jars slapped together in one other jar, or refer to them individually?
Alternatively one can point to an entire hadoop-????.tar.gz and have the build unpack the entire Hadoop and compile against it.
I have not yet tried the best approach. For example, would one re-package the entire thing into a single jar, or just compile against the loose pieces.
Not sure (yet) how the decision would impact the final classpath when running pig.
> Split hadoop-20.jar into separate components
> --------------------------------------------
>
> Key: PIG-1855
> URL: https://issues.apache.org/jira/browse/PIG-1855
> Project: Pig
> Issue Type: Improvement
> Reporter: Joep Rottinghuis
>
> Currently Pig compiles against a pre-packaged, source controlled set of versions of Hadoop:
> lib/hadoop18.jar and lib/hadoop20.jar.
> While this is convenient for Pig development as a standalone activity it makes it a pain to be able to use:
> 1) The hadoop??.jar contains classes that do not belong to the o.a.h namespace. In other words it contains com.sun.*, org.apache.jasper and all sorts of other auxiliary classes used by hadoop.
> 2) When building Pig against your own version of Hadoop one has to untar the hadoop20.jar, splice in the o.a.h classes, re-pack (and commit to source control?)
> One would expect the ability to be able to point Pig build to output from own Hadoop build.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira