You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Iván de Prado <iv...@gmail.com> on 2008/02/18 14:18:51 UTC
Conflicts with embeded dependencies
I have seen that you are embedding the binary classes from several
projects into pig.jar. The idea is package all in a single jar to
simplify its use and reduce the size. I thinks is a good idea.
By the other hand, people that are developing their own LoadFunc and
other code based in Pig needs to include the pig jar in their projects.
But because pig.jar contains binaries from other OS projects, dependency
conflicts can appears (like conflicts with Jetty).
I have change the build.xml to be able to generate two jars:
- pig.jar (The current default)
- pig-core.jar
The pig-core.jar doesn't includes the dependencies into the jar, so
pig-core.jar can be included safely in people projects without generate
dependency conflicts.
I attach the new build.xml. It would be good if something like that is
added to future versions.
Congratulations for your nice project,
Iván de Prado
Re: Conflicts with embeded dependencies
Posted by Benjamin Reed <br...@yahoo-inc.com>.
I would think this should be broken into front-end and back-end. The point
being that if you are writing a UDF and you get a Jetty conflict at compile
time, it's probably a good thing to know because you will get it at runtime.
(I realize the example of Jetty in a UDF is a bit far fetched :)
I imagine that eventually Hadoop will get a real class manager and then this
will not be an issue. (I seem to remember an OSGi patch a while ago.) If you
are not careful, you will trade a compile time problem for a runtime problem.
ben
On Monday 18 February 2008 05:18:51 Iván de Prado wrote:
> I have seen that you are embedding the binary classes from several
> projects into pig.jar. The idea is package all in a single jar to
> simplify its use and reduce the size. I thinks is a good idea.
>
> By the other hand, people that are developing their own LoadFunc and
> other code based in Pig needs to include the pig jar in their projects.
> But because pig.jar contains binaries from other OS projects, dependency
> conflicts can appears (like conflicts with Jetty).
>
> I have change the build.xml to be able to generate two jars:
> - pig.jar (The current default)
> - pig-core.jar
>
> The pig-core.jar doesn't includes the dependencies into the jar, so
> pig-core.jar can be included safely in people projects without generate
> dependency conflicts.
>
> I attach the new build.xml. It would be good if something like that is
> added to future versions.
>
> Congratulations for your nice project,
> Iván de Prado
Re: Conflicts with embeded dependencies
Posted by Stefan Groschupf <sg...@101tec.com>.
On Feb 19, 2008, at 11:02 AM, Olga Natkovich wrote:
> How is this script going to be different from pig.pl?
No difference. I will patch the perl script.
Stefan
RE: Conflicts with embeded dependencies
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
How is this script going to be different from pig.pl?
Olga
> -----Original Message-----
> From: Stefan Groschupf [mailto:sg@101tec.com]
> Sent: Tuesday, February 19, 2008 2:26 AM
> To: pig-user@incubator.apache.org
> Subject: Re: Conflicts with embeded dependencies
>
> > seems like something that should be addressed. I've no
> opinion on if
> > it should be added to PIG-68 or be added as a new JIRA.
>
> I added this into my contribution to PIG-68
> (build.xml-PIG-68-v06- SG.patch). I suggest as next step, we
> should write a little start
> script that setup the class path with the required libraries and
> starts Grunt or pig.
> If people agree I can start working on that tomorrow morning.
>
>
>
Re: Conflicts with embeded dependencies
Posted by Stefan Groschupf <sg...@101tec.com>.
> seems like something that should be addressed. I've no opinion on
> if it should be added to PIG-68 or be added as a new JIRA.
I added this into my contribution to PIG-68 (build.xml-PIG-68-v06-
SG.patch). I suggest as next step, we should write a little start
script that setup the class path with the required libraries and
starts Grunt or pig.
If people agree I can start working on that tomorrow morning.
Re: Conflicts with embeded dependencies
Posted by Eric Baldeschwieler <er...@yahoo-inc.com>.
+1
seems like something that should be addressed. I've no opinion on if
it should be added to PIG-68 or be added as a new JIRA.
On Feb 18, 2008, at 8:20 AM, Benjamin Francisoud wrote:
> I can have a look at it and include it in PIG-68, if commiters are
> interested...
>
> --
> Benjamin Francisoud
>
> Iván de Prado a écrit :
>> I have seen that you are embedding the binary classes from several
>> projects into pig.jar. The idea is package all in a single jar to
>> simplify its use and reduce the size. I thinks is a good idea.
>> By the other hand, people that are developing their own LoadFunc and
>> other code based in Pig needs to include the pig jar in their
>> projects.
>> But because pig.jar contains binaries from other OS projects,
>> dependency
>> conflicts can appears (like conflicts with Jetty).
>> I have change the build.xml to be able to generate two jars:
>> - pig.jar (The current default)
>> - pig-core.jar
>>
>> The pig-core.jar doesn't includes the dependencies into the jar, so
>> pig-core.jar can be included safely in people projects without
>> generate
>> dependency conflicts.
>>
>> I attach the new build.xml. It would be good if something like
>> that is
>> added to future versions.
>>
>> Congratulations for your nice project,
>> Iván de Prado
>>
>>
>>
>
RE: Conflicts with embeded dependencies
Posted by Olga Natkovich <ol...@yahoo-inc.com>.
Lets open a separate JIRA on this and attach the diffs just for this change.
Thanks,
Olga
> -----Original Message-----
> From: Benjamin Francisoud [mailto:benjamin.francisoud@joost.com]
> Sent: Monday, February 18, 2008 8:20 AM
> To: pig-user@incubator.apache.org
> Subject: Re: Conflicts with embeded dependencies
>
> I can have a look at it and include it in PIG-68, if
> commiters are interested...
>
> --
> Benjamin Francisoud
>
> Iván de Prado a écrit :
> > I have seen that you are embedding the binary classes from several
> > projects into pig.jar. The idea is package all in a single jar to
> > simplify its use and reduce the size. I thinks is a good idea.
> >
> > By the other hand, people that are developing their own
> LoadFunc and
> > other code based in Pig needs to include the pig jar in
> their projects.
> > But because pig.jar contains binaries from other OS projects,
> > dependency conflicts can appears (like conflicts with Jetty).
> >
> > I have change the build.xml to be able to generate two jars:
> > - pig.jar (The current default)
> > - pig-core.jar
> >
> > The pig-core.jar doesn't includes the dependencies into the jar, so
> > pig-core.jar can be included safely in people projects without
> > generate dependency conflicts.
> >
> > I attach the new build.xml. It would be good if something
> like that is
> > added to future versions.
> >
> > Congratulations for your nice project, Iván de Prado
> >
> >
> >
>
>
Re: Conflicts with embeded dependencies
Posted by Benjamin Francisoud <be...@joost.com>.
I can have a look at it and include it in PIG-68, if commiters are
interested...
--
Benjamin Francisoud
Iván de Prado a écrit :
> I have seen that you are embedding the binary classes from several
> projects into pig.jar. The idea is package all in a single jar to
> simplify its use and reduce the size. I thinks is a good idea.
>
> By the other hand, people that are developing their own LoadFunc and
> other code based in Pig needs to include the pig jar in their projects.
> But because pig.jar contains binaries from other OS projects, dependency
> conflicts can appears (like conflicts with Jetty).
>
> I have change the build.xml to be able to generate two jars:
> - pig.jar (The current default)
> - pig-core.jar
>
> The pig-core.jar doesn't includes the dependencies into the jar, so
> pig-core.jar can be included safely in people projects without generate
> dependency conflicts.
>
> I attach the new build.xml. It would be good if something like that is
> added to future versions.
>
> Congratulations for your nice project,
> Iván de Prado
>
>
>