You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by steven shingler <sh...@gmail.com> on 2006/08/15 18:05:05 UTC
Any plans to move to build Nutchusing Maven?
Hi all,
I know this has come up at least once before, but I just thought I'd raise
the question again:
Are there any plans to move to building Nutch using Maven?
Has anyone done this?
All thoughts gratefully received.
Thanks,
Steven
Re: Any plans to move to build Nutchusing Maven?
Posted by Jukka Zitting <ju...@gmail.com>.
Hi,
On 8/16/06, Lukas Vlcek <lu...@gmail.com> wrote:
> I have just noticed that there is going on some activity called Tika
> (http://code.google.com/p/tika/) and these guys are starting directly
> with Maven2 (am I right?).
The Maven 2 choise for the Tika staging ground was purely my personal
choise. I have no problems migrating to a different build system if
needed.
That said, I don't think an Ant build could ever come close to the
descriptive power of the Maven POM like the one in
http://tika.googlecode.com/svn/trunk/pom.xml. Combined with sticking
with the Maven conventions that gives me an instant build environment,
IDE integration, automatic site generation, and all sorts of nice
extra features like automatic DOAP file generation for
projects.apache.org.
BR,
Jukka Zitting
--
Yukatan - http://yukatan.fi/ - info@yukatan.fi
Software craftsmanship, JCR consulting, and Java development
Re: Any plans to move to build Nutchusing Maven?
Posted by Lukas Vlcek <lu...@gmail.com>.
Hi,
I have just noticed that there is going on some activity called Tika
(http://code.google.com/p/tika/) and these guys are starting directly
with Maven2 (am I right?).
The more Nutch/Lucene/Hadoop/[Tika]/[?] thing grows the more
sophisticated project management tool will be needed I think.
Lukas
On 8/16/06, Nicolas Lalevée <ni...@anyware-tech.com> wrote:
> Le Mercredi 16 Août 2006 17:18, Sami Siren a écrit:
> > Lukas Vlcek wrote:
> > > Hi,
> > >
> > > I have almost no experience with maven subprojects but somehow I feel
> > > this could help us with Nutch plugins. Am I correct?
> > > In maven we can always call ant goals as well and Jelly is a fun to
> > > use. With maven one of the biggest benefit would be that eclipse (or
> > > other IDE) classpath settings hell will be over which could help naive
> > > users (like me) a lot.
> > >
> > > (and silently I hope that what I have said now holds for M2 too)
> >
> > Just for clarification when I was talking about maven I meant maven 2,
> > maven 1 is a dead end.
> Hi,
>
> I am an outsider too, but my experience with maven 2 was not as awful as
> expected. The problem with maven is that you would depend enormously of the
> repository service. If it's down, just hope that you have download everything
> before, otherwise you have to wait until it's up again. Two or three month
> ago, the main repositories were down, and even more, one was corrupted
> (because of a disk failure or something like that, I don't remember). So our
> team of 5 people lost 3 days...
> The ASF Cocoon project has mooved to maven and now the README explains that :
> > Build times will vary dramatically depending on how good your connectivity
> > to the Maven repository/mirror is, and how well it performs, and on whether
> > you already have a loaded local Maven repository. A full build can take
> > from 5 minutes to maybe 4 hours...
> http://svn.apache.org/viewvc/cocoon/trunk/README.txt?view=markup
>
> Now, for our ex-maven2ish project, we use a combinaison of ant tasks and ivy
> for the dependencies (http://jayasoft.org/ivy). The difference between maven
> and ivy is that ivy runs completely offline, and maven download only on
> demand. So if you do a svn up on your project, you probably immediately want
> to do a mvn install to download every dependencies. But you also should run
> every maven task before getting offline : maven jetty6:run, maven source:jar,
> maven deploy...
>
> As I said, I am a completely outsider, I am not a Nutch user, just a Lucene
> user that read sometimes this dev list to read some interesting discussion
> about "search". I just wanted to add my unlucky maven user experience.
>
> Here are some notes about Sylvain, one of my coworker that had some trouble
> with maven :
> http://bluxte.net/blog/2006-05/21-25-04.html
> http://bluxte.net/blog/2006-06/30-32-27.html
>
> Nicolas
>
Re: Any plans to move to build Nutchusing Maven?
Posted by Nicolas Lalevée <ni...@anyware-tech.com>.
Le Mercredi 16 Août 2006 17:18, Sami Siren a écrit :
> Lukas Vlcek wrote:
> > Hi,
> >
> > I have almost no experience with maven subprojects but somehow I feel
> > this could help us with Nutch plugins. Am I correct?
> > In maven we can always call ant goals as well and Jelly is a fun to
> > use. With maven one of the biggest benefit would be that eclipse (or
> > other IDE) classpath settings hell will be over which could help naive
> > users (like me) a lot.
> >
> > (and silently I hope that what I have said now holds for M2 too)
>
> Just for clarification when I was talking about maven I meant maven 2,
> maven 1 is a dead end.
Hi,
I am an outsider too, but my experience with maven 2 was not as awful as
expected. The problem with maven is that you would depend enormously of the
repository service. If it's down, just hope that you have download everything
before, otherwise you have to wait until it's up again. Two or three month
ago, the main repositories were down, and even more, one was corrupted
(because of a disk failure or something like that, I don't remember). So our
team of 5 people lost 3 days...
The ASF Cocoon project has mooved to maven and now the README explains that :
> Build times will vary dramatically depending on how good your connectivity
> to the Maven repository/mirror is, and how well it performs, and on whether
> you already have a loaded local Maven repository. A full build can take
> from 5 minutes to maybe 4 hours...
http://svn.apache.org/viewvc/cocoon/trunk/README.txt?view=markup
Now, for our ex-maven2ish project, we use a combinaison of ant tasks and ivy
for the dependencies (http://jayasoft.org/ivy). The difference between maven
and ivy is that ivy runs completely offline, and maven download only on
demand. So if you do a svn up on your project, you probably immediately want
to do a mvn install to download every dependencies. But you also should run
every maven task before getting offline : maven jetty6:run, maven source:jar,
maven deploy...
As I said, I am a completely outsider, I am not a Nutch user, just a Lucene
user that read sometimes this dev list to read some interesting discussion
about "search". I just wanted to add my unlucky maven user experience.
Here are some notes about Sylvain, one of my coworker that had some trouble
with maven :
http://bluxte.net/blog/2006-05/21-25-04.html
http://bluxte.net/blog/2006-06/30-32-27.html
Nicolas
Re: Any plans to move to build Nutchusing Maven?
Posted by Sami Siren <ss...@gmail.com>.
Lukas Vlcek wrote:
> Hi,
>
> I have almost no experience with maven subprojects but somehow I feel
> this could help us with Nutch plugins. Am I correct?
> In maven we can always call ant goals as well and Jelly is a fun to
> use. With maven one of the biggest benefit would be that eclipse (or
> other IDE) classpath settings hell will be over which could help naive
> users (like me) a lot.
>
> (and silently I hope that what I have said now holds for M2 too)
>
Just for clarification when I was talking about maven I meant maven 2,
maven 1 is a dead end.
--
Sami Siren
Re: Any plans to move to build Nutchusing Maven?
Posted by Lukas Vlcek <lu...@gmail.com>.
Hi,
I have almost no experience with maven subprojects but somehow I feel
this could help us with Nutch plugins. Am I correct?
In maven we can always call ant goals as well and Jelly is a fun to
use. With maven one of the biggest benefit would be that eclipse (or
other IDE) classpath settings hell will be over which could help naive
users (like me) a lot.
(and silently I hope that what I have said now holds for M2 too)
Regards,
Lukas
On 8/16/06, William Surowiec <ws...@gmail.com> wrote:
> I am an outsider, I hope my comments do not cause grief.
>
> AFAIK, Maven is transitioning to version 2. The newer version is more
> attractive, but noticeably different than version 1 in implementation
> but, closer to home, in configuration.
>
> It might be wise to explore some of the comments from other, open
> sourced, projects regarding maven (about a month ago I was researching a
> maven problem and bumped into some negative comments that were unexpected.)
>
> This is a big step. Due diligence before commitment might be beneficial.
>
> With all those negatives, I want maven, especially 2, to be successful.
> I believe it will solve many tedious problems.
>
> Bill
>
Re: Any plans to move to build Nutchusing Maven?
Posted by William Surowiec <ws...@gmail.com>.
I am an outsider, I hope my comments do not cause grief.
AFAIK, Maven is transitioning to version 2. The newer version is more
attractive, but noticeably different than version 1 in implementation
but, closer to home, in configuration.
It might be wise to explore some of the comments from other, open
sourced, projects regarding maven (about a month ago I was researching a
maven problem and bumped into some negative comments that were unexpected.)
This is a big step. Due diligence before commitment might be beneficial.
With all those negatives, I want maven, especially 2, to be successful.
I believe it will solve many tedious problems.
Bill
Re: Any plans to move to build Nutchusing Maven?
Posted by Chris Mattmann <ch...@jpl.nasa.gov>.
Hi Steven,
On 8/16/06 7:36 AM, "steven shingler" <sh...@gmail.com> wrote:
> (This thread moved from the User List.)
>
> OK Lukas, lets open it up to the dev list! :)
>
> Particularly, does the group feel moving to Maven would be _a good thing_ ?
+1
I suggested this (however did not make any progress on realizing it ;) ) a
while back. I think it makes a * lot of sense *. Maven's dependency system
would significantly reduce the size of the CM'ed Nutch source code, as all
the jars required by Nutch could be referenced externally (plugins are a
different beast, but we're working on that). Additionally, Maven would allow
automatic generation of a soft of "nightly build" Nutch site, showing recent
commits, unit test results and more.
>
> Even if so, what are the problems?
The main problem I see is the plugin system, and how to appropriate
represent plugin dependencies in Maven (or just neglect to elegantly handle
them, and treat them like invididual projects, like nutch, which requires
CM'ing jar files). Additionally, I think it will probably require writing
some custom Jelly scripts to do all the neat ant build stuff that Nutch does
on the side (e.g., unpack Hadoop, etc.).
>
> There are currently two versions of Lucene in the Maven repos, but Hadoop
> would have to be added manually, I think.
It would probably make most sense to run a Maven repo explicitly for Nutch
off of the Lucene Nutch site. Something like
(http://lucene.apache.org/nutch/maven/) might be sensible.
Just my 2 cents.
Cheers,
Chris
>
> All thoughts gratefully received.
> Cheers
> Steven
>
> On 8/16/06, Lukas Vlcek <lu...@gmail.com> wrote:
>>
>> Hi,
>>
>> I would like to help. But first of all I would suggest to start wider
>> discussion in dev list to get more feedback/suggestions. I think one
>> problem can be that Nutch depends on both Lucene and Hadoop libraries
>> and it won't be easy to maintain these dependencies if recent versions
>> are not yet committed into some maven accesible repo.
>>
>> Regards,
>> Lukas
>>
>> On 8/16/06, steven shingler <sh...@gmail.com> wrote:
>>> Well I'm up for giving it a try. My current work has me looking at both
>>> Nutch and Maven, so what better way to understand both projects :)
>>>
>>> I agree it is far from trivial - so if anyone here would like to
>> collaborate
>>> on it, that would be great.
>>> Cheers,
>>> Steven
>>>
>>>
>>> On 8/15/06, Lukas Vlcek <lu...@gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I would warmly appreciate this activity. At least it would help more
>>>> people to understand/join this great project. But I don't think this
>>>> will be an easy step (this reminds me what N.Armstrong said on moon:
>>>> That's one small step for [a] man, one giant leap for mankind.)
>>>> :-)
>>>>
>>>> Regards,
>>>> Lukas
>>>>
>>>> On 8/15/06, Sami Siren <ss...@gmail.com> wrote:
>>>>> steven shingler wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I know this has come up at least once before, but I just thought
>> I'd
>>>> raise
>>>>>> the question again:
>>>>>>
>>>>>> Are there any plans to move to building Nutch using Maven?
>>>>>
>>>>> Haven't heard of such activities, however if you or somebody else
>>>>> can put such thing together and it proves to be a good thing to do
>> then
>>>>> I certainly don't have anything against it.
>>>>>
>>>>> --
>>>>> Sami Siren
>>>>>
>>>>
>>>
>>>
>>
Re: Any plans to move to build Nutchusing Maven?
Posted by og...@yahoo.com.
Old issue. I don't think there were any conclusions. Not sure if Maven2 would be a good thing, because I haven't used Maven in 2+ years, and I understand it's changed a lot since then.
The best way to move anything forward is to contribute the solution/fix/patch and then persuade others to give it a try and hope they like it. When that happens, your contribution typically goes in.
Otis
----- Original Message ----
From: steven shingler <sh...@gmail.com>
To: nutch-dev@lucene.apache.org
Sent: Wednesday, August 16, 2006 10:36:17 AM
Subject: Re: Any plans to move to build Nutchusing Maven?
(This thread moved from the User List.)
OK Lukas, lets open it up to the dev list! :)
Particularly, does the group feel moving to Maven would be _a good thing_ ?
Even if so, what are the problems?
There are currently two versions of Lucene in the Maven repos, but Hadoop
would have to be added manually, I think.
All thoughts gratefully received.
Cheers
Steven
On 8/16/06, Lukas Vlcek <lu...@gmail.com> wrote:
>
> Hi,
>
> I would like to help. But first of all I would suggest to start wider
> discussion in dev list to get more feedback/suggestions. I think one
> problem can be that Nutch depends on both Lucene and Hadoop libraries
> and it won't be easy to maintain these dependencies if recent versions
> are not yet committed into some maven accesible repo.
>
> Regards,
> Lukas
>
> On 8/16/06, steven shingler <sh...@gmail.com> wrote:
> > Well I'm up for giving it a try. My current work has me looking at both
> > Nutch and Maven, so what better way to understand both projects :)
> >
> > I agree it is far from trivial - so if anyone here would like to
> collaborate
> > on it, that would be great.
> > Cheers,
> > Steven
> >
> >
> > On 8/15/06, Lukas Vlcek <lu...@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I would warmly appreciate this activity. At least it would help more
> > > people to understand/join this great project. But I don't think this
> > > will be an easy step (this reminds me what N.Armstrong said on moon:
> > > That's one small step for [a] man, one giant leap for mankind.)
> > > :-)
> > >
> > > Regards,
> > > Lukas
> > >
> > > On 8/15/06, Sami Siren <ss...@gmail.com> wrote:
> > > > steven shingler wrote:
> > > > > Hi all,
> > > > >
> > > > > I know this has come up at least once before, but I just thought
> I'd
> > > raise
> > > > > the question again:
> > > > >
> > > > > Are there any plans to move to building Nutch using Maven?
> > > >
> > > > Haven't heard of such activities, however if you or somebody else
> > > > can put such thing together and it proves to be a good thing to do
> then
> > > > I certainly don't have anything against it.
> > > >
> > > > --
> > > > Sami Siren
> > > >
> > >
> >
> >
>
Re: Any plans to move to build Nutchusing Maven?
Posted by steven shingler <sh...@gmail.com>.
(This thread moved from the User List.)
OK Lukas, lets open it up to the dev list! :)
Particularly, does the group feel moving to Maven would be _a good thing_ ?
Even if so, what are the problems?
There are currently two versions of Lucene in the Maven repos, but Hadoop
would have to be added manually, I think.
All thoughts gratefully received.
Cheers
Steven
On 8/16/06, Lukas Vlcek <lu...@gmail.com> wrote:
>
> Hi,
>
> I would like to help. But first of all I would suggest to start wider
> discussion in dev list to get more feedback/suggestions. I think one
> problem can be that Nutch depends on both Lucene and Hadoop libraries
> and it won't be easy to maintain these dependencies if recent versions
> are not yet committed into some maven accesible repo.
>
> Regards,
> Lukas
>
> On 8/16/06, steven shingler <sh...@gmail.com> wrote:
> > Well I'm up for giving it a try. My current work has me looking at both
> > Nutch and Maven, so what better way to understand both projects :)
> >
> > I agree it is far from trivial - so if anyone here would like to
> collaborate
> > on it, that would be great.
> > Cheers,
> > Steven
> >
> >
> > On 8/15/06, Lukas Vlcek <lu...@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > I would warmly appreciate this activity. At least it would help more
> > > people to understand/join this great project. But I don't think this
> > > will be an easy step (this reminds me what N.Armstrong said on moon:
> > > That's one small step for [a] man, one giant leap for mankind.)
> > > :-)
> > >
> > > Regards,
> > > Lukas
> > >
> > > On 8/15/06, Sami Siren <ss...@gmail.com> wrote:
> > > > steven shingler wrote:
> > > > > Hi all,
> > > > >
> > > > > I know this has come up at least once before, but I just thought
> I'd
> > > raise
> > > > > the question again:
> > > > >
> > > > > Are there any plans to move to building Nutch using Maven?
> > > >
> > > > Haven't heard of such activities, however if you or somebody else
> > > > can put such thing together and it proves to be a good thing to do
> then
> > > > I certainly don't have anything against it.
> > > >
> > > > --
> > > > Sami Siren
> > > >
> > >
> >
> >
>
Re: Any plans to move to build Nutchusing Maven?
Posted by Lukas Vlcek <lu...@gmail.com>.
Hi,
I would like to help. But first of all I would suggest to start wider
discussion in dev list to get more feedback/suggestions. I think one
problem can be that Nutch depends on both Lucene and Hadoop libraries
and it won't be easy to maintain these dependencies if recent versions
are not yet committed into some maven accesible repo.
Regards,
Lukas
On 8/16/06, steven shingler <sh...@gmail.com> wrote:
> Well I'm up for giving it a try. My current work has me looking at both
> Nutch and Maven, so what better way to understand both projects :)
>
> I agree it is far from trivial - so if anyone here would like to collaborate
> on it, that would be great.
> Cheers,
> Steven
>
>
> On 8/15/06, Lukas Vlcek <lu...@gmail.com> wrote:
> >
> > Hi,
> >
> > I would warmly appreciate this activity. At least it would help more
> > people to understand/join this great project. But I don't think this
> > will be an easy step (this reminds me what N.Armstrong said on moon:
> > That's one small step for [a] man, one giant leap for mankind.)
> > :-)
> >
> > Regards,
> > Lukas
> >
> > On 8/15/06, Sami Siren <ss...@gmail.com> wrote:
> > > steven shingler wrote:
> > > > Hi all,
> > > >
> > > > I know this has come up at least once before, but I just thought I'd
> > raise
> > > > the question again:
> > > >
> > > > Are there any plans to move to building Nutch using Maven?
> > >
> > > Haven't heard of such activities, however if you or somebody else
> > > can put such thing together and it proves to be a good thing to do then
> > > I certainly don't have anything against it.
> > >
> > > --
> > > Sami Siren
> > >
> >
>
>
Re: Any plans to move to build Nutchusing Maven?
Posted by steven shingler <sh...@gmail.com>.
Well I'm up for giving it a try. My current work has me looking at both
Nutch and Maven, so what better way to understand both projects :)
I agree it is far from trivial - so if anyone here would like to collaborate
on it, that would be great.
Cheers,
Steven
On 8/15/06, Lukas Vlcek <lu...@gmail.com> wrote:
>
> Hi,
>
> I would warmly appreciate this activity. At least it would help more
> people to understand/join this great project. But I don't think this
> will be an easy step (this reminds me what N.Armstrong said on moon:
> That's one small step for [a] man, one giant leap for mankind.)
> :-)
>
> Regards,
> Lukas
>
> On 8/15/06, Sami Siren <ss...@gmail.com> wrote:
> > steven shingler wrote:
> > > Hi all,
> > >
> > > I know this has come up at least once before, but I just thought I'd
> raise
> > > the question again:
> > >
> > > Are there any plans to move to building Nutch using Maven?
> >
> > Haven't heard of such activities, however if you or somebody else
> > can put such thing together and it proves to be a good thing to do then
> > I certainly don't have anything against it.
> >
> > --
> > Sami Siren
> >
>
Re: Any plans to move to build Nutchusing Maven?
Posted by Lukas Vlcek <lu...@gmail.com>.
Hi,
I would warmly appreciate this activity. At least it would help more
people to understand/join this great project. But I don't think this
will be an easy step (this reminds me what N.Armstrong said on moon:
That's one small step for [a] man, one giant leap for mankind.)
:-)
Regards,
Lukas
On 8/15/06, Sami Siren <ss...@gmail.com> wrote:
> steven shingler wrote:
> > Hi all,
> >
> > I know this has come up at least once before, but I just thought I'd raise
> > the question again:
> >
> > Are there any plans to move to building Nutch using Maven?
>
> Haven't heard of such activities, however if you or somebody else
> can put such thing together and it proves to be a good thing to do then
> I certainly don't have anything against it.
>
> --
> Sami Siren
>
Re: Any plans to move to build Nutchusing Maven?
Posted by Sami Siren <ss...@gmail.com>.
steven shingler wrote:
> Hi all,
>
> I know this has come up at least once before, but I just thought I'd raise
> the question again:
>
> Are there any plans to move to building Nutch using Maven?
Haven't heard of such activities, however if you or somebody else
can put such thing together and it proves to be a good thing to do then
I certainly don't have anything against it.
--
Sami Siren