You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Markus Jelsma <ma...@openindex.io> on 2011/09/27 23:01:11 UTC
Prepare for 1.4 release?
Hi,
There are some bad issues in 1.3 that are fixed in early 1.4 revisions. Also,
1.4 has some nice improvements and new features. I know some would like to
include more advanced changes in the 1.4 release. But i don't see why we can
already ship a proper 1.4 right now and include difficult changes (of which
some are not yet even in progress or buggy) in a later release.
Nutch is going faster since 1.2 and therefore i think we should release more
often.
I, hereby, propose to open a vote for preparing a 1.4 release, creating a 1.5
version in Jira to which we move open issues and fix the last small changes
and blockers and push it out in the next few weeks.
Thanks,
Re: 1.4 release - newer hadoop jars
Posted by Sebastian Nagel <wa...@googlemail.com>.
> can you package 1.4 with updated hadoop jars? i have problems with running nutch in local mode. If i
> run multiple tasks at once, they delete each other temporary files. Its worth a try if newer hadoop
> libs will fix that.
Hi Radim,
I don't know whether current versions of hadoop fix this problem.
It's a bitter experience made by many Nutch users that running tasks
simultaneously in local mode may fail accidentally from time to time,
see http://www.mail-archive.com/user@nutch.apache.org/msg03239.html
Set hadoop.tmp.dir for each job so that it points to an unique directory.
Instead of setting it in the xml config file which is uncomfortable
you can set it via java command line options or the NUTCH_OPTS environment
variable:
NUTCH_OPTS=-Dhadoop.tmp.dir=/tmp/uniq_dir $NUTCH_HOME/bin/nutch ...
Sebastian
Re: 1.4 release - newer hadoop jars
Posted by Sebastian Nagel <wa...@googlemail.com>.
> can you package 1.4 with updated hadoop jars? i have problems with running nutch in local mode. If i
> run multiple tasks at once, they delete each other temporary files. Its worth a try if newer hadoop
> libs will fix that.
Hi Radim,
I don't know whether current versions of hadoop fix this problem.
It's a bitter experience made by many Nutch users that running tasks
simultaneously in local mode may fail accidentally from time to time,
see http://www.mail-archive.com/user@nutch.apache.org/msg03239.html
Set hadoop.tmp.dir for each job so that it points to an unique directory.
Instead of setting it in the xml config file which is uncomfortable
you can set it via java command line options or the NUTCH_OPTS environment
variable:
NUTCH_OPTS=-Dhadoop.tmp.dir=/tmp/uniq_dir $NUTCH_HOME/bin/nutch ...
Sebastian
1.4 release - newer hadoop jars
Posted by Radim Kolar <hs...@sendmail.cz>.
can you package 1.4 with updated hadoop jars? i have problems with
running nutch in local mode. If i run multiple tasks at once, they
delete each other temporary files. Its worth a try if newer hadoop libs
will fix that.
Re: Prepare for 1.4 release?
Posted by Julien Nioche <li...@gmail.com>.
BTW I have renamed the version '2.0' in JIRA into 'nutchgora' to reflect the
location of the code + have deleted the version '2.1' which was empty
On 29 September 2011 12:08, lewis john mcgibbney
<le...@gmail.com>wrote:
> Hi,
>
> On the JIRA I see 32 unresolved issues for 1.4... Is it possible for us to
> agree some kind of programme for establishing what we would like to be in
> the 1.4 release? I am keen to focus on something which we all have a common
> interest in progressing.
>
> Thanks
>
>
> On Wed, Sep 28, 2011 at 10:13 AM, lewis john mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> yes +1
>>
>> Thanks for bringing this up Markus.
>>
>> I would like to get NUTCH-1078 sorted out ASAP. However I'll comment on
>> that issue separately.
>>
>>
>> On Wed, Sep 28, 2011 at 9:46 AM, Julien Nioche <
>> lists.digitalpebble@gmail.com> wrote:
>>
>>> +1
>>>
>>> have created a 1.5 version in JIRA.
>>>
>>> Thanks
>>>
>>> Julien
>>>
>>>
>>> On 27 September 2011 22:01, Markus Jelsma <ma...@openindex.io>wrote:
>>>
>>>> Hi,
>>>>
>>>> There are some bad issues in 1.3 that are fixed in early 1.4 revisions.
>>>> Also,
>>>> 1.4 has some nice improvements and new features. I know some would like
>>>> to
>>>> include more advanced changes in the 1.4 release. But i don't see why we
>>>> can
>>>> already ship a proper 1.4 right now and include difficult changes (of
>>>> which
>>>> some are not yet even in progress or buggy) in a later release.
>>>>
>>>> Nutch is going faster since 1.2 and therefore i think we should release
>>>> more
>>>> often.
>>>>
>>>> I, hereby, propose to open a vote for preparing a 1.4 release, creating
>>>> a 1.5
>>>> version in Jira to which we move open issues and fix the last small
>>>> changes
>>>> and blockers and push it out in the next few weeks.
>>>>
>>>> Thanks,
>>>>
>>>
>>>
>>>
>>> --
>>> *
>>> *Open Source Solutions for Text Engineering
>>>
>>> http://digitalpebble.blogspot.com/
>>> http://www.digitalpebble.com
>>>
>>
>>
>>
>> --
>> *Lewis*
>>
>>
>
>
> --
> *Lewis*
>
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
Re: Prepare for 1.4 release?
Posted by lewis john mcgibbney <le...@gmail.com>.
Hi,
On the JIRA I see 32 unresolved issues for 1.4... Is it possible for us to
agree some kind of programme for establishing what we would like to be in
the 1.4 release? I am keen to focus on something which we all have a common
interest in progressing.
Thanks
On Wed, Sep 28, 2011 at 10:13 AM, lewis john mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> yes +1
>
> Thanks for bringing this up Markus.
>
> I would like to get NUTCH-1078 sorted out ASAP. However I'll comment on
> that issue separately.
>
>
> On Wed, Sep 28, 2011 at 9:46 AM, Julien Nioche <
> lists.digitalpebble@gmail.com> wrote:
>
>> +1
>>
>> have created a 1.5 version in JIRA.
>>
>> Thanks
>>
>> Julien
>>
>>
>> On 27 September 2011 22:01, Markus Jelsma <ma...@openindex.io>wrote:
>>
>>> Hi,
>>>
>>> There are some bad issues in 1.3 that are fixed in early 1.4 revisions.
>>> Also,
>>> 1.4 has some nice improvements and new features. I know some would like
>>> to
>>> include more advanced changes in the 1.4 release. But i don't see why we
>>> can
>>> already ship a proper 1.4 right now and include difficult changes (of
>>> which
>>> some are not yet even in progress or buggy) in a later release.
>>>
>>> Nutch is going faster since 1.2 and therefore i think we should release
>>> more
>>> often.
>>>
>>> I, hereby, propose to open a vote for preparing a 1.4 release, creating a
>>> 1.5
>>> version in Jira to which we move open issues and fix the last small
>>> changes
>>> and blockers and push it out in the next few weeks.
>>>
>>> Thanks,
>>>
>>
>>
>>
>> --
>> *
>> *Open Source Solutions for Text Engineering
>>
>> http://digitalpebble.blogspot.com/
>> http://www.digitalpebble.com
>>
>
>
>
> --
> *Lewis*
>
>
--
*Lewis*
Re: Prepare for 1.4 release?
Posted by lewis john mcgibbney <le...@gmail.com>.
yes +1
Thanks for bringing this up Markus.
I would like to get NUTCH-1078 sorted out ASAP. However I'll comment on that
issue separately.
On Wed, Sep 28, 2011 at 9:46 AM, Julien Nioche <
lists.digitalpebble@gmail.com> wrote:
> +1
>
> have created a 1.5 version in JIRA.
>
> Thanks
>
> Julien
>
>
> On 27 September 2011 22:01, Markus Jelsma <ma...@openindex.io>wrote:
>
>> Hi,
>>
>> There are some bad issues in 1.3 that are fixed in early 1.4 revisions.
>> Also,
>> 1.4 has some nice improvements and new features. I know some would like to
>> include more advanced changes in the 1.4 release. But i don't see why we
>> can
>> already ship a proper 1.4 right now and include difficult changes (of
>> which
>> some are not yet even in progress or buggy) in a later release.
>>
>> Nutch is going faster since 1.2 and therefore i think we should release
>> more
>> often.
>>
>> I, hereby, propose to open a vote for preparing a 1.4 release, creating a
>> 1.5
>> version in Jira to which we move open issues and fix the last small
>> changes
>> and blockers and push it out in the next few weeks.
>>
>> Thanks,
>>
>
>
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
>
--
*Lewis*
Re: Prepare for 1.4 release?
Posted by Julien Nioche <li...@gmail.com>.
+1
have created a 1.5 version in JIRA.
Thanks
Julien
On 27 September 2011 22:01, Markus Jelsma <ma...@openindex.io>wrote:
> Hi,
>
> There are some bad issues in 1.3 that are fixed in early 1.4 revisions.
> Also,
> 1.4 has some nice improvements and new features. I know some would like to
> include more advanced changes in the 1.4 release. But i don't see why we
> can
> already ship a proper 1.4 right now and include difficult changes (of which
> some are not yet even in progress or buggy) in a later release.
>
> Nutch is going faster since 1.2 and therefore i think we should release
> more
> often.
>
> I, hereby, propose to open a vote for preparing a 1.4 release, creating a
> 1.5
> version in Jira to which we move open issues and fix the last small changes
> and blockers and push it out in the next few weeks.
>
> Thanks,
>
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
Re: Prepare for 1.4 release?
Posted by Radim Kolar <hs...@sendmail.cz>.
can you add NUTCH-1098 to 1.4?
Re: Prepare for 1.4 release?
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Markus,
Sure I can take a pass at it.
Cheers,
Chris
On Sep 27, 2011, at 2:38 PM, Markus Jelsma wrote:
>> +1 from me.
>>
>> I'd be happy to RM it too, if that helps.
>
> Sure, but i'd appreciate updating the release howto wiki page as it's a bit
> outdated.
>
>>
>> Cheers,
>> Chris
>>
>> On Sep 27, 2011, at 2:01 PM, Markus Jelsma wrote:
>>> Hi,
>>>
>>> There are some bad issues in 1.3 that are fixed in early 1.4 revisions.
>>> Also, 1.4 has some nice improvements and new features. I know some would
>>> like to include more advanced changes in the 1.4 release. But i don't
>>> see why we can already ship a proper 1.4 right now and include difficult
>>> changes (of which some are not yet even in progress or buggy) in a later
>>> release.
>>>
>>> Nutch is going faster since 1.2 and therefore i think we should release
>>> more often.
>>>
>>> I, hereby, propose to open a vote for preparing a 1.4 release, creating a
>>> 1.5 version in Jira to which we move open issues and fix the last small
>>> changes and blockers and push it out in the next few weeks.
>>>
>>> Thanks,
>>
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW: http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Prepare for 1.4 release?
Posted by Markus Jelsma <ma...@openindex.io>.
> +1 from me.
>
> I'd be happy to RM it too, if that helps.
Sure, but i'd appreciate updating the release howto wiki page as it's a bit
outdated.
>
> Cheers,
> Chris
>
> On Sep 27, 2011, at 2:01 PM, Markus Jelsma wrote:
> > Hi,
> >
> > There are some bad issues in 1.3 that are fixed in early 1.4 revisions.
> > Also, 1.4 has some nice improvements and new features. I know some would
> > like to include more advanced changes in the 1.4 release. But i don't
> > see why we can already ship a proper 1.4 right now and include difficult
> > changes (of which some are not yet even in progress or buggy) in a later
> > release.
> >
> > Nutch is going faster since 1.2 and therefore i think we should release
> > more often.
> >
> > I, hereby, propose to open a vote for preparing a 1.4 release, creating a
> > 1.5 version in Jira to which we move open issues and fix the last small
> > changes and blockers and push it out in the next few weeks.
> >
> > Thanks,
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Re: Prepare for 1.4 release?
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
+1 from me.
I'd be happy to RM it too, if that helps.
Cheers,
Chris
On Sep 27, 2011, at 2:01 PM, Markus Jelsma wrote:
> Hi,
>
> There are some bad issues in 1.3 that are fixed in early 1.4 revisions. Also,
> 1.4 has some nice improvements and new features. I know some would like to
> include more advanced changes in the 1.4 release. But i don't see why we can
> already ship a proper 1.4 right now and include difficult changes (of which
> some are not yet even in progress or buggy) in a later release.
>
> Nutch is going faster since 1.2 and therefore i think we should release more
> often.
>
> I, hereby, propose to open a vote for preparing a 1.4 release, creating a 1.5
> version in Jira to which we move open issues and fix the last small changes
> and blockers and push it out in the next few weeks.
>
> Thanks,
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++