You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Julien Nioche (JIRA)" <ji...@apache.org> on 2011/01/05 11:25:47 UTC
[jira] Created: (NUTCH-951) Backport changes from 2.0 into 1.3
Backport changes from 2.0 into 1.3
----------------------------------
Key: NUTCH-951
URL: https://issues.apache.org/jira/browse/NUTCH-951
Project: Nutch
Issue Type: Task
Affects Versions: 1.3
Reporter: Julien Nioche
Priority: Blocker
Fix For: 1.3
I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
* NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
* NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
* NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
* NUTCH-851 Port logging to slf4j (jnioche)
* NUTCH-861 Renamed HTMLParseFilter into ParseFilter
* NUTCH-872 Change the default fetcher.parse to FALSE (ab).
* NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
* NUTCH-880 REST API for Nutch (ab)
* NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
* NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
* NUTCH-886 A .gitignore file for Nutch (dogacan)
* NUTCH-894 Move statistical language identification from indexing to parsing step
* NUTCH-921 Reduce dependency of Nutch on config files (ab)
* NUTCH-930 Remove remaining dependencies on Lucene API (ab)
* NUTCH-931 Simple admin API to fetch status and stop the service (ab)
* NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
* NUTCH-936 LanguageIdentifier should not set empty lang field on NutchDocument (Markus Jelsma via jnioche)
Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Mattmann, Chris A (388J) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977854#action_12977854 ]
Mattmann, Chris A (388J) commented on NUTCH-951:
------------------------------------------------
Hey Julien,
I'll take care of the ones with my name on them below (NUTCH-564 and NUTCH-825).
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004488#comment-13004488 ]
Andrzej Bialecki commented on NUTCH-951:
-----------------------------------------
* Ported NUTCH-872 in rev. 1079746.
* Ported NUTCH-876 in rev. 1079753.
* Ported NUTCH-921 in rev. 1079760.
* NUTCH-884 is not applicable to 1.3 because here fetching executes in map tasks, so there's a correct number of them already.
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Assigned: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche reassigned NUTCH-951:
-----------------------------------
Assignee: Andrzej Bialecki
Cheekily assigned the task to AB as a friendly reminder :-) This is valid for Chris as well.
Gentlemen, any chance we could get that out of the way? If you are far too busy I could try and find the time to do it myself
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Closed: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by Andrzej Bialecki <ab...@getopt.org>.
On 3/10/11 10:57 PM, Julien Nioche (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Julien Nioche closed NUTCH-951.
> -------------------------------
>
>
> NUTCH-825 committed in revision 1080368
> All the known improvements from 2.0 have been backported into 1.3 now
The only remaining issue to address before rolling out a 1.3 release is
NUTCH-914 Implement Apache Project Branding Requirements (and subtasks...)
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
[jira] Closed: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche closed NUTCH-951.
-------------------------------
NUTCH-825 committed in revision 1080368
All the known improvements from 2.0 have been backported into 1.3 now
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche updated NUTCH-951:
--------------------------------
Description:
I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
* NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
* NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
* NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
* NUTCH-851 Port logging to slf4j (jnioche)
* NUTCH-861 Renamed HTMLParseFilter into ParseFilter
* NUTCH-872 Change the default fetcher.parse to FALSE (ab).
* NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
* NUTCH-880 REST API for Nutch (ab)
* NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
* NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
* NUTCH-886 A .gitignore file for Nutch (dogacan)
* NUTCH-894 Move statistical language identification from indexing to parsing step
* NUTCH-921 Reduce dependency of Nutch on config files (ab)
* NUTCH-930 Remove remaining dependencies on Lucene API (ab)
* NUTCH-931 Simple admin API to fetch status and stop the service (ab)
* NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
Let's go through this and decide what to port to 1.3
was:
I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
* NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
* NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
* NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
* NUTCH-851 Port logging to slf4j (jnioche)
* NUTCH-861 Renamed HTMLParseFilter into ParseFilter
* NUTCH-872 Change the default fetcher.parse to FALSE (ab).
* NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
* NUTCH-880 REST API for Nutch (ab)
* NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
* NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
* NUTCH-886 A .gitignore file for Nutch (dogacan)
* NUTCH-894 Move statistical language identification from indexing to parsing step
* NUTCH-921 Reduce dependency of Nutch on config files (ab)
* NUTCH-930 Remove remaining dependencies on Lucene API (ab)
* NUTCH-931 Simple admin API to fetch status and stop the service (ab)
* NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
* NUTCH-936 LanguageIdentifier should not set empty lang field on NutchDocument (Markus Jelsma via jnioche)
Let's go through this and decide what to port to 1.3
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12988724#comment-12988724 ]
Julien Nioche commented on NUTCH-951:
-------------------------------------
NUTCH-564 backported in revision 1065552
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Resolved: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrzej Bialecki resolved NUTCH-951.
-------------------------------------
Resolution: Fixed
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004490#comment-13004490 ]
Andrzej Bialecki commented on NUTCH-951:
-----------------------------------------
All changes have been ported. Thanks everyone!
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13004626#comment-13004626 ]
Julien Nioche commented on NUTCH-951:
-------------------------------------
OK, just NUTCH-825 to port and we're done
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977819#action_12977819 ]
Julien Nioche commented on NUTCH-951:
-------------------------------------
ported NUTCH-883 to 1.3 in rev 1055503
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977824#action_12977824 ]
Julien Nioche commented on NUTCH-951:
-------------------------------------
ported NUTCH-886 to 1.3 in rev 1055512
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>.
Hey Julien,
I'll take care of the ones with my name on them below (NUTCH-564 and NUTCH-825).
Cheers,
Chris
On Jan 5, 2011, at 8:36 AM, Julien Nioche (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977828#action_12977828 ]
>
> Julien Nioche commented on NUTCH-951:
> -------------------------------------
>
> NUTCH-894 : has been written for 2.0 and would need some effort to backport to 1.3
> I suggest that we leave it there.
>
> The list of things that IMHO are worth porting to 1.3 are now
>
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
>
> Any volunteers?
>
>
>> Backport changes from 2.0 into 1.3
>> ----------------------------------
>>
>> Key: NUTCH-951
>> URL: https://issues.apache.org/jira/browse/NUTCH-951
>> Project: Nutch
>> Issue Type: Task
>> Affects Versions: 1.3
>> Reporter: Julien Nioche
>> Priority: Blocker
>> Fix For: 1.3
>>
>>
>> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
>> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
>> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
>> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
>> * NUTCH-851 Port logging to slf4j (jnioche)
>> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
>> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
>> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
>> * NUTCH-880 REST API for Nutch (ab)
>> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
>> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
>> * NUTCH-886 A .gitignore file for Nutch (dogacan)
>> * NUTCH-894 Move statistical language identification from indexing to parsing step
>> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
>> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
>> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
>> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
>> Let's go through this and decide what to port to 1.3
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977828#action_12977828 ]
Julien Nioche commented on NUTCH-951:
-------------------------------------
NUTCH-894 : has been written for 2.0 and would need some effort to backport to 1.3
I suggest that we leave it there.
The list of things that IMHO are worth porting to 1.3 are now
* NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
* NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
* NUTCH-872 Change the default fetcher.parse to FALSE (ab).
* NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
* NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
* NUTCH-921 Reduce dependency of Nutch on config files (ab)
Any volunteers?
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-951) Backport changes from 2.0 into 1.3
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12977823#action_12977823 ]
Julien Nioche commented on NUTCH-951:
-------------------------------------
NUTCH-930 had already been done. added to CHANGES.txt in revision 1055509
> Backport changes from 2.0 into 1.3
> ----------------------------------
>
> Key: NUTCH-951
> URL: https://issues.apache.org/jira/browse/NUTCH-951
> Project: Nutch
> Issue Type: Task
> Affects Versions: 1.3
> Reporter: Julien Nioche
> Priority: Blocker
> Fix For: 1.3
>
>
> I've compared the changes from 2.0 with 1.3 and found the following differences (excluding anything specific to 2.0/GORA)
> * NUTCH-564 External parser supports encoding attribute (Antony Bowesman, mattmann)
> * NUTCH-714 Need a SFTP and SCP Protocol Handler (Sanjoy Ghosh, mattmann)
> * NUTCH-825 Publish nutch artifacts to central maven repository (mattmann)
> * NUTCH-851 Port logging to slf4j (jnioche)
> * NUTCH-861 Renamed HTMLParseFilter into ParseFilter
> * NUTCH-872 Change the default fetcher.parse to FALSE (ab).
> * NUTCH-876 Remove remaining robots/IP blocking code in lib-http (ab)
> * NUTCH-880 REST API for Nutch (ab)
> * NUTCH-883 Remove unused parameters from nutch-default.xml (jnioche)
> * NUTCH-884 FetcherJob should run more reduce tasks than default (ab)
> * NUTCH-886 A .gitignore file for Nutch (dogacan)
> * NUTCH-894 Move statistical language identification from indexing to parsing step
> * NUTCH-921 Reduce dependency of Nutch on config files (ab)
> * NUTCH-930 Remove remaining dependencies on Lucene API (ab)
> * NUTCH-931 Simple admin API to fetch status and stop the service (ab)
> * NUTCH-932 Bulk REST API to retrieve crawl results as JSON (ab)
> Let's go through this and decide what to port to 1.3
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.