You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "julien nioche (JIRA)" <ji...@apache.org> on 2008/10/31 17:31:44 UTC
[jira] Created: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Add Counter for # of doc fetched in Reporter
--------------------------------------------
Key: NUTCH-658
URL: https://issues.apache.org/jira/browse/NUTCH-658
Project: Nutch
Issue Type: Improvement
Components: fetcher
Reporter: julien nioche
Priority: Trivial
Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "julien nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
julien nioche updated NUTCH-658:
--------------------------------
Attachment: Fetcher.patch
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: julien nioche
> Priority: Trivial
> Attachments: Fetcher.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche closed NUTCH-658.
-------------------------------
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: Julien Nioche
> Priority: Trivial
> Fix For: 1.1
>
> Attachments: Fetcher.patch, ReporterCounter.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "Doğacan Güney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651383#action_12651383 ]
Doğacan Güney commented on NUTCH-658:
-------------------------------------
This is a great idea. I think this should go in before 1.0.
But more counters would be nice. For fetch, gone, error, etc as you mentioned. For parse, number of urls succesfully parsed or failed. .....
Can you send an updated patch? If you don't have the time I can do it too.
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: julien nioche
> Priority: Trivial
> Attachments: Fetcher.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "julien nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
julien nioche updated NUTCH-658:
--------------------------------
Attachment: ReporterCounter.patch
Hi,
I eventually managed to make the change. The new patch (ReporterCounter) has been generated against the Revision 724410. Both implementations of the Fetchers now report statistics on all possible status, including the status for the parsing step. Stand alone parsing with ParseSegment does the same.
J.
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: julien nioche
> Priority: Trivial
> Attachments: Fetcher.patch, ReporterCounter.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796133#action_12796133 ]
Julien Nioche commented on NUTCH-658:
-------------------------------------
If no one objects I'll commit this one in the next couple of days. OK for everybody?
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: Julien Nioche
> Priority: Trivial
> Attachments: Fetcher.patch, ReporterCounter.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "Julien Nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Nioche resolved NUTCH-658.
---------------------------------
Resolution: Fixed
Fix Version/s: 1.1
Committed revision 895972
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: Julien Nioche
> Priority: Trivial
> Fix For: 1.1
>
> Attachments: Fetcher.patch, ReporterCounter.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (NUTCH-658) Add Counter for # of doc fetched in
Reporter
Posted by "julien nioche (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651406#action_12651406 ]
julien nioche commented on NUTCH-658:
-------------------------------------
Hi Dogacan,
I am off work for several weeks and won't be able to work on that. If you want to do it yourselves that would be great otherwise I'd be happy to do it in January.
> Add Counter for # of doc fetched in Reporter
> --------------------------------------------
>
> Key: NUTCH-658
> URL: https://issues.apache.org/jira/browse/NUTCH-658
> Project: Nutch
> Issue Type: Improvement
> Components: fetcher
> Reporter: julien nioche
> Priority: Trivial
> Attachments: Fetcher.patch
>
>
> Having a Counter for the number of documents fetched duplicates the information in the status of each Map but should be summed across all the Map instances and displayed along with the standard Counters. The same mechanism could be used for the other possible status of a URL (redir / gone etc...)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.