You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Edward Drapkin (Commented) (JIRA)" <ji...@apache.org> on 2012/01/17 19:50:39 UTC
[jira] [Commented] (NUTCH-1201) Allow for different FetcherThread
impls
[ https://issues.apache.org/jira/browse/NUTCH-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187874#comment-13187874 ]
Edward Drapkin commented on NUTCH-1201:
---------------------------------------
Does this still need to be done? It seems pretty easy and I'll volunteer to do it if it needs to be done.
I was thinking of breaking all of Fetcher apart into more easily compartmentalized and pluggable units for my own benefit (as right now it's an enormous class that's extremely daunting and hard to change). If this issue still needs work, I think I can break Fetcher apart to allow for pluggable fetchers, item queues, queue feeders and fetcher threads, but I don't want to invest time into reinventing a wheel that you've already invented (and may not have updated JIRA).
Let me know!
> Allow for different FetcherThread impls
> ---------------------------------------
>
> Key: NUTCH-1201
> URL: https://issues.apache.org/jira/browse/NUTCH-1201
> Project: Nutch
> Issue Type: New Feature
> Components: fetcher
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Fix For: 1.5
>
>
> For certain cases we need to modify parts in FetcherThread and make it pluggable. This introduces a new config directive fetcher.impl that takes a FQCN and uses that setting Fetcher.fetch to load a class to use for job.setMapRunnerClass(). This new class has to extend Fetcher and and inner class FetcherThread. This allows for overriding methods in FetcherThread but also methods in Fetcher itself if required.
> A follow up on this issue would be to refactor parts of FetcherThread to make it easier to override small sections instead of copying the entire method body for a small change, which is now the case.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira