You are viewing a plain text version of this content. The canonical link for it is here.
Posted to droids-dev@incubator.apache.org by "Mingfai Ma (JIRA)" <ji...@apache.org> on 2009/07/17 10:09:14 UTC
[jira] Created: (DROIDS-57) Create a Fetcher for crawler to
replace/wrap the use of Protocol+Worker
Create a Fetcher for crawler to replace/wrap the use of Protocol+Worker
-----------------------------------------------------------------------
Key: DROIDS-57
URL: https://issues.apache.org/jira/browse/DROIDS-57
Project: Droids
Issue Type: New Feature
Affects Versions: 0.01
Reporter: Mingfai Ma
- the main argument is, "fetcher" is a common terminology in crawling
- a fetcher seems to be only required by droids-crawler but not in core.
- this task is more about the creation of an interface and a concept. fetching function exists already in HttpProtocol.
the fetcher shall have a "fetch()" method. I propose the following 2 main interface. let it be evolved over time
public Fetcher<T> fetch(T link) throws FetcherException;
public Fetcher<T> fetch(T link, Map params) throws FetcherException;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DROIDS-57) Create a Fetcher for crawler to
replace/wrap the use of Protocol+Worker
Posted by "Thorsten Scherler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DROIDS-57?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12754115#action_12754115 ]
Thorsten Scherler commented on DROIDS-57:
-----------------------------------------
More information can be found around this topic in
http://mail-archives.apache.org/mod_mbox/incubator-droids-dev/200906.mbox/<bc...@mail.gmail.com>
> Create a Fetcher for crawler to replace/wrap the use of Protocol+Worker
> -----------------------------------------------------------------------
>
> Key: DROIDS-57
> URL: https://issues.apache.org/jira/browse/DROIDS-57
> Project: Droids
> Issue Type: New Feature
> Affects Versions: 0.01
> Reporter: Mingfai Ma
>
> - the main argument is, "fetcher" is a common terminology in crawling
> - a fetcher seems to be only required by droids-crawler but not in core.
> - this task is more about the creation of an interface and a concept. fetching function exists already in HttpProtocol.
> the fetcher shall have a "fetch()" method. I propose the following 2 main interface. let it be evolved over time
> public Fetcher<T> fetch(T link) throws FetcherException;
> public Fetcher<T> fetch(T link, Map params) throws FetcherException;
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.