[jira] Updated: (NUTCH-54) Fetcher improvements

["Andrzej Bialecki (JIRA)" <ji...@apache.org>]

     [ http://issues.apache.org/jira/browse/NUTCH-54?page=all ]

Andrzej Bialecki  updated NUTCH-54:
-----------------------------------

    Attachment: parsestatus.patch

* HTML "meta" tags processor has been extended, so that it collects and processes all meta tags. Convenience methods have been added to handle refresh meta tag, so that Fetcher can support multiple redirects.

* the interaction between content parsers and their users has been changed, from exception-driven to status-driven. This gives a much better control over the logic flow, and enables us to communicate more information than just a plaintext message. I plan to make similar changes for protocol handlers, which should greatly simplify the logic in Fetcher.

* preliminary changes to Fetcher to support automatic redirection loop, if parsers report a "refresh" meta directive.

* scaffolding to support parsing complete pages (i.e. pages fetched together with all their elements, such as JavaScript and CSS).

Any comments and suggestions are welcome!

> Fetcher  improvements
> ---------------------
>
>          Key: NUTCH-54
>          URL: http://issues.apache.org/jira/browse/NUTCH-54
>      Project: Nutch
>         Type: Improvement
>   Components: fetcher
>     Reporter: Andrzej Bialecki 
>     Assignee: Andrzej Bialecki 
>  Attachments: parsestatus.patch
>
> Fetcher improvements.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira