You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "stack@archive.org (JIRA)" <ji...@apache.org> on 2007/01/04 21:12:28 UTC
[jira] Created: (NUTCH-426) parse-js skips parsing if found URL
fails java.net.URL parse
parse-js skips parsing if found URL fails java.net.URL parse
------------------------------------------------------------
Key: NUTCH-426
URL: https://issues.apache.org/jira/browse/NUTCH-426
Project: Nutch
Issue Type: Bug
Components: fetcher
Affects Versions: 0.9.0
Reporter: stack@archive.org
Priority: Minor
The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string of javascript. Any matches that do not begin 'www' are given to java.net.URL with base URL to test 'URLness'. Often this test will fail. Currently, when it fails, nutch skips processing any more of the javascript snippet logs an error.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (NUTCH-426) parse-js skips parsing if found URL
fails java.net.URL parse
Posted by "stack@archive.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462307 ]
stack@archive.org commented on NUTCH-426:
-----------------------------------------
Just attached a patch that catches the MalformedURLException, logs the failure at trace level, and then continues to pickup on any more matches found in remaining javascript.
> parse-js skips parsing if found URL fails java.net.URL parse
> ------------------------------------------------------------
>
> Key: NUTCH-426
> URL: https://issues.apache.org/jira/browse/NUTCH-426
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.9.0
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: nutch426.patch
>
>
> The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string of javascript. Any matches that do not begin 'www' are given to java.net.URL with base URL to test 'URLness'. Often this test will fail. Currently, when it fails, nutch skips processing any more of the javascript snippet logs an error.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Closed: (NUTCH-426) parse-js skips parsing if found URL
fails java.net.URL parse
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrzej Bialecki closed NUTCH-426.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.9.0
Assignee: Andrzej Bialecki
Fixed in rev. 493085. Thank you!
> parse-js skips parsing if found URL fails java.net.URL parse
> ------------------------------------------------------------
>
> Key: NUTCH-426
> URL: https://issues.apache.org/jira/browse/NUTCH-426
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.9.0
> Reporter: stack@archive.org
> Assigned To: Andrzej Bialecki
> Priority: Minor
> Fix For: 0.9.0
>
> Attachments: nutch426.patch
>
>
> The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string of javascript. Any matches that do not begin 'www' are given to java.net.URL with base URL to test 'URLness'. Often this test will fail. Currently, when it fails, nutch skips processing any more of the javascript snippet logs an error.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (NUTCH-426) parse-js skips parsing if found URL
fails java.net.URL parse
Posted by "stack@archive.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack@archive.org updated NUTCH-426:
------------------------------------
Attachment: nutch426.patch
> parse-js skips parsing if found URL fails java.net.URL parse
> ------------------------------------------------------------
>
> Key: NUTCH-426
> URL: https://issues.apache.org/jira/browse/NUTCH-426
> Project: Nutch
> Issue Type: Bug
> Components: fetcher
> Affects Versions: 0.9.0
> Reporter: stack@archive.org
> Priority: Minor
> Attachments: nutch426.patch
>
>
> The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string of javascript. Any matches that do not begin 'www' are given to java.net.URL with base URL to test 'URLness'. Often this test will fail. Currently, when it fails, nutch skips processing any more of the javascript snippet logs an error.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira