You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Markus Jelsma (JIRA)" <ji...@apache.org> on 2011/06/22 15:12:47 UTC
[jira] [Commented] (NUTCH-1006) meta equiv with single quotes not
accepted
[ https://issues.apache.org/jira/browse/NUTCH-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13053238#comment-13053238 ]
Markus Jelsma commented on NUTCH-1006:
--------------------------------------
Objections? Commit?
> meta equiv with single quotes not accepted
> ------------------------------------------
>
> Key: NUTCH-1006
> URL: https://issues.apache.org/jira/browse/NUTCH-1006
> Project: Nutch
> Issue Type: Bug
> Components: parser
> Affects Versions: 1.2, 1.3, 1.4, 2.0
> Reporter: Markus Jelsma
> Fix For: 1.4, 2.0
>
> Attachments: NUTCH-1006-104.patch, NUTCH-1006-2.0.patch
>
>
> As posted by Alex F:
> the regex metaPattern inside org.apache.nutch.parse.html.HtmlParser is not
> suitable for sites using single quotes for <meta http-equiv....>
> Example: <meta http-equiv='Content-Type' content='text/html;
> charset=iso-8859-1'>
> We experienced a couple of pages with that kind of quotes and Nutch-1.2
> was not able to handle it.
> Is there any fallback or would it be good to use the following
> regex: "<meta\\s+([^>]*http-equiv=(\"|')?content-type(\"|')?[^>]*)>" (single
> or regular quotes are accepted)?
> See this thread:
> http://lucene.472066.n3.nabble.com/Character-encoding-on-Html-Pages-td3034850.html
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira