You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "Erik Hetzner (JIRA)" <ji...@apache.org> on 2009/11/18 02:48:39 UTC

[jira] Created: (TIKA-327) Parsing "HTML" as DcXML

Parsing "HTML" as DcXML
-----------------------

                 Key: TIKA-327
                 URL: https://issues.apache.org/jira/browse/TIKA-327
             Project: Tika
          Issue Type: Bug
    Affects Versions: 0.6
            Reporter: Erik Hetzner
            Priority: Minor
         Attachments: nheri.html

As of r881342, I encounter the following error when trying to parse the "HTML" from <http://www.nheri.org/> with the AutoDetectParser. (I will attach my example.) It works fine if I use the HtmlParser explicitly.

org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.xml.DcXMLParser@45d64c37
	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:130)
	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:101)
	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:114)
	at org.cdlib.was.tikaIndexer.parser$.main(Parser.scala:18)
	at org.cdlib.was.tikaIndexer.parser.main(Parser.scala)
Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1422)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
	at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
	at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
	at org.apache.tika.parser.xml.XMLParser.parse(XMLParser.java:72)
	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:120)



-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Resolved: (TIKA-327) Parsing "HTML" as DcXML

Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/TIKA-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris A. Mattmann resolved TIKA-327.
------------------------------------

    Resolution: Fixed

- addressed this issue by updating the tika-mimetypes.xml config to add namespaceless html and link tags as root-XML for the "text/html" mimetype. For completeness, we should probably add the rest of the weird HTML tags that you can encounter, but that would be a large enumeration. This fixes the issue for now. I've included your sample file and a test case for regression. Thanks for the pointer on this, Erik!

> Parsing "HTML" as DcXML
> -----------------------
>
>                 Key: TIKA-327
>                 URL: https://issues.apache.org/jira/browse/TIKA-327
>             Project: Tika
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.5
>            Reporter: Erik Hetzner
>            Assignee: Chris A. Mattmann
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: nheri.html
>
>
> As of r881342, I encounter the following error when trying to parse the "HTML" from <http://www.nheri.org/> with the AutoDetectParser. (I will attach my example.) It works fine if I use the HtmlParser explicitly.
> org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.xml.DcXMLParser@45d64c37
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:130)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:101)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:114)
> 	at org.cdlib.was.tikaIndexer.parser$.main(Parser.scala:18)
> 	at org.cdlib.was.tikaIndexer.parser.main(Parser.scala)
> Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
> 	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
> 	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1422)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> 	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
> 	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> 	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> 	at org.apache.tika.parser.xml.XMLParser.parse(XMLParser.java:72)
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:120)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (TIKA-327) Parsing "HTML" as DcXML

Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/TIKA-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783519#action_12783519 ] 

Jukka Zitting commented on TIKA-327:
------------------------------------

Hmm, that really is a bit tricky "HTML". I guess we could make the XML root element detector to fall back to HTML whenever it detects a tag like <head>, <body>, <link>, <title>, <meta>, or <p> with no declared namespace. That should solve this issue.

> Parsing "HTML" as DcXML
> -----------------------
>
>                 Key: TIKA-327
>                 URL: https://issues.apache.org/jira/browse/TIKA-327
>             Project: Tika
>          Issue Type: Bug
>    Affects Versions: 0.6
>            Reporter: Erik Hetzner
>            Priority: Minor
>         Attachments: nheri.html
>
>
> As of r881342, I encounter the following error when trying to parse the "HTML" from <http://www.nheri.org/> with the AutoDetectParser. (I will attach my example.) It works fine if I use the HtmlParser explicitly.
> org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.xml.DcXMLParser@45d64c37
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:130)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:101)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:114)
> 	at org.cdlib.was.tikaIndexer.parser$.main(Parser.scala:18)
> 	at org.cdlib.was.tikaIndexer.parser.main(Parser.scala)
> Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
> 	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
> 	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1422)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> 	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
> 	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> 	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> 	at org.apache.tika.parser.xml.XMLParser.parse(XMLParser.java:72)
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:120)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (TIKA-327) Parsing "HTML" as DcXML

Posted by "Chris A. Mattmann (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/TIKA-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chris A. Mattmann updated TIKA-327:
-----------------------------------

          Component/s: parser
    Affects Version/s:     (was: 0.6)
                       0.5
        Fix Version/s: 0.6
             Assignee: Chris A. Mattmann

> Parsing "HTML" as DcXML
> -----------------------
>
>                 Key: TIKA-327
>                 URL: https://issues.apache.org/jira/browse/TIKA-327
>             Project: Tika
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.5
>            Reporter: Erik Hetzner
>            Assignee: Chris A. Mattmann
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: nheri.html
>
>
> As of r881342, I encounter the following error when trying to parse the "HTML" from <http://www.nheri.org/> with the AutoDetectParser. (I will attach my example.) It works fine if I use the HtmlParser explicitly.
> org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.xml.DcXMLParser@45d64c37
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:130)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:101)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:114)
> 	at org.cdlib.was.tikaIndexer.parser$.main(Parser.scala:18)
> 	at org.cdlib.was.tikaIndexer.parser.main(Parser.scala)
> Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
> 	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
> 	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1422)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> 	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
> 	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> 	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> 	at org.apache.tika.parser.xml.XMLParser.parse(XMLParser.java:72)
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:120)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Commented: (TIKA-327) Parsing "HTML" as DcXML

Posted by "Erik Hetzner (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/TIKA-327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12801217#action_12801217 ] 

Erik Hetzner commented on TIKA-327:
-----------------------------------

Thank you! 

> Parsing "HTML" as DcXML
> -----------------------
>
>                 Key: TIKA-327
>                 URL: https://issues.apache.org/jira/browse/TIKA-327
>             Project: Tika
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 0.5
>            Reporter: Erik Hetzner
>            Assignee: Chris A. Mattmann
>            Priority: Minor
>             Fix For: 0.6
>
>         Attachments: nheri.html
>
>
> As of r881342, I encounter the following error when trying to parse the "HTML" from <http://www.nheri.org/> with the AutoDetectParser. (I will attach my example.) It works fine if I use the HtmlParser explicitly.
> org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.xml.DcXMLParser@45d64c37
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:130)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:101)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:114)
> 	at org.cdlib.was.tikaIndexer.parser$.main(Parser.scala:18)
> 	at org.cdlib.was.tikaIndexer.parser.main(Parser.scala)
> Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
> 	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
> 	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1422)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> 	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
> 	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> 	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> 	at org.apache.tika.parser.xml.XMLParser.parse(XMLParser.java:72)
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:120)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

[jira] Updated: (TIKA-327) Parsing "HTML" as DcXML

Posted by "Erik Hetzner (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/TIKA-327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erik Hetzner updated TIKA-327:
------------------------------

    Attachment: nheri.html

HTML file to trigger issue.

> Parsing "HTML" as DcXML
> -----------------------
>
>                 Key: TIKA-327
>                 URL: https://issues.apache.org/jira/browse/TIKA-327
>             Project: Tika
>          Issue Type: Bug
>    Affects Versions: 0.6
>            Reporter: Erik Hetzner
>            Priority: Minor
>         Attachments: nheri.html
>
>
> As of r881342, I encounter the following error when trying to parse the "HTML" from <http://www.nheri.org/> with the AutoDetectParser. (I will attach my example.) It works fine if I use the HtmlParser explicitly.
> org.apache.tika.exception.TikaException: TIKA-237: Illegal SAXException from org.apache.tika.parser.xml.DcXMLParser@45d64c37
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:130)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:101)
> 	at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:114)
> 	at org.cdlib.was.tikaIndexer.parser$.main(Parser.scala:18)
> 	at org.cdlib.was.tikaIndexer.parser.main(Parser.scala)
> Caused by: org.xml.sax.SAXParseException: The markup in the document following the root element must be well-formed.
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
> 	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
> 	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
> 	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$TrailingMiscDriver.next(XMLDocumentScannerImpl.java:1422)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
> 	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
> 	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
> 	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
> 	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
> 	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
> 	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
> 	at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
> 	at org.apache.tika.parser.xml.XMLParser.parse(XMLParser.java:72)
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:120)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.