[tika] 02/02: TIKA-2419 If we detect XML but the XML is broken, try the HTML magics before declaring it to be broken xml = plain text. Needed because, to avoid false positives on html-like formats such as email, XML has a higher magic priority than HTML

[ni...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

nick pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tika.git

commit 383015235d4fc855c16d8d65c0c3cae96488951d
Author: Nick Burch <ni...@gagravarr.org>
AuthorDate: Wed Jul 5 15:01:14 2017 +0100

    TIKA-2419 If we detect XML but the XML is broken, try the HTML magics before declaring it to be broken xml = plain text. Needed because, to avoid false positives on html-like formats such as email, XML has a higher magic priority than HTML
---
 .../main/java/org/apache/tika/mime/MimeTypes.java  | 30 +++++++++++++++++++---
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java b/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java
index 05d959f..46b9d8f 100644
--- a/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java
+++ b/tika-core/src/main/java/org/apache/tika/mime/MimeTypes.java
@@ -86,7 +86,12 @@ public final class MimeTypes implements Detector, Serializable {
      */
     private final MimeType textMimeType;
 
-    /*
+    /**
+     * html type, text/html
+     */
+    private final MimeType htmlMimeType;
+
+    /**
      * xml type, application/xml
      */
     private final MimeType xmlMimeType;
@@ -112,6 +117,7 @@ public final class MimeTypes implements Detector, Serializable {
     public MimeTypes() {
         rootMimeType = new MimeType(MediaType.OCTET_STREAM);
         textMimeType = new MimeType(MediaType.TEXT_PLAIN);
+        htmlMimeType = new MimeType(MediaType.TEXT_HTML);
         xmlMimeType = new MimeType(MediaType.APPLICATION_XML);
         
         rootMimeTypeL = Collections.singletonList(rootMimeType);
@@ -214,9 +220,25 @@ public final class MimeTypes implements Detector, Serializable {
                             }
                         }
                     } else if ("application/xml".equals(matched.getName())) {
-                        // Downgrade from application/xml to text/plain since
-                        // the document seems not to be well-formed.
-                        result.set(i, textMimeType);
+                        // Our XML magic is higher than our HTML magic
+                        // So, if we got here, we might have a HTML file that's
+                        //  invalid XML. So, try our HTML magics explicitly (TIKA-2419)
+                        boolean isHTML = false;
+                        for (Magic magic : magics) {
+                            if (! magic.getType().equals(htmlMimeType)) continue;
+                            if (magic.eval(data)) {
+                                isHTML = true;
+                                break;
+                            }
+                        }
+                        
+                        // Otherwise, downgrade from application/xml to text/plain
+                        //  since the document seems not to be well-formed.
+                        if (isHTML) {
+                            result.set(i, htmlMimeType);
+                        } else {
+                            result.set(i, textMimeType);
+                        }
                     }
                 }
             }

-- 
To stop receiving notification emails like this one, please contact
"commits@tika.apache.org" <co...@tika.apache.org>.