[jira] [Updated] (TIKA-1599) Switch from TagSoup to JSoup

["Tim Allison (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TIKA-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Allison updated TIKA-1599:
------------------------------
    Attachment: tagsoup_vs_jsoup_reports.zip

I ran a comparison on our 80k html docs.  It looks like we gain about 5% in common English words if we move to JSoup at the cost of 4 new exceptions.

I'm currently getting far fewer metadata items with JSoup, but that's almost certain to be the fault of my initial implementation.

If anyone has a chance to look through contents/content_diffs.xlsx, I'd appreciate all feedback.

> Switch from TagSoup to JSoup
> ----------------------------
>
>                 Key: TIKA-1599
>                 URL: https://issues.apache.org/jira/browse/TIKA-1599
>             Project: Tika
>          Issue Type: Improvement
>          Components: parser
>    Affects Versions: 1.7, 1.8
>            Reporter: Ken Krugler
>            Assignee: Ken Krugler
>            Priority: Minor
>         Attachments: tagsoup_vs_jsoup_reports.zip
>
>
> There are several Tika issues related to how TagSoup cleans up HTML ([TIKA-381], [TIKA-985], maybe [TIKA-715]), but TagSoup doesn't seem to be under active development.
> On the other hand I know of several projects that are now using [JSoup|https://github.com/jhy/jsoup], which is an active project (albeit only one main contributor) under the MIT license.
> I haven't looked into how hard it would be to switch this dependency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)