You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tika.apache.org by "Hudson (Jira)" <ji...@apache.org> on 2020/11/26 20:29:00 UTC

[jira] [Commented] (TIKA-3237) Great optimization in ForkParser

    [ https://issues.apache.org/jira/browse/TIKA-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17239411#comment-17239411 ] 

Hudson commented on TIKA-3237:
------------------------------

UNSTABLE: Integrated in Jenkins build Tika » tika-branch1x-jdk8 #36 (See [https://ci-builds.apache.org/job/Tika/job/tika-branch1x-jdk8/36/])
TIKA-3237: great optimization in ForkParser (lfcnassif: [https://github.com/apache/tika/commit/0207bd0053b6824fda168d8a0a66282b7199951f])
* (edit) tika-core/src/test/java/org/apache/tika/fork/ForkParserTest.java
* (edit) tika-core/src/main/java/org/apache/tika/fork/ContentHandlerProxy.java
* (edit) CHANGES.txt
* (edit) tika-core/src/main/java/org/apache/tika/fork/ContentHandlerResource.java


> Great optimization in ForkParser
> --------------------------------
>
>                 Key: TIKA-3237
>                 URL: https://issues.apache.org/jira/browse/TIKA-3237
>             Project: Tika
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.24.1
>         Environment: Windows 10, Oracle JDK 1.8.0_261 x64, 24 cores
>            Reporter: Luís Filipe Nassif
>            Assignee: Luís Filipe Nassif
>            Priority: Major
>             Fix For: 2.0
>
>
> There is a huge overhead in ForkParser ContentHandlerProxy and ContentHandlerResource read/write char[]/string methods. A simple change to not loop reading/writing each char of strings can result in speed ups of several orders of magnitude.
>  
> Tests with ~10k small RFC822 files (22MB), 24 parsing threads, have shown parse times below:
> ForkParser off: 4s
> ForkParser on: 30min
> ForkParser patched: 17s
>  
> Also, there is a bug when running at least with Oracle JDK8 that, if a String greater than 65535 chars is written, a UTFDataFormatException is thrown from DataOutputStream.writeUTF(String) method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)