You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@tika.apache.org by "Luís Filipe Nassif (Jira)" <ji...@apache.org> on 2020/11/26 18:23:00 UTC

[jira] [Created] (TIKA-3237) Great optimization in ForkParser

Luís Filipe Nassif created TIKA-3237:
----------------------------------------

             Summary: Great optimization in ForkParser
                 Key: TIKA-3237
                 URL: https://issues.apache.org/jira/browse/TIKA-3237
             Project: Tika
          Issue Type: Improvement
          Components: core
    Affects Versions: 1.24.1
         Environment: Windows 10, Oracle JDK 1.8.0_261 x64, 24 cores
            Reporter: Luís Filipe Nassif
            Assignee: Luís Filipe Nassif


There is a huge overhead in ForkParser ContentHandlerProxy and ContentHandlerResource read/write char[]/string methods. A simple change to not loop reading/writing each char of strings can result in speed ups of several orders of magnitude.

 

Tests with ~10k small RFC822 files (22MB), 24 parsing threads, have shown parse times below:

ForkParser off: 4s

ForkParser on: 30min

ForkParser patched: 17s

 

Also, there is a bug when running at least with Oracle JDK8 that, if a String greater than 65535 is written, a UTFDataFormatException is thrown from DataOutputStream.writeUTF(String) method.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)