[jira] [Commented] (TIKA-3361) Improve intelligence of OCRStrategy=AUTO

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TIKA-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17324303#comment-17324303 ] 

ASF GitHub Bot commented on TIKA-3361:
--------------------------------------

peterkronenberg opened a new pull request #428:
URL: https://github.com/apache/tika/pull/428


   Thanks for your contribution to [Apache Tika](https://tika.apache.org/)! Your help is appreciated!
   
   Before opening the pull request, please verify that
   * there is an open issue on the [Tika issue tracker](https://issues.apache.org/jira/projects/TIKA) which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
   * the issue ID (`TIKA-XXXX`)
     - is referenced in the title of the pull request
     - and placed in front of your commit messages surrounded by square brackets (`[TIKA-XXXX] Issue or pull request title`)
   * commits are squashed into a single one (or few commits for larger changes)
   * Tika is successfully built and unit tests pass by running `mvn clean test`
   * there should be no conflicts when merging the pull request branch into the *recent* `main` branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled `main` branch.
   
   We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the [Tika mailing list](http://tika.apache.org/mail-lists.html). Thanks!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


>  Improve intelligence of OCRStrategy=AUTO
> -----------------------------------------
>
>                 Key: TIKA-3361
>                 URL: https://issues.apache.org/jira/browse/TIKA-3361
>             Project: Tika
>          Issue Type: Improvement
>            Reporter: Peter Kronenberg
>            Priority: Major
>
> Didn’t get a whole lot of feedback on the mailing list, so here’s my attempt at improving OCRStrategy=Auto
> Currently, this strategy performs the following test
> {code:java}
> if (totalCharsPerPage < 10 || unmappedUnicodeCharsPerPage > 10) {
>                     doOCROnCurrentPage(AUTO);
>                 }
> {code}
> I added a way to change the new numbers involved: the threshold for the total characters per page (below which, we OCR the page), and the threshold for unmapped characters (above which we OCR the page)
> My main concern is with the unmapped characters. OCR adds a lot of overhead, which might not be necessary for simply a few unmapped characters
> I added a new config, *OCRStrategyAuto*, which is only used if OCRStrategy=AUTO. Its format is
> {code:java}
> ocrStrategyAuto = best|fast|m[%], n
> {code}
> ‘best’ and ‘fast’ are shortcuts. More later
> m, n – m is the threshold for the number of unmapped characters per page. It can also be specified as a percentage. So, m=20 means if your page has more than 20 unmapped characters, it will OCR. m=20% means if the unmapped characters are more than 20% of the total characters, then it will OCR.
> n is the threshold for the total number of characters on the page. n does not need to be specified and defaults to 10
> {code:java}
> <param name="ocrStrategyAuto" type="string">20</param>
> {code}
> is equivalent to
> {code:java}
> <param name="ocrStrategyAuto" type="string">20, 10</param>
> {code}
> *best* is shorthand for *20,10*
> {code:java}
> <param name="ocrStrategyAuto" type="string">best</param>
> {code}
> is equivalent to
> {code:java}
> <param name="ocrStrategyAuto" type="string">20, 10</param>
> {code}
> *best* is the default and is equivalent to the current behavior
>  *fast* is a shortcut for *10%, 10*, which will avoid OCR unless the number of unmapped characters is greater than 10%
> {code:java}
> <param name="ocrStrategyAuto" type="string">fast</param>
> {code}
> is equivalent to
> {code:java}
> <param name="ocrStrategyAuto" type="string">10%, 10</param>
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)