[jira] [Commented] (TIKA-4901) Don't inflate an embedded document twice to digest it

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TIKA-4901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116543#comment-18116543 ] 

ASF GitHub Bot commented on TIKA-4901:
--------------------------------------

tballison opened a new pull request, #3191:
URL: https://github.com/apache/tika/pull/3191

   <!--
     Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
   
       http://www.apache.org/licenses/LICENSE-2.0
   
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
     KIND, either express or implied.  See the License for the
     specific language governing permissions and limitations
     under the License.
   -->
   
   Thanks for your contribution to [Apache Tika](https://tika.apache.org/)! Your help is appreciated!
   
   Before opening the pull request, please verify that
   * there is an open issue on the [Tika issue tracker](https://issues.apache.org/jira/projects/TIKA) which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
   * the issue ID (`TIKA-XXXX`)
     - is referenced in the title of the pull request
     - and placed in front of your commit messages surrounded by square brackets (`[TIKA-XXXX] Issue or pull request title`)
   * commits are squashed into a single one (or few commits for larger changes)
   * Tika builds and unit tests pass with `./mvnw clean install` (`clean test` alone cannot resolve the pipes plugin zips)
   * if you used a generative AI tool: follow the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) (`Generated-by: <tool>` in the commit message), and consider running the pre-flight in `.skills/devs/pr-review/SKILL.md` — fix what it finds; don't paste its report here
   * there should be no conflicts when merging the pull request branch into the *recent* `main` branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled `main` branch
   * if you add new module that downstream users will depend upon add it to relevant group in `tika-bom/pom.xml`.
   
   We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the [Tika mailing list](http://tika.apache.org/mail-lists.html). Thanks!
   




> Don't inflate an embedded document twice to digest it
> -----------------------------------------------------
>
>                 Key: TIKA-4901
>                 URL: https://issues.apache.org/jira/browse/TIKA-4901
>             Project: Tika
>          Issue Type: Improvement
>            Reporter: Tim Allison
>            Priority: Minor
>
> :robot: description
> {noformat}
>    InputStreamDigester reads a stream to hash it, then rewinds so the parse can read it again. For a file that costs nothing — the page cache serves 
>   the second read at ~6 GB/s. For an embedded document it is not a file: every ReopenableSource is a re-openable supplier, and re-opening a zip entry
>    means inflating it again, measured at 480–670 MB/s. The page cache holds the compressed container, so it does not help.
>   
>    TikaInputSource gains an advisory tryRetainInMemory(), default false. ReopenableSource implements it by reusing its existing in-memory drain and 
>   budget reservation, refusing when the length is unknown, when the content exceeds the 1 MB floor with no budget, or when the budget cannot cover it
>    — so it never spills and never starts a drain that cannot finish. ensureOpen() and seekTo() now serve from the retained content when there is 
>    some. InputStreamDigester asks for retention after enableRewind; FileSource and CachingSource take the default and are unaffected.
>   
>    Full recursive parse of 120 zips × 16 × 128 KB entries with embedded digesting on, only tika-core swapped, 3 reps × 2 rounds: wall 3.875–4.081 s → 
>    3.047–3.141 s, CPU 4.59–5.64 s → 3.78–4.83 s. Output identical (digests and extracted lengths checksum the same in every run). The durable number 
>    is the absolute one — about 3.4 ms per MB of embedded content, one inflate — since the ratio depends on how expensive the embedded parser is; text 
>    entries make the inflate share look large.
>   
>   Limits: entries above the 1 MB floor need budget headroom or retention is refused, and zip is the best case — an OLE2 document stream or a PST item
>   re-opens more cheaply.
>   Also fixes a latent bug in the same method: ensureOpen() at a non-zero position re-read from byte 0 while continuing to report positions as if it 
>   had not.
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)