[jira] [Created] (TIKA-2244) excessive memory usage when parsing a large nested package file

["Joshua Hight (JIRA)" <ji...@apache.org>]

Joshua Hight created TIKA-2244:
----------------------------------

             Summary: excessive memory usage when parsing a large nested package file
                 Key: TIKA-2244
                 URL: https://issues.apache.org/jira/browse/TIKA-2244
             Project: Tika
          Issue Type: Bug
          Components: core, parser
    Affects Versions: 2.0
            Reporter: Joshua Hight
            Priority: Minor


When parsing large nested files(a couple good examples are maven jars and git objects), a large number of BufferedInputStreams get generated taking up large amounts of memory with their buffers. Upon looking through the relevant code I saw that many of these allocations were coming from TikaInputStream.get(InputStream, TemporaryResources)
which checks if the InputStream is a BufferedInputStream or ByteArrayInputStream in order to determine whether on not mark is supported. Unfortunately it is common practice to wrap InputStreams in CloseShieldInputStreams, causing it to fail even if mark is in fact supported.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)