potential memory issue in FileBackedDataSource (TIKA-2058)

["Allison, Timothy B." <ta...@mitre.org>]

All,
  On TIKA-2058, Tim Barrett reported some OOM problems and posted an hprof of the issue.  Luis Filipe Nassif analyzed the hprof and identified POI's FileBackedDataSource as a potential source of the problem.
  We haven't yet determined if this is a single-triggering file type of OOM or a static-cache type of OOM.  If any fellow devs have time to chime in for diagnostics/ideas on potential causes, that'd be great.

  Thank you!

                Cheers,

                    Tim

[1] https://issues.apache.org/jira/browse/TIKA-2058

Re: potential memory issue in FileBackedDataSource (TIKA-2058)

[Nick Burch <ap...@gagravarr.org>]

On Mon, 12 Sep 2016, Allison, Timothy B. wrote:
> On TIKA-2058, Tim Barrett reported some OOM problems and posted an hprof 
> of the issue.  Luis Filipe Nassif analyzed the hprof and identified 
> POI's FileBackedDataSource as a potential source of the problem.

It could be. FileBackedDataSource does NIO mmap operations via a 
FileChannel. Sadly Sun forgot to finish that API of properly, and there's 
no explicit unmap, it happens "magically" at some later date once the file 
is closed:
http://stackoverflow.com/questions/2972986/how-to-unmap-a-file-from-memory-mapped-using-filechannel-in-java

Maybe try with the unsupported sun.misc cleaner to see if that frees the 
memory?

Nick

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@poi.apache.org
For additional commands, e-mail: dev-help@poi.apache.org