[jira] [Created] (UIMA-4281) Gradually shrink internal core arrays on reset actions

["Marshall Schor (JIRA)" <de...@uima.apache.org>]

Marshall Schor created UIMA-4281:
------------------------------------

             Summary: Gradually shrink internal core arrays on reset actions
                 Key: UIMA-4281
                 URL: https://issues.apache.org/jira/browse/UIMA-4281
             Project: UIMA
          Issue Type: Improvement
          Components: Core Java Framework
            Reporter: Marshall Schor
            Priority: Minor
             Fix For: 2.7.1SDK


Many internal core data structures composed of arrays (typically of ints) expand as needed while running.  When a reset event occurs (e.g., cas.reset()), many of these are cleared, but not "shrunk".  So the effect of running, say, one Document which is very large might cause many internal data structures to expand, and from then on consume potentially large but unused space.  

This isn't too bad an issue with modern OSs; except for movement in the heap, it would be expected that the pages containing the unused space would gradually be paged out.

Nevertheless, it would be good to gradually shrink these spaces back down after some peak.  The shrinkage should be gradual, to avoid needing to expand the arrays too often.  I think a good rule of thumb would be to remember one previous size, and to shrink only if both previous sizes were small enough to be contained in 1 size shrinkage.
This should reduce oscillating back and forth around a particular size, while recovering space over many "resets" from an early abnormal expansion.

I'm thinking that the 1 size shrinkage algorithm should run the expansion algorithm backwards - that is, in many of our data structures, expand by doubling up to some switch point, and then expand linearly by adding. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)