You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by "Dennis Bauer (JIRA)" <de...@uima.apache.org> on 2017/02/28 09:45:45 UTC
[jira] [Comment Edited] (UIMA-5306) Memory Improvement -
Unnecessary leaks
[ https://issues.apache.org/jira/browse/UIMA-5306?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887662#comment-15887662 ]
Dennis Bauer edited comment on UIMA-5306 at 2/28/17 9:45 AM:
-------------------------------------------------------------
We've evaluated your 4 versions, thanks for that. Your active one and the RutaBasic4 class didn't work in my setup and threw the following exception:
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index (4794300) is greater than or equal to list size (2)
at it.unimi.dsi.fastutil.ints.IntArrayList.removeInt(IntArrayList.java:346)
at org.apache.uima.ruta.type.RutaBasic.removeBegin(RutaBasic.java:296)
at org.apache.uima.ruta.RutaStream.removeAnnotation(RutaStream.java:344)
at org.apache.uima.ruta.RutaStream.removeAnnotation(RutaStream.java:328)
at org.apache.uima.ruta.action.UnmarkAllAction.execute(UnmarkAllAction.java:73)
at org.apache.uima.ruta.rule.AbstractRuleElement.apply(AbstractRuleElement.java:132)
at org.apache.uima.ruta.rule.RuleElementCaretaker.applyRuleElements(RuleElementCaretaker.java:111)
at org.apache.uima.ruta.rule.ComposedRuleElement.applyRuleElements(ComposedRuleElement.java:593)
at org.apache.uima.ruta.rule.AbstractRuleElement.doneMatching(AbstractRuleElement.java:84)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallback(ComposedRuleElement.java:514)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallbackContinue(ComposedRuleElement.java:415)
at org.apache.uima.ruta.rule.RutaRuleElement.startMatch(RutaRuleElement.java:102)
at org.apache.uima.ruta.rule.ComposedRuleElement.startMatch(ComposedRuleElement.java:74)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:54)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:45)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:1)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.block.OnlyOnceBlock.apply(OnlyOnceBlock.java:43)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.action.CallAction.callScript(CallAction.java:85)
at org.apache.uima.ruta.action.CallAction.execute(CallAction.java:71)
at org.apache.uima.ruta.rule.AbstractRuleElement.apply(AbstractRuleElement.java:132)
at org.apache.uima.ruta.rule.RuleElementCaretaker.applyRuleElements(RuleElementCaretaker.java:111)
at org.apache.uima.ruta.rule.ComposedRuleElement.applyRuleElements(ComposedRuleElement.java:593)
at org.apache.uima.ruta.rule.AbstractRuleElement.doneMatching(AbstractRuleElement.java:84)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallback(ComposedRuleElement.java:514)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallbackContinue(ComposedRuleElement.java:415)
at org.apache.uima.ruta.rule.RutaRuleElement.startMatch(RutaRuleElement.java:102)
at org.apache.uima.ruta.rule.ComposedRuleElement.startMatch(ComposedRuleElement.java:74)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:54)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:45)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:1)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.action.CallAction.callScript(CallAction.java:85)
at org.apache.uima.ruta.action.CallAction.execute(CallAction.java:71)
at org.apache.uima.ruta.rule.AbstractRuleElement.apply(AbstractRuleElement.java:132)
at org.apache.uima.ruta.rule.RuleElementCaretaker.applyRuleElements(RuleElementCaretaker.java:111)
at org.apache.uima.ruta.rule.ComposedRuleElement.applyRuleElements(ComposedRuleElement.java:593)
at org.apache.uima.ruta.rule.AbstractRuleElement.doneMatching(AbstractRuleElement.java:84)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallback(ComposedRuleElement.java:514)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallbackContinue(ComposedRuleElement.java:415)
at org.apache.uima.ruta.rule.RutaRuleElement.startMatch(RutaRuleElement.java:102)
at org.apache.uima.ruta.rule.ComposedRuleElement.startMatch(ComposedRuleElement.java:74)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:54)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:45)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:1)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.RutaModule.apply(RutaModule.java:48)
at org.apache.uima.ruta.engine.RutaEngine.process(RutaEngine.java:536)
... 58 more
{code}
We'll check now, if there is any performance improvement done with the other two remaining classes.
Also, we're looking for a significant profile trend in memory consumption of the two implementations that threw an exception.
was (Author: dennisemp):
We've evaluated your 4 version, thanks for that. Your active one and the RutaBasic4 class didn't work in my setup and threw the following exception:
{code}
Caused by: java.lang.IndexOutOfBoundsException: Index (4794300) is greater than or equal to list size (2)
at it.unimi.dsi.fastutil.ints.IntArrayList.removeInt(IntArrayList.java:346)
at org.apache.uima.ruta.type.RutaBasic.removeBegin(RutaBasic.java:296)
at org.apache.uima.ruta.RutaStream.removeAnnotation(RutaStream.java:344)
at org.apache.uima.ruta.RutaStream.removeAnnotation(RutaStream.java:328)
at org.apache.uima.ruta.action.UnmarkAllAction.execute(UnmarkAllAction.java:73)
at org.apache.uima.ruta.rule.AbstractRuleElement.apply(AbstractRuleElement.java:132)
at org.apache.uima.ruta.rule.RuleElementCaretaker.applyRuleElements(RuleElementCaretaker.java:111)
at org.apache.uima.ruta.rule.ComposedRuleElement.applyRuleElements(ComposedRuleElement.java:593)
at org.apache.uima.ruta.rule.AbstractRuleElement.doneMatching(AbstractRuleElement.java:84)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallback(ComposedRuleElement.java:514)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallbackContinue(ComposedRuleElement.java:415)
at org.apache.uima.ruta.rule.RutaRuleElement.startMatch(RutaRuleElement.java:102)
at org.apache.uima.ruta.rule.ComposedRuleElement.startMatch(ComposedRuleElement.java:74)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:54)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:45)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:1)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.block.OnlyOnceBlock.apply(OnlyOnceBlock.java:43)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.action.CallAction.callScript(CallAction.java:85)
at org.apache.uima.ruta.action.CallAction.execute(CallAction.java:71)
at org.apache.uima.ruta.rule.AbstractRuleElement.apply(AbstractRuleElement.java:132)
at org.apache.uima.ruta.rule.RuleElementCaretaker.applyRuleElements(RuleElementCaretaker.java:111)
at org.apache.uima.ruta.rule.ComposedRuleElement.applyRuleElements(ComposedRuleElement.java:593)
at org.apache.uima.ruta.rule.AbstractRuleElement.doneMatching(AbstractRuleElement.java:84)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallback(ComposedRuleElement.java:514)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallbackContinue(ComposedRuleElement.java:415)
at org.apache.uima.ruta.rule.RutaRuleElement.startMatch(RutaRuleElement.java:102)
at org.apache.uima.ruta.rule.ComposedRuleElement.startMatch(ComposedRuleElement.java:74)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:54)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:45)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:1)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.action.CallAction.callScript(CallAction.java:85)
at org.apache.uima.ruta.action.CallAction.execute(CallAction.java:71)
at org.apache.uima.ruta.rule.AbstractRuleElement.apply(AbstractRuleElement.java:132)
at org.apache.uima.ruta.rule.RuleElementCaretaker.applyRuleElements(RuleElementCaretaker.java:111)
at org.apache.uima.ruta.rule.ComposedRuleElement.applyRuleElements(ComposedRuleElement.java:593)
at org.apache.uima.ruta.rule.AbstractRuleElement.doneMatching(AbstractRuleElement.java:84)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallback(ComposedRuleElement.java:514)
at org.apache.uima.ruta.rule.ComposedRuleElement.fallbackContinue(ComposedRuleElement.java:415)
at org.apache.uima.ruta.rule.RutaRuleElement.startMatch(RutaRuleElement.java:102)
at org.apache.uima.ruta.rule.ComposedRuleElement.startMatch(ComposedRuleElement.java:74)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:54)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:45)
at org.apache.uima.ruta.rule.RutaRule.apply(RutaRule.java:1)
at org.apache.uima.ruta.RutaScriptBlock.apply(RutaScriptBlock.java:63)
at org.apache.uima.ruta.RutaModule.apply(RutaModule.java:48)
at org.apache.uima.ruta.engine.RutaEngine.process(RutaEngine.java:536)
... 58 more
{code}
We'll check now, if there is any performance improvement done with the other two remaining classes.
Also, we're looking for a significant profile trend in memory consumption of the two implementations that threw an exception.
> Memory Improvement - Unnecessary leaks
> --------------------------------------
>
> Key: UIMA-5306
> URL: https://issues.apache.org/jira/browse/UIMA-5306
> Project: UIMA
> Issue Type: Improvement
> Components: Ruta
> Affects Versions: 2.3
> Environment: Windows 10, JVM with -Xmx 1024, Java JDK 1.8., 16gb memory
> Reporter: Dennis Bauer
> Assignee: Peter Klügl
>
> In a productive setup we figured out, that there is a huge memory usage of Ruta itself. With JVisualVM it's easy to see, that there is a relative small amount of arrays of Arraylists but with a high memory consumption (250k instances result in 243 000 000 byte memory that are reserved)
> The problem is, that in a clustered SaaS environment with less memory, these arrays block relevant space in memory. A deeper look into these Arrays of Arraylist let suggest the class org.apache.uima.ruta.type.RutaBasic
> A look at this class show three arrays that are instanced with the max. possible value, that can be returned by the typesystem of CAS.
> {code:Java}
> private int[] partOf = new int[((TypeSystemImpl) getCAS().getTypeSystem()).getLargestTypeCode()];
> private Collection<?>[] beginMap = new ArrayList<?>[((TypeSystemImpl) getCAS().getTypeSystem())
> .getLargestTypeCode()];
> private Collection<?>[] endMap = new ArrayList<?>[((TypeSystemImpl) getCAS().getTypeSystem())
> .getLargestTypeCode()];
>
> {code}
> In this improvement should be done an dynamic allocation of memory usage for these arrays, so the total memory consumption would be reduced.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)