[jira] [Comment Edited] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation

["Praneeth (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16726008#comment-16726008 ] 

Praneeth edited comment on OOZIE-3409 at 12/20/18 4:46 PM:
-----------------------------------------------------------

Actually, I don't think it is just the actions that have EL expressions that sit in the cache, I think the parsing is taking place for every action (or some part of workflow config), whether or not it has an EL (call to parsing takes place irrespectively) [1]. If it doesn't have an EL, the cache would store the same content as both key and value. I was able to verify this by looking at the contents of the cache.

This is only to point out that it is not just for EL based content. I do not have a good proposal for a solution given the dependency is dormant. What is the general practice for fixes to dormant projects ? Is it to try to find an alternative and if none is found, try to revive the project ?

I agree that it is probably a problem to few deployments like ours which are long running and relatively large workflows, but this crashes our instances once in 2-3 months.

 
 # I was checking the usage of [https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L201.]


was (Author: praneeth.varma):
Actually, I don't think it is just the actions that have EL expressions that sit in the cache, I think the parsing is taking place for every action (or some part of workflow config), whether or not it has an EL (call to parsing takes place irrespectively) [1]. If it doesn't have an EL, the cache would store the same content as both key and value. I was able to verify this by looking at the contents of the cache.

This is only to point out that it is not just for EL based content. I do not have a good proposal for a solution given the dependency is dormant. What is the general practice for fixes to dormant projects ? Is it to try to find an alternative and if none is found, try to revive the project ?

I agree that it is probably a problem to few deployments like ours which are long running and relatively workflows, but this crashes our instances once in 2-3 months.

 
 # I was checking the usage of [https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L201.]

> Oozie Server : Possible memory leak in EL evaluation
> ----------------------------------------------------
>
>                 Key: OOZIE-3409
>                 URL: https://issues.apache.org/jira/browse/OOZIE-3409
>             Project: Oozie
>          Issue Type: Bug
>    Affects Versions: 5.1.0
>            Reporter: Praneeth
>            Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers have been running out of memory occasionally. It happens more often when we have more activity. Upon analysis of the heap dump, it seems that the culprit here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from {{commons-el}} which is used in the Oozie server through {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses a static {{Map}} [2] to cache the EL evaluations. This map never gets cleared, eventually taking up all the memory. So, in an environment with heavy usage and long running oozie servers, this would lead to an OOM. The configurations for workflow actions we have are possibly larger than normal which is worsening the situation as the Map has to maintain larger strings.
>  # [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
>  # [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)