[jira] [Commented] (ATLAS-471) Atlas Server could run out of memory due to Scala memory leak

["Hemanth Yamijala (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ATLAS-471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132218#comment-15132218 ] 

Hemanth Yamijala commented on ATLAS-471:
----------------------------------------

Note that we're using Scala 2.10. And it looks like there is a fairly popular bug involving this piece of code, that introduces a memory leak. Few references:

[1] https://issues.scala-lang.org/browse/SI-4929?focusedCommentId=61731&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-61731
[2] http://searler.github.io/2014/01/26/working-around-scala-parser-memory-leak.html
[3] https://issues.apache.org/jira/browse/SPARK-10155

I suspect we are hitting this problem, given the symptoms are similar. A fix along the lines of SPARK-10155 might work: https://github.com/apache/spark/pull/8357/commits

> Atlas Server could run out of memory due to Scala memory leak
> -------------------------------------------------------------
>
>                 Key: ATLAS-471
>                 URL: https://issues.apache.org/jira/browse/ATLAS-471
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Hemanth Yamijala
>            Priority: Critical
>
> We noticed our servers running OutOfMemory occasionally, particularly under heavy query loads. Profiling the app with YourKit, I found a lot of instances of {{scala.util.parsing.combinator.Parsers}}, {{java.lang.ThreadLocal$ThreadLocalMap}} and {{org.apache.atlas.query.QueryLexer}} objects. These object counts increase steadily over time as seen after taking multiple snapshots.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)