[jira] [Updated] (ATLAS-471) Atlas Server could run out of memory due to Scala memory leak

["Hemanth Yamijala (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/ATLAS-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hemanth Yamijala updated ATLAS-471:
-----------------------------------
    Attachment: ATLAS-471-v1.patch

Attaching a patch for initial review. The main change is to convert the QueryParser into a singleton object from a Scala class. Given in Scala 2.10 the Parser is threadsafe, it's probably OK to do this. This also seems to be the nature of fix in SPARK-10155.

I've run this patch with 2 simultaneous clients firing DSL and gremlin queries in a loop and watching the Atlas memory via YourKit. It appears the heap is under control fairly well. The parser objects did not figure in the top objects of the profiler. There were no exceptions in the Atlas log (checked to see if there was any multi-thread type exceptions). I will test more exhaustively tomorrow, but if someone can give it a quick look, that would be great.

> Atlas Server could run out of memory due to Scala memory leak
> -------------------------------------------------------------
>
>                 Key: ATLAS-471
>                 URL: https://issues.apache.org/jira/browse/ATLAS-471
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Hemanth Yamijala
>            Priority: Critical
>         Attachments: ATLAS-471-v1.patch
>
>
> We noticed our servers running OutOfMemory occasionally, particularly under heavy query loads. Profiling the app with YourKit, I found a lot of instances of {{scala.util.parsing.combinator.Parsers}}, {{java.lang.ThreadLocal$ThreadLocalMap}} and {{org.apache.atlas.query.QueryLexer}} objects. These object counts increase steadily over time as seen after taking multiple snapshots.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)