You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@atlas.apache.org by "Sarath Subramanian (Jira)" <ji...@apache.org> on 2019/11/21 19:45:00 UTC

[jira] [Updated] (ATLAS-3519) Reuse script engine to improve lineage query performance

     [ https://issues.apache.org/jira/browse/ATLAS-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sarath Subramanian updated ATLAS-3519:
--------------------------------------
    Summary: Reuse script engine to improve lineage query performance  (was: pool performance in hive lineage query )

> Reuse script engine to improve lineage query performance
> --------------------------------------------------------
>
>                 Key: ATLAS-3519
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3519
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>    Affects Versions: 1.1.0, 2.0.0
>            Reporter: zhangxiong
>            Assignee: zhangxiong
>            Priority: Minor
>              Labels: lineage, performance
>             Fix For: 2.1.0
>
>         Attachments: ATLAS-3519.patch
>
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> when query lineage info for hive, the throughput reachs only 3 query per second.
> we use jstack and jfr+flame graph tools and found that the  GremlinGroovyScriptEngine object loading class  logic takes much of the time. so we make the GremlinGroovyScriptEngine a global singleton object, which would be shared accross all query threads. finally the query performance improved much and got 200 qps.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)