You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@atlas.apache.org by "Sarath Subramanian (Jira)" <ji...@apache.org> on 2019/12/11 23:43:01 UTC

[jira] [Created] (ATLAS-3558) Improve lineage performance using in-memory traversal

Sarath Subramanian created ATLAS-3558:
-----------------------------------------

             Summary: Improve lineage performance using in-memory traversal
                 Key: ATLAS-3558
                 URL: https://issues.apache.org/jira/browse/ATLAS-3558
             Project: Atlas
          Issue Type: Improvement
          Components:  atlas-core
    Affects Versions: 2.0.0
            Reporter: Sarath Subramanian
            Assignee: Sarath Subramanian
             Fix For: 2.1.0


Lineage in atlas uses graph query to compute lineage across entities (inputs, outputs or both). Lineage rendering performance have degraded after using janusgraph version 0.4.0

On investigation, lineage graph query initialization and execution using gremlin script engine has been found to be the bottleneck.

Alternate in-memory computation of lineage has improved performance by many folds (~90% improvement). This Jira is about adding alternate in-memory computation of lineage.

*"atlas.use.graph.query.for.lineage"* property can be used to toggle between graph query and in-memory computation of lineage. The default option will be in-memory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)