[jira] [Commented] (ATLAS-3702) Entity Creation: Improve Edge Creation

["ASF subversion and git services (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ATLAS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17073969#comment-17073969 ] 

ASF subversion and git services commented on ATLAS-3702:
--------------------------------------------------------

Commit 159afa8044fac64436268e1f64bfe43beff65096 in atlas's branch refs/heads/master from Ashutosh Mestry
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=159afa8 ]

ATLAS-3702: Edge creation performance improvements.


> Entity Creation: Improve Edge Creation
> --------------------------------------
>
>                 Key: ATLAS-3702
>                 URL: https://issues.apache.org/jira/browse/ATLAS-3702
>             Project: Atlas
>          Issue Type: Improvement
>          Components:  atlas-core
>    Affects Versions: trunk
>            Reporter: Ashutosh Mestry
>            Assignee: Ashutosh Mestry
>            Priority: Major
>             Fix For: trunk
>
>
> *Background*
> When creating entities with Atlas at scale, few inefficiencies were observed.
> They were:
>  * _GraphHelper.getAdjacentEdgesByLabel_: The existing approach results in high memory usage. The method could use graph api that already exists.
>  * _AtlasRelationshipStoreV2.getRelationshipEdge_: This does full vertex comparison. This is expensive.
>  * Edge label is not index. Indexing this will improve speed of edge by label lookup.
> *Solution*
> Profile Atlas with (using metrics) to establish baseline.
> Use the fixes mentioned to the places above and profile.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)