[jira] [Commented] (ATLAS-1114) Performance improvements for create/update entities

["Srikanth Venkat (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ATLAS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840088#comment-15840088 ] 

Srikanth Venkat commented on ATLAS-1114:
----------------------------------------

Jeff,

It would be useful if along with the patch you can provide test cases or scenarios that were used to identify this issue and can be used to baseline and can be used to validate/benchmark improvements in these areas moving forward.

> Performance improvements for create/update entities
> ---------------------------------------------------
>
>                 Key: ATLAS-1114
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1114
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Jeffrey Hagelberg
>            Assignee: Jeffrey Hagelberg
>         Attachments: rb51092.patch
>
>
> General Atlas Perforance Improvements
> Apply performance fixes for create/update entities from IBM fork to Atlas.  During our performance profiling, we found a number of performance hotspots in JProfiler.  Our main findings were
>    - multiple queries were being executed for each instance being created/updated to find matches by unique attribute.  
>   - one query was being executed for each instance being created/updated to find the corresponding vertex if there is one
>   - Calculating the value of the full text property was taking a significant portion of the time to create/update entities, mainly due to its calls to getVertexForGUID
> The changes we put in do the following:
> - batch lookups by guid when create/update entities.  Execute one AtlasGraphQuery to find them all.
> - batch lookups by unique attribute when create/update entities.  Execute one AtlasGraphQuery per class to find unique attribute matches.
> - find all existing vertices up front during create/update entity.  Use those vertices during the graph mapping process to avoid running unnecessary graph queries
> - reuse reference vertices from instance to graph mapping when computing full text property



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)