You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Jeffrey Hagelberg (JIRA)" <ji...@apache.org> on 2016/08/10 15:53:20 UTC
[jira] [Updated] (ATLAS-1114) Performance improvements for
create/update entities
[ https://issues.apache.org/jira/browse/ATLAS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeffrey Hagelberg updated ATLAS-1114:
-------------------------------------
Description:
General Atlas Perforance Improvements
Apply performance fixes for create/update entities from IBM fork to Atlas. During our performance profiling, we found a number of performance hotspots in JProfiler. Our main findings were
- multiple queries were being executed for each instance being created/updated to find matches by unique attribute.
- one query was being executed for each instance being created/updated to find the corresponding vertex if there is one
- Calculating the value of the full text property was taking a significant portion of the time to create/update entities, mainly due to its calls to getVertexForGUID
The changes we put in do the following:
- batch lookups by guid when create/update entities. Execute one AtlasGraphQuery to find them all.
- batch lookups by unique attribute when create/update entities. Execute one AtlasGraphQuery per class to find unique attribute matches.
- find all existing vertices up front during create/update entity. Use those vertices during the graph mapping process to avoid running unnecessary graph queries
- reuse reference vertices from instance to graph mapping when computing full text property
was:
General Atlas Perforance Improvements
Apply performance fixes for create/update entities from IBM fork to Atlas. During our performance profiling, we found a number of performance hotspots in JProfiler. Note that our profiling was done with the IBM Graph implementation, but the performance optimizations are very general in nature and help with performance on Titan as well. Our main findings were
- multiple queries were being executed for each instance being created/updated to find matches by unique attribute.
- one query was being executed for each instance being created/updated to find the corresponding vertex if there is one
- Calculating the value of the full text property was taking a significant portion of the time to create/update entities, mainly due to its calls to getVertexForGUID
The changes we put in do the following:
- batch lookups by guid when create/update entities. Execute one AtlasGraphQuery to find them all.
- batch lookups by unique attribute when create/update entities. Execute one AtlasGraphQuery per class to find unique attribute matches.
- find all existing vertices up front during create/update entity. Use those vertices during the graph mapping process to avoid running unnecessary graph queries
- reuse reference vertices from instance to graph mapping when computing full text property
> Performance improvements for create/update entities
> ---------------------------------------------------
>
> Key: ATLAS-1114
> URL: https://issues.apache.org/jira/browse/ATLAS-1114
> Project: Atlas
> Issue Type: Bug
> Reporter: Jeffrey Hagelberg
> Assignee: Jeffrey Hagelberg
>
> General Atlas Perforance Improvements
> Apply performance fixes for create/update entities from IBM fork to Atlas. During our performance profiling, we found a number of performance hotspots in JProfiler. Our main findings were
> - multiple queries were being executed for each instance being created/updated to find matches by unique attribute.
> - one query was being executed for each instance being created/updated to find the corresponding vertex if there is one
> - Calculating the value of the full text property was taking a significant portion of the time to create/update entities, mainly due to its calls to getVertexForGUID
> The changes we put in do the following:
> - batch lookups by guid when create/update entities. Execute one AtlasGraphQuery to find them all.
> - batch lookups by unique attribute when create/update entities. Execute one AtlasGraphQuery per class to find unique attribute matches.
> - find all existing vertices up front during create/update entity. Use those vertices during the graph mapping process to avoid running unnecessary graph queries
> - reuse reference vertices from instance to graph mapping when computing full text property
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)