You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Ashutosh Mestry (Jira)" <ji...@apache.org> on 2020/04/06 00:42:00 UTC
[jira] [Updated] (ATLAS-3715) Atlas: Improve Speed of Primitive
Attribute Assignment
[ https://issues.apache.org/jira/browse/ATLAS-3715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Mestry updated ATLAS-3715:
-----------------------------------
Summary: Atlas: Improve Speed of Primitive Attribute Assignment (was: ZipFileMigrationImporter: Improve Speed of Primitive Attribute Assignment)
> Atlas: Improve Speed of Primitive Attribute Assignment
> ------------------------------------------------------
>
> Key: ATLAS-3715
> URL: https://issues.apache.org/jira/browse/ATLAS-3715
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Affects Versions: trunk
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Priority: Major
> Fix For: trunk
>
>
> *Background*
> While profiling entity creation as part of zip file-based migration import, it was observed that over 50% of time is spent in assigning values to attributes that are of primitive type viz. string, date, etc.
> *Methodology Used*
> The entity creation flow already has _MetricRecorder_ added to key steps within the entity creation flow. Additional methods (like _mapPrimitiveValue_) were profiled to get complete breakdown.
> Data set contained large number of entities of various types. Ones used were _hdfs_path, hive_db, hive_table, hive_column, hive_process, hive_process_execution, hive_column_lineage_.
> *Solution*
> Within the migration request context, use the _setEncodedProperty_ instead of using the current route of creating _AtttributeMutationContext_.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)