You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Nikhil Bonte (Jira)" <ji...@apache.org> on 2020/05/15 14:20:00 UTC
[jira] [Commented] (ATLAS-3799) Migration Import: EntityConsumer
Adds GUIDs to Result that Will Not Get Cleared
[ https://issues.apache.org/jira/browse/ATLAS-3799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17108331#comment-17108331 ]
Nikhil Bonte commented on ATLAS-3799:
-------------------------------------
+1 for the patch.
Thanks [~amestry]
> Migration Import: EntityConsumer Adds GUIDs to Result that Will Not Get Cleared
> -------------------------------------------------------------------------------
>
> Key: ATLAS-3799
> URL: https://issues.apache.org/jira/browse/ATLAS-3799
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Affects Versions: trunk
> Reporter: Ashutosh Mestry
> Assignee: Ashutosh Mestry
> Priority: Major
> Fix For: trunk
>
> Attachments: ATLAS-3799-EntityConumer-only-adds-entity-GUIDs-that.patch
>
>
> *Background*
> Zip file-based migration import introduced _EntityConsumer_. This uses _StatusReporter_ that is used to determine which which of the produced items are processed. This, in turn, helps determine the right position to set in case of resuming the import.
> The _EntityConumer_ class is responsible for adding the right GUIDs to the result that helps _StatusReporter_ determine the right GUIDs. This is done correctly except that it also adds additional GUIDs that were updated or deleted. This causes _StatusReporter_._processedSet_ to be filled with GUIDs that will never be cleared. This will grow over time and can potentially lead to memory pressure.
> *Solution*
> Add only the GUIDs that were processed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)