You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry <am...@hortonworks.com> on 2017/07/19 04:56:31 UTC
Review Request 60954: ATLAS-1948: Addressed Case Where Export Order
List Did Not Add Process Guids At the End In All Cases
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60954/
-----------------------------------------------------------
Review request for atlas and Madhan Neethiraj.
Bugs: ATLAS-1948
https://issues.apache.org/jira/browse/ATLAS-1948
Repository: atlas
Description
-------
**Background & Analysis**
Please see [ATLAS-1948](https://issues.apache.org/jira/browse/ATLAS-1948) for scenario details.
_Root Cause_
A case where lineage links 2 separate databases, the order of entity creation had process entities inserted between the 2 database entities. This caused process entities not to be created correctly.
_Side Effect_
An export with the above problem would result in process entities not getting created, thereby causing entire Import operation to be aborted with "entitiy id" not found error.
**Implementation**
_Export API_
The export operation was addressed to maintain the process entitities to be created after all the other entities are created.
This is a low-impact fix.
_Import API_
The import operation now maintains a separated list of failed entities. These entities are attempted to be imporated once all the non-process entities are imported.
This was achieved by implementing a _EntityImportStreamWithResidualList_. This private class accepts the _EntityImportStream_ and _List_ as 2 parameters. The _List_ has guids of entities that failed during regular import process. Once all the entities from the _EntitiyImportStream_ are exhausted, this list gets processed.
Diffs
-----
repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 8f45e9fd
repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java edb816fc
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java de8e7ef3
repository/src/test/resources/ctas.zip PRE-CREATION
Diff: https://reviews.apache.org/r/60954/diff/1/
Testing
-------
**Unit tests**
Additional unit tests for that uses this test case.
**Functional test**
Regular tests cases.
Thanks,
Ashutosh Mestry
Re: Review Request 60954: ATLAS-1948: Addressed Case Where Export
Order List Did Not Add Process Guids At the End In All Cases
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60954/#review180906
-----------------------------------------------------------
Fix it, then Ship it!
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
Line 176 (original), 217 (patched)
<https://reviews.apache.org/r/60954/#comment256249>
"resp" is not used outside try {}. Consider moving it inside.
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
Lines 229 (patched)
<https://reviews.apache.org/r/60954/#comment256248>
{ }
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
Lines 241 (patched)
<https://reviews.apache.org/r/60954/#comment256247>
- "ATLAS-400-00-021" ==> AtlasErrorCode.INVALID_OBJECT_ID.getErrorCode()
- { }
- Madhan Neethiraj
On July 19, 2017, 4:56 a.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60954/
> -----------------------------------------------------------
>
> (Updated July 19, 2017, 4:56 a.m.)
>
>
> Review request for atlas and Madhan Neethiraj.
>
>
> Bugs: ATLAS-1948
> https://issues.apache.org/jira/browse/ATLAS-1948
>
>
> Repository: atlas
>
>
> Description
> -------
>
> **Background & Analysis**
> Please see [ATLAS-1948](https://issues.apache.org/jira/browse/ATLAS-1948) for scenario details.
>
> _Root Cause_
> A case where lineage links 2 separate databases, the order of entity creation had process entities inserted between the 2 database entities. This caused process entities not to be created correctly.
>
> _Side Effect_
> An export with the above problem would result in process entities not getting created, thereby causing entire Import operation to be aborted with "entitiy id" not found error.
>
> **Implementation**
>
> _Export API_
> The export operation was addressed to maintain the process entitities to be created after all the other entities are created.
>
> This is a low-impact fix.
>
> _Import API_
> The import operation now maintains a separated list of failed entities. These entities are attempted to be imporated once all the non-process entities are imported.
>
> This was achieved by implementing a _EntityImportStreamWithResidualList_. This private class accepts the _EntityImportStream_ and _List_ as 2 parameters. The _List_ has guids of entities that failed during regular import process. Once all the entities from the _EntitiyImportStream_ are exhausted, this list gets processed.
>
>
> Diffs
> -----
>
> repository/src/main/java/org/apache/atlas/repository/impexp/ExportService.java 8f45e9fd
> repository/src/main/java/org/apache/atlas/repository/impexp/ZipSource.java edb816fc
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 5ea4ff20
> repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java de8e7ef3
> repository/src/test/resources/ctas.zip PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/60954/diff/1/
>
>
> Testing
> -------
>
> **Unit tests**
>
> Additional unit tests for that uses this test case.
>
> **Functional test**
>
> Regular tests cases.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>