Re: Review Request 72177: Import Service: Support zipDirect Format

[Nikhil Bonte <ni...@freestoneinfotech.com>]

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72177/#review219714
-----------------------------------------------------------


Ship it!




Ship It!

- Nikhil Bonte


On Feb. 28, 2020, 9:52 p.m., Ashutosh Mestry wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/72177/
> -----------------------------------------------------------
> 
> (Updated Feb. 28, 2020, 9:52 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Nikhil Bonte, Nixon Rodrigues, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-3641
>     https://issues.apache.org/jira/browse/ATLAS-3641
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> **Approach**
> During genration entities are stores as 1 zip entry, _entities.json_.
> 
> New: _ZipSourceDirect_ contains a light-weight JSON parser to read the _entities.json_. This parser drives _next_ and _hasNext_ methods.
> Modified: _AtlasImportRequest_ adds new key and value to specify import format. 
> Modified: _ImportService_ supports handling of this format.
> 
> _import-options.json_
> ```
> {
> "options": {
>   "format": "zipDirect"
>  }
> }
> ```
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportRequest.java 0b3ede93f 
>   intg/src/main/java/org/apache/atlas/utils/AtlasJson.java 1c13860cf 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ImportService.java 27001e3a9 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ZipExportFileNames.java 351b47536 
>   repository/src/main/java/org/apache/atlas/repository/impexp/ZipSourceDirect.java PRE-CREATION 
>   repository/src/test/java/org/apache/atlas/repository/impexp/ImportServiceTest.java c14850f43 
>   repository/src/test/java/org/apache/atlas/repository/impexp/ZipDirectTest.java PRE-CREATION 
>   repository/src/test/resources/zip-direct-1.zip PRE-CREATION 
>   repository/src/test/resources/zip-direct-2.zip PRE-CREATION 
>   repository/src/test/resources/zip-direct-3.zip PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/72177/diff/1/
> 
> 
> Testing
> -------
> 
> **Unit tests**
> - Tests for reading this format.
> - _ImportServiceTest_ performs end-to-end test.
> 
> **Volume test**
> Files with this format lends itself to volume. File with 10 million entities was performed.
> 
> **Pre-commit**
> https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/
> 
> 
> Thanks,
> 
> Ashutosh Mestry
> 
>