You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ashutosh Mestry <am...@hortonworks.com> on 2017/02/17 19:57:19 UTC
Review Request 56795: Import REST API
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56795/
-----------------------------------------------------------
Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
Bugs: ATLAS-1503
https://issues.apache.org/jira/browse/ATLAS-1503
Repository: atlas
Description
-------
The patch includes the optimization done for import:
- Handling of large databases (151 tables, each with 80 columns)
- Improved handling of zip file inputs.
- Improved metrics data. This now includes granular breakdown of individual entities and list of all entities imported.
Import is implemented in 2 flavors:
- ZIP file upload in POST.
CURL:
```
curl -X POST -u admin:admin -H "Content-Type: application/octet-stream" -H "Cache-Control: no-cache" --data-binary @../docs/DefaultDB-Stocks.zip "http://localhost:21000/api/atlas/admin/import"
```
- Specify ZIP file as part of query parameter in GET.
CURL:
```
curl -X POST -u admin:admin -H "Cache-Control: no-cache" "http://localhost:21000/api/atlas/admin/importFile?FILENAME=/Users/amestry/Documents/DefaultDB-Sales.zip"
```
Diffs
-----
intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 584bf25
intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportResult.java a5eeef1
intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 74e3c57
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java 1cd4375
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 4312287
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStream.java 010b626
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStreamForImport.java PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 2e71ab8
webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java c8c0099
webapp/src/main/java/org/apache/atlas/web/resources/ExportService.java 7d3d442
webapp/src/main/java/org/apache/atlas/web/resources/ImportService.java 7554cdb
webapp/src/main/java/org/apache/atlas/web/resources/ZipSource.java ea62862
Diff: https://reviews.apache.org/r/56795/diff/
Testing
-------
Status:
- Large DB (151 tables): Export.
- - Quick start databases: Done.
- Includes lineage information.
- hive_db database with upload from CSV file: Done.
- Unit tests: Pending.
Thanks,
Ashutosh Mestry
Re: Review Request 56795: Import REST API
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56795/#review165997
-----------------------------------------------------------
Ship it!
Ship It!
- Madhan Neethiraj
On Feb. 17, 2017, 7:57 p.m., Ashutosh Mestry wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56795/
> -----------------------------------------------------------
>
> (Updated Feb. 17, 2017, 7:57 p.m.)
>
>
> Review request for atlas, Madhan Neethiraj and Sarath Subramanian.
>
>
> Bugs: ATLAS-1503
> https://issues.apache.org/jira/browse/ATLAS-1503
>
>
> Repository: atlas
>
>
> Description
> -------
>
> The patch includes the optimization done for import:
> - Handling of large databases (151 tables, each with 80 columns)
> - Improved handling of zip file inputs.
> - Improved metrics data. This now includes granular breakdown of individual entities and list of all entities imported.
>
> Import is implemented in 2 flavors:
> - ZIP file upload in POST.
> CURL:
> ```
> curl -X POST -u admin:admin -H "Content-Type: application/octet-stream" -H "Cache-Control: no-cache" --data-binary @../docs/DefaultDB-Stocks.zip "http://localhost:21000/api/atlas/admin/import"
> ```
> - Specify ZIP file as part of query parameter in GET.
> CURL:
> ```
> curl -X POST -u admin:admin -H "Cache-Control: no-cache" "http://localhost:21000/api/atlas/admin/importFile?FILENAME=/Users/amestry/Documents/DefaultDB-Sales.zip"
> ```
>
>
> Diffs
> -----
>
> intg/src/main/java/org/apache/atlas/AtlasErrorCode.java 584bf25
> intg/src/main/java/org/apache/atlas/model/impexp/AtlasImportResult.java a5eeef1
> intg/src/main/java/org/apache/atlas/model/instance/EntityMutations.java 74e3c57
> repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java 1cd4375
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java 4312287
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStream.java 010b626
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStreamForImport.java PRE-CREATION
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java 2e71ab8
> webapp/src/main/java/org/apache/atlas/web/resources/AdminResource.java c8c0099
> webapp/src/main/java/org/apache/atlas/web/resources/ExportService.java 7d3d442
> webapp/src/main/java/org/apache/atlas/web/resources/ImportService.java 7554cdb
> webapp/src/main/java/org/apache/atlas/web/resources/ZipSource.java ea62862
>
> Diff: https://reviews.apache.org/r/56795/diff/
>
>
> Testing
> -------
>
> Status:
> - Large DB (151 tables): Export.
> - - Quick start databases: Done.
> - Includes lineage information.
> - hive_db database with upload from CSV file: Done.
> - Unit tests: Pending.
>
>
> Thanks,
>
> Ashutosh Mestry
>
>