You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Madhan Neethiraj <ma...@apache.org> on 2018/06/19 07:01:05 UTC
Review Request 67641: ATLAS-2760: updated Hive hook to create S3
entities for references to S3 path
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67641/
-----------------------------------------------------------
Review request for atlas, Apoorv Naik, Ashutosh Mestry, Don Bosco Durai, Nixon Rodrigues, Ramesh Mani, and Sarath Subramanian.
Bugs: ATLAS-2760
https://issues.apache.org/jira/browse/ATLAS-2760
Repository: atlas
Description
-------
Updated Hive hook to create entities S3 entities of type aws_s3_bucket, aws_s3_pseudo_dir, aws_s3_object for references to S3 paths
Diffs
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java ce032877c
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java daf5c860e
Diff: https://reviews.apache.org/r/67641/diff/1/
Testing
-------
Verified that Hive data export to S3 creates relevant S3 entities (bucket, pseudo-dir) and shows lineage between Hive table and S3 pesudo-dir.
Thanks,
Madhan Neethiraj
Re: Review Request 67641: ATLAS-2760: updated Hive hook to create S3
entities for references to S3 path
Posted by Ashutosh Mestry <am...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67641/#review205006
-----------------------------------------------------------
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
Lines 532 (patched)
<https://reviews.apache.org/r/67641/#comment287856>
Consider creating a method say 'isPathS3'
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java
Lines 533 (patched)
<https://reviews.apache.org/r/67641/#comment287857>
Consider moving this logic into new method say 'processS3Path'.
- Ashutosh Mestry
On June 19, 2018, 7:01 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67641/
> -----------------------------------------------------------
>
> (Updated June 19, 2018, 7:01 a.m.)
>
>
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Don Bosco Durai, Nixon Rodrigues, Ramesh Mani, and Sarath Subramanian.
>
>
> Bugs: ATLAS-2760
> https://issues.apache.org/jira/browse/ATLAS-2760
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Updated Hive hook to create entities S3 entities of type aws_s3_bucket, aws_s3_pseudo_dir, aws_s3_object for references to S3 paths
>
>
> Diffs
> -----
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java ce032877c
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java daf5c860e
>
>
> Diff: https://reviews.apache.org/r/67641/diff/1/
>
>
> Testing
> -------
>
> Verified that Hive data export to S3 creates relevant S3 entities (bucket, pseudo-dir) and shows lineage between Hive table and S3 pesudo-dir.
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 67641: ATLAS-2760: updated Hive hook to create S3
entities for references to S3 path
Posted by Ashutosh Mestry <am...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67641/#review205086
-----------------------------------------------------------
Ship it!
Ship It!
- Ashutosh Mestry
On June 20, 2018, 6:27 a.m., Madhan Neethiraj wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67641/
> -----------------------------------------------------------
>
> (Updated June 20, 2018, 6:27 a.m.)
>
>
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Don Bosco Durai, Nixon Rodrigues, Ramesh Mani, and Sarath Subramanian.
>
>
> Bugs: ATLAS-2760
> https://issues.apache.org/jira/browse/ATLAS-2760
>
>
> Repository: atlas
>
>
> Description
> -------
>
> Updated Hive hook to create entities S3 entities of type aws_s3_bucket, aws_s3_pseudo_dir, aws_s3_object for references to S3 paths
>
>
> Diffs
> -----
>
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java ce032877c
> addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java daf5c860e
>
>
> Diff: https://reviews.apache.org/r/67641/diff/2/
>
>
> Testing
> -------
>
> Verified that Hive data export to S3 creates relevant S3 entities (bucket, pseudo-dir) and shows lineage between Hive table and S3 pesudo-dir.
>
>
> Thanks,
>
> Madhan Neethiraj
>
>
Re: Review Request 67641: ATLAS-2760: updated Hive hook to create S3
entities for references to S3 path
Posted by Madhan Neethiraj <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67641/
-----------------------------------------------------------
(Updated June 20, 2018, 6:27 a.m.)
Review request for atlas, Apoorv Naik, Ashutosh Mestry, Don Bosco Durai, Nixon Rodrigues, Ramesh Mani, and Sarath Subramanian.
Changes
-------
addressed review comments
Bugs: ATLAS-2760
https://issues.apache.org/jira/browse/ATLAS-2760
Repository: atlas
Description
-------
Updated Hive hook to create entities S3 entities of type aws_s3_bucket, aws_s3_pseudo_dir, aws_s3_object for references to S3 paths
Diffs (updated)
-----
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/BaseHiveEvent.java ce032877c
addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/events/CreateTable.java daf5c860e
Diff: https://reviews.apache.org/r/67641/diff/2/
Changes: https://reviews.apache.org/r/67641/diff/1-2/
Testing
-------
Verified that Hive data export to S3 creates relevant S3 entities (bucket, pseudo-dir) and shows lineage between Hive table and S3 pesudo-dir.
Thanks,
Madhan Neethiraj