You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Balu Vellanki <bv...@hortonworks.com> on 2016/01/29 01:23:50 UTC
Review Request 42931: Provide Apache Atlas integration
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/
-----------------------------------------------------------
Review request for Sqoop and Venkat Ranganathan.
Bugs: sqoop-2609
https://issues.apache.org/jira/browse/sqoop-2609
Repository: sqoop-trunk
Description
-------
We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
This will allow us to identify the lineage of data.
Diffs
-----
conf/sqoop-site-template.xml 368af26
src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
Diff: https://reviews.apache.org/r/42931/diff/
Testing
-------
end2end testing done.
Thanks,
Balu Vellanki
Re: Review Request 42931: Provide Apache Atlas integration
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/#review116947
-----------------------------------------------------------
Thanks for testing this @Balu.
Also can we add a small doc note that this is the currently for imports and will be enhanced for exports also
src/java/org/apache/sqoop/SqoopJobDataPublisher.java (line 115)
<https://reviews.apache.org/r/42931/#comment178027>
Let us add them as String constants so that we can refer them in tests and in other places
src/java/org/apache/sqoop/mapreduce/ImportJobBase.java (line 284)
<https://reviews.apache.org/r/42931/#comment178028>
Let us move this as a constant to ConfigurationConstants
- Venkat Ranganathan
On Jan. 28, 2016, 4:23 p.m., Balu Vellanki wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42931/
> -----------------------------------------------------------
>
> (Updated Jan. 28, 2016, 4:23 p.m.)
>
>
> Review request for Sqoop and Venkat Ranganathan.
>
>
> Bugs: sqoop-2609
> https://issues.apache.org/jira/browse/sqoop-2609
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
>
> This will allow us to identify the lineage of data.
>
>
> Diffs
> -----
>
> conf/sqoop-site-template.xml 368af26
> src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
> src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
> src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42931/diff/
>
>
> Testing
> -------
>
> end2end testing done.
>
>
> Thanks,
>
> Balu Vellanki
>
>
Re: Review Request 42931: Provide Apache Atlas integration
Posted by Balu Vellanki <bv...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/
-----------------------------------------------------------
(Updated Feb. 25, 2016, 5:54 p.m.)
Review request for Sqoop and Venkat Ranganathan.
Changes
-------
Removed doc entry in hive.txt as suggested by Venkat Ranganathan.
Bugs: sqoop-2609
https://issues.apache.org/jira/browse/sqoop-2609
Repository: sqoop-trunk
Description
-------
We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
This will allow us to identify the lineage of data.
Diffs (updated)
-----
conf/sqoop-site-template.xml 368af26
src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
src/java/org/apache/sqoop/config/ConfigurationConstants.java bd6e99b
src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
Diff: https://reviews.apache.org/r/42931/diff/
Testing
-------
end2end testing done.
Thanks,
Balu Vellanki
Re: Review Request 42931: Provide Apache Atlas integration
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/#review120026
-----------------------------------------------------------
src/docs/user/hive.txt (line 104)
<https://reviews.apache.org/r/42931/#comment181391>
Can you please remove this and reupload the patch
- Venkat Ranganathan
On Jan. 29, 2016, 10:32 a.m., Balu Vellanki wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42931/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2016, 10:32 a.m.)
>
>
> Review request for Sqoop and Venkat Ranganathan.
>
>
> Bugs: sqoop-2609
> https://issues.apache.org/jira/browse/sqoop-2609
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
>
> This will allow us to identify the lineage of data.
>
>
> Diffs
> -----
>
> conf/sqoop-site-template.xml 368af26
> src/docs/user/hive.txt 3dc8bb4
> src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
> src/java/org/apache/sqoop/config/ConfigurationConstants.java e19c17b
> src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
> src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42931/diff/
>
>
> Testing
> -------
>
> end2end testing done.
>
>
> Thanks,
>
> Balu Vellanki
>
>
Re: Review Request 42931: Provide Apache Atlas integration
Posted by Balu Vellanki <bv...@hortonworks.com>.
> On Feb. 1, 2016, 2:43 a.m., Venkat Ranganathan wrote:
> > src/docs/user/hive.txt, line 104
> > <https://reviews.apache.org/r/42931/diff/2/?file=1225839#file1225839line104>
> >
> > We support for hive import and hcatalog table imports today.
I will create docs as part of separate Jira? I created one at https://issues.apache.org/jira/browse/SQOOP-2815 , please let me know if that is ok.
- Balu
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/#review117174
-----------------------------------------------------------
On Jan. 29, 2016, 6:32 p.m., Balu Vellanki wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42931/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2016, 6:32 p.m.)
>
>
> Review request for Sqoop and Venkat Ranganathan.
>
>
> Bugs: sqoop-2609
> https://issues.apache.org/jira/browse/sqoop-2609
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
>
> This will allow us to identify the lineage of data.
>
>
> Diffs
> -----
>
> conf/sqoop-site-template.xml 368af26
> src/docs/user/hive.txt 3dc8bb4
> src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
> src/java/org/apache/sqoop/config/ConfigurationConstants.java e19c17b
> src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
> src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42931/diff/
>
>
> Testing
> -------
>
> end2end testing done.
>
>
> Thanks,
>
> Balu Vellanki
>
>
Re: Review Request 42931: Provide Apache Atlas integration
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/#review117174
-----------------------------------------------------------
Thanks Balu. Please update the docs for hcatalog and hive also. I would suggest to move this to import documentation instead of hive documentation text
src/docs/user/hive.txt (line 104)
<https://reviews.apache.org/r/42931/#comment178260>
We support for hive import and hcatalog table imports today.
- Venkat Ranganathan
On Jan. 29, 2016, 10:32 a.m., Balu Vellanki wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42931/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2016, 10:32 a.m.)
>
>
> Review request for Sqoop and Venkat Ranganathan.
>
>
> Bugs: sqoop-2609
> https://issues.apache.org/jira/browse/sqoop-2609
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
>
> This will allow us to identify the lineage of data.
>
>
> Diffs
> -----
>
> conf/sqoop-site-template.xml 368af26
> src/docs/user/hive.txt 3dc8bb4
> src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
> src/java/org/apache/sqoop/config/ConfigurationConstants.java e19c17b
> src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
> src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42931/diff/
>
>
> Testing
> -------
>
> end2end testing done.
>
>
> Thanks,
>
> Balu Vellanki
>
>
Re: Review Request 42931: Provide Apache Atlas integration
Posted by Balu Vellanki <bv...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42931/
-----------------------------------------------------------
(Updated Jan. 29, 2016, 6:32 p.m.)
Review request for Sqoop and Venkat Ranganathan.
Changes
-------
Updated patch to reflect feedback from Venkat Ranganathan
Bugs: sqoop-2609
https://issues.apache.org/jira/browse/sqoop-2609
Repository: sqoop-trunk
Description
-------
We would like to provide lineage information to Apache Atlas when data is ingested into Hive via Sqoop as the first step.
This will allow us to identify the lineage of data.
Diffs (updated)
-----
conf/sqoop-site-template.xml 368af26
src/docs/user/hive.txt 3dc8bb4
src/java/org/apache/sqoop/SqoopJobDataPublisher.java PRE-CREATION
src/java/org/apache/sqoop/config/ConfigurationConstants.java e19c17b
src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 04d60fd
src/test/org/apache/sqoop/TestSqoopJobDataPublisher.java PRE-CREATION
Diff: https://reviews.apache.org/r/42931/diff/
Testing
-------
end2end testing done.
Thanks,
Balu Vellanki