You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@falcon.apache.org by Venkatesan Ramachandran <me...@gmail.com> on 2016/01/22 04:27:47 UTC
Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/
-----------------------------------------------------------
Review request for Falcon.
Repository: falcon-git
Description
-------
Support password alias for database import and export using java keystore
Diffs
-----
client/src/main/resources/datasource-0.1.xsd beb82cc
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
webapp/src/test/java/org/apache/falcon/resource/AbstractTestContext.java 413dfde
webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
webapp/src/test/resources/datasource-template.xml fb7a329
webapp/src/test/resources/datasource-template1.xml PRE-CREATION
webapp/src/test/resources/datasource-template2.xml PRE-CREATION
webapp/src/test/resources/feed-template3.xml a6c1d6b
webapp/src/test/resources/feed-template4.xml PRE-CREATION
Diff: https://reviews.apache.org/r/42642/diff/
Testing
-------
Unit tests and Manual end to end testing both on regular and secure cluster.
Thanks,
Venkatesan Ramachandran
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkat Ranganathan <n....@live.com>.
> On Feb. 1, 2016, 1:40 p.m., Venkat Ranganathan wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 211
> > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211>
> >
> > I think you are doing this for ServiceLoader META-INF entries not properly merged. We should see if we can have some maven strategy to fix this
> >
> > How will this work other filesystem solutions like WASB?
> >
> > How are we passing the principal for remote Filesystems when the file is on a remote system?
>
> Venkatesan Ramachandran wrote:
> The datasouce definition specifies the HDFS pathname for the jceks provider -- it is inferred as HDFS for now and so the conf.set(). Regarding WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, this should also work since I used the CredentiaProviderFactory comes with hadoop-common.
>
> Similarly, the jceks provider file has to be on the cluster where the datasource entity exists.
>
> Venkat Ranganathan wrote:
> You are right. Sqoop can work with HDFS and WASB OOB - and sqoop actions when run via oozie have the jks file localized or accessed as a HDFS/WASB location as the case may be. But here the FS implementation is explicitly set for HDFS because of the implementation for other FS are not available because of some issues. May be I don't fully grok the details, but can't we make it work independent of the FS where the file is located? For example, in a HDI installation, fs.defaultFS will be a WASB location - it will not work there. It will be good to not restrict to HDFS like this as there are other filesystems used also.
>
> Regarding the security, here there is a call to new Configuration(). It will get the current Falcon cluster configuration with the principals from the current cluster where Falcon server is running. What if the file is on a remote cluster? Don't we have to make sure we use the filesystem and principals related to that Filesystem that we pick up from the corresponding cluster entity?
>
> Venkatesan Ramachandran wrote:
> Let me check if it works without HDFS impl hard coded. if so, we should be ok with WASB.
>
> Not sure I understand the issue about file on a remote cluster. Let's talk about it offline.
>
> Venkatesan Ramachandran wrote:
> hard coding filesystem is superflous. removed it and made sure it works ends to end with UT/IT.
> conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem");
>
> Regarding the remote cluster accessing the password provider file on another cluster with kerberos -- will open a separate JIRA (https://issues.apache.org/jira/browse/FALCON-1814) to address. This feature should work with kerberos and one cluster setup.
I am ok with the remote cluster setup being tracked and handled properly. It looks like we did not have ServiceLoader META-INF/ entries being squashed that I initially thought was the reason you had the explicit fs.hdfs.. values.
Thanks for creating a separate JIRA for that
- Venkat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117274
-----------------------------------------------------------
On Feb. 2, 2016, 6:08 p.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Feb. 2, 2016, 6:08 p.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml 12672bd
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java b55d660
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
> On Feb. 1, 2016, 9:40 p.m., Venkat Ranganathan wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 211
> > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211>
> >
> > I think you are doing this for ServiceLoader META-INF entries not properly merged. We should see if we can have some maven strategy to fix this
> >
> > How will this work other filesystem solutions like WASB?
> >
> > How are we passing the principal for remote Filesystems when the file is on a remote system?
The datasouce definition specifies the HDFS pathname for the jceks provider -- it is inferred as HDFS for now and so the conf.set(). Regarding WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, this should also work since I used the CredentiaProviderFactory comes with hadoop-common.
Similarly, the jceks provider file has to be on the cluster where the datasource entity exists.
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117274
-----------------------------------------------------------
On Jan. 30, 2016, 3:01 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 30, 2016, 3:01 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkat Ranganathan <n....@live.com>.
> On Feb. 1, 2016, 1:40 p.m., Venkat Ranganathan wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 211
> > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211>
> >
> > I think you are doing this for ServiceLoader META-INF entries not properly merged. We should see if we can have some maven strategy to fix this
> >
> > How will this work other filesystem solutions like WASB?
> >
> > How are we passing the principal for remote Filesystems when the file is on a remote system?
>
> Venkatesan Ramachandran wrote:
> The datasouce definition specifies the HDFS pathname for the jceks provider -- it is inferred as HDFS for now and so the conf.set(). Regarding WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, this should also work since I used the CredentiaProviderFactory comes with hadoop-common.
>
> Similarly, the jceks provider file has to be on the cluster where the datasource entity exists.
You are right. Sqoop can work with HDFS and WASB OOB - and sqoop actions when run via oozie have the jks file localized or accessed as a HDFS/WASB location as the case may be. But here the FS implementation is explicitly set for HDFS because of the implementation for other FS are not available because of some issues. May be I don't fully grok the details, but can't we make it work independent of the FS where the file is located? For example, in a HDI installation, fs.defaultFS will be a WASB location - it will not work there. It will be good to not restrict to HDFS like this as there are other filesystems used also.
Regarding the security, here there is a call to new Configuration(). It will get the current Falcon cluster configuration with the principals from the current cluster where Falcon server is running. What if the file is on a remote cluster? Don't we have to make sure we use the filesystem and principals related to that Filesystem that we pick up from the corresponding cluster entity?
- Venkat
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117274
-----------------------------------------------------------
On Jan. 29, 2016, 7:01 p.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2016, 7:01 p.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
> On Feb. 1, 2016, 9:40 p.m., Venkat Ranganathan wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 211
> > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211>
> >
> > I think you are doing this for ServiceLoader META-INF entries not properly merged. We should see if we can have some maven strategy to fix this
> >
> > How will this work other filesystem solutions like WASB?
> >
> > How are we passing the principal for remote Filesystems when the file is on a remote system?
>
> Venkatesan Ramachandran wrote:
> The datasouce definition specifies the HDFS pathname for the jceks provider -- it is inferred as HDFS for now and so the conf.set(). Regarding WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, this should also work since I used the CredentiaProviderFactory comes with hadoop-common.
>
> Similarly, the jceks provider file has to be on the cluster where the datasource entity exists.
>
> Venkat Ranganathan wrote:
> You are right. Sqoop can work with HDFS and WASB OOB - and sqoop actions when run via oozie have the jks file localized or accessed as a HDFS/WASB location as the case may be. But here the FS implementation is explicitly set for HDFS because of the implementation for other FS are not available because of some issues. May be I don't fully grok the details, but can't we make it work independent of the FS where the file is located? For example, in a HDI installation, fs.defaultFS will be a WASB location - it will not work there. It will be good to not restrict to HDFS like this as there are other filesystems used also.
>
> Regarding the security, here there is a call to new Configuration(). It will get the current Falcon cluster configuration with the principals from the current cluster where Falcon server is running. What if the file is on a remote cluster? Don't we have to make sure we use the filesystem and principals related to that Filesystem that we pick up from the corresponding cluster entity?
Let me check if it works without HDFS impl hard coded. if so, we should be ok with WASB.
Not sure I understand the issue about file on a remote cluster. Let's talk about it offline.
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117274
-----------------------------------------------------------
On Jan. 30, 2016, 3:01 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 30, 2016, 3:01 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
> On Feb. 1, 2016, 9:40 p.m., Venkat Ranganathan wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 211
> > <https://reviews.apache.org/r/42642/diff/2/?file=1227071#file1227071line211>
> >
> > I think you are doing this for ServiceLoader META-INF entries not properly merged. We should see if we can have some maven strategy to fix this
> >
> > How will this work other filesystem solutions like WASB?
> >
> > How are we passing the principal for remote Filesystems when the file is on a remote system?
>
> Venkatesan Ramachandran wrote:
> The datasouce definition specifies the HDFS pathname for the jceks provider -- it is inferred as HDFS for now and so the conf.set(). Regarding WASB, I have not tried it, but I assume Sqoop can work with WASB OOB. If so, this should also work since I used the CredentiaProviderFactory comes with hadoop-common.
>
> Similarly, the jceks provider file has to be on the cluster where the datasource entity exists.
>
> Venkat Ranganathan wrote:
> You are right. Sqoop can work with HDFS and WASB OOB - and sqoop actions when run via oozie have the jks file localized or accessed as a HDFS/WASB location as the case may be. But here the FS implementation is explicitly set for HDFS because of the implementation for other FS are not available because of some issues. May be I don't fully grok the details, but can't we make it work independent of the FS where the file is located? For example, in a HDI installation, fs.defaultFS will be a WASB location - it will not work there. It will be good to not restrict to HDFS like this as there are other filesystems used also.
>
> Regarding the security, here there is a call to new Configuration(). It will get the current Falcon cluster configuration with the principals from the current cluster where Falcon server is running. What if the file is on a remote cluster? Don't we have to make sure we use the filesystem and principals related to that Filesystem that we pick up from the corresponding cluster entity?
>
> Venkatesan Ramachandran wrote:
> Let me check if it works without HDFS impl hard coded. if so, we should be ok with WASB.
>
> Not sure I understand the issue about file on a remote cluster. Let's talk about it offline.
hard coding filesystem is superflous. removed it and made sure it works ends to end with UT/IT.
conf.set("fs.hdfs.impl", "org.apache.hadoop.hdfs.DistributedFileSystem");
Regarding the remote cluster accessing the password provider file on another cluster with kerberos -- will open a separate JIRA (https://issues.apache.org/jira/browse/FALCON-1814) to address. This feature should work with kerberos and one cluster setup.
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117274
-----------------------------------------------------------
On Jan. 30, 2016, 3:01 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 30, 2016, 3:01 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkat Ranganathan <n....@live.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117274
-----------------------------------------------------------
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java (line 199)
<https://reviews.apache.org/r/42642/#comment178425>
I think you are doing this for ServiceLoader META-INF entries not properly merged. We should see if we can have some maven strategy to fix this
How will this work other filesystem solutions like WASB?
How are we passing the principal for remote Filesystems when the file is on a remote system?
- Venkat Ranganathan
On Jan. 29, 2016, 7:01 p.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 29, 2016, 7:01 p.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/
-----------------------------------------------------------
(Updated Feb. 3, 2016, 2:08 a.m.)
Review request for Falcon.
Repository: falcon-git
Description
-------
Support password alias for database import and export using java keystore
Diffs (updated)
-----
client/src/main/resources/datasource-0.1.xsd beb82cc
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
pom.xml 12672bd
webapp/pom.xml 7ecfbaf
webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java b55d660
webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
webapp/src/test/resources/credential_provider.jceks PRE-CREATION
webapp/src/test/resources/datasource-template.xml fb7a329
webapp/src/test/resources/datasource-template1.xml PRE-CREATION
webapp/src/test/resources/datasource-template2.xml PRE-CREATION
webapp/src/test/resources/datasource-template3.xml PRE-CREATION
webapp/src/test/resources/datasource-template4.xml PRE-CREATION
webapp/src/test/resources/feed-template3.xml a6c1d6b
webapp/src/test/resources/feed-template4.xml PRE-CREATION
Diff: https://reviews.apache.org/r/42642/diff/
Testing
-------
Unit tests and Manual end to end testing both on regular and secure cluster.
Thanks,
Venkatesan Ramachandran
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
> On Feb. 1, 2016, 10:19 p.m., Balu Vellanki wrote:
> > pom.xml, line 124
> > <https://reviews.apache.org/r/42642/diff/2/?file=1227083#file1227083line124>
> >
> > This change is already committed, so patch wont apply cleanly.
Ok, let me fix it.
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117292
-----------------------------------------------------------
On Jan. 30, 2016, 3:01 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 30, 2016, 3:01 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Balu Vellanki <bv...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review117292
-----------------------------------------------------------
pom.xml (line 124)
<https://reviews.apache.org/r/42642/#comment178446>
This change is already committed, so patch wont apply cleanly.
- Balu Vellanki
On Jan. 30, 2016, 3:01 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 30, 2016, 3:01 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> addons/designer/pom.xml 28a977a
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/pom.xml 8605358
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
> common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
> common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
> falcon-regression/pom.xml 6bf3d50
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
> oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
> pom.xml a80822f
> webapp/pom.xml 7ecfbaf
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
> webapp/src/test/resources/credential_provider.jceks PRE-CREATION
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/datasource-template3.xml PRE-CREATION
> webapp/src/test/resources/datasource-template4.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/
-----------------------------------------------------------
(Updated Jan. 30, 2016, 3:01 a.m.)
Review request for Falcon.
Changes
-------
Updated review board after adding IT tests
Repository: falcon-git
Description
-------
Support password alias for database import and export using java keystore
Diffs (updated)
-----
addons/designer/pom.xml 28a977a
client/src/main/resources/datasource-0.1.xsd beb82cc
common/pom.xml 8605358
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
common/src/test/java/org/apache/falcon/entity/parser/DatasourceEntityParserTest.java 9567eab
common/src/test/resources/config/datasource/datasource-file-0.1.xml 3ee40ed
common/src/test/resources/config/datasource/datasource-file-0.2.xml PRE-CREATION
falcon-regression/pom.xml 6bf3d50
oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
oozie/src/main/java/org/apache/falcon/oozie/ImportExportCommon.java PRE-CREATION
oozie/src/main/java/org/apache/falcon/oozie/ImportWorkflowBuilder.java 4892ecb
pom.xml a80822f
webapp/pom.xml 7ecfbaf
webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
webapp/src/test/java/org/apache/falcon/util/HsqldbTestUtils.java a92629f
webapp/src/test/resources/credential_provider.jceks PRE-CREATION
webapp/src/test/resources/datasource-template.xml fb7a329
webapp/src/test/resources/datasource-template1.xml PRE-CREATION
webapp/src/test/resources/datasource-template2.xml PRE-CREATION
webapp/src/test/resources/datasource-template3.xml PRE-CREATION
webapp/src/test/resources/datasource-template4.xml PRE-CREATION
webapp/src/test/resources/feed-template3.xml a6c1d6b
webapp/src/test/resources/feed-template4.xml PRE-CREATION
Diff: https://reviews.apache.org/r/42642/diff/
Testing
-------
Unit tests and Manual end to end testing both on regular and secure cluster.
Thanks,
Venkatesan Ramachandran
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
> On Jan. 27, 2016, 4:47 p.m., Peeyush Bishnoi wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 134
> > <https://reviews.apache.org/r/42642/diff/1/?file=1205293#file1205293line134>
> >
> > Can actualPasswd be null or empty. If yes, how it is handled either by logging or throwing exception etc.
You can give an empty password text in the XML or jceks alias (but not null). The connection should work with empty password.
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review116590
-----------------------------------------------------------
On Jan. 22, 2016, 3:27 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2016, 3:27 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/AbstractTestContext.java 413dfde
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Peeyush Bishnoi <bp...@yahoo.co.in>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review116590
-----------------------------------------------------------
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java (line 133)
<https://reviews.apache.org/r/42642/#comment177634>
Can actualPasswd be null or empty. If yes, how it is handled either by logging or throwing exception etc.
- Peeyush Bishnoi
On Jan. 22, 2016, 3:27 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2016, 3:27 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/AbstractTestContext.java 413dfde
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Venkatesan Ramachandran <me...@gmail.com>.
> On Jan. 27, 2016, 1:17 a.m., Balu Vellanki wrote:
> > Please add a test that uses datasource with passwordAlias and passwordFile instead of passwordText.
Added integration test for plain text password, passowrd file and password alias. Since a HSQL server needs to be started up, adding IT test is cleaner than Unit Test.
> On Jan. 27, 2016, 1:17 a.m., Balu Vellanki wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 101
> > <https://reviews.apache.org/r/42642/diff/1/?file=1205293#file1205293line101>
> >
> > My understanding from the code is that there can be one read and one write interface. The xsd for datasource does not seem to reflect that. Please help me understand if I am wrong.
Yes, there can be one READ and one WRITE interface. These interfaces can supply their own credential. If the interfaces do not provide a credential, the default credential will be used.
> On Jan. 27, 2016, 1:17 a.m., Balu Vellanki wrote:
> > common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java, line 105
> > <https://reviews.apache.org/r/42642/diff/1/?file=1205293#file1205293line105>
> >
> > Can we add a unit test which will test this codepath. Have a test with input datasource where there is no credential for read and write interface, but there exists default credential and this default credential is used.
Added integration test to use the default credential with both the READ and WRITE interfaces.
> On Jan. 27, 2016, 1:17 a.m., Balu Vellanki wrote:
> > oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java, line 129
> > <https://reviews.apache.org/r/42642/diff/1/?file=1205298#file1205298line129>
> >
> > There is code repetition here and DatabaseImportWorkflowBuilder.java, It might help to have a class DatabaseWorkflowBuilder.java which is extended by import and export classes.
Good catch.
- Venkatesan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review116501
-----------------------------------------------------------
On Jan. 22, 2016, 3:27 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2016, 3:27 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/AbstractTestContext.java 413dfde
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>
Re: Review Request 42642: FALCON-1729 - Database Import and Export to
support password alias via Java keystore
Posted by Balu Vellanki <bv...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42642/#review116501
-----------------------------------------------------------
Please add a test that uses datasource with passwordAlias and passwordFile instead of passwordText.
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java (line 89)
<https://reviews.apache.org/r/42642/#comment177513>
It might be cleaner to say getPasswordInfo(Datasource db, Interface ifs) and have a single method. I see code being repeated for these methods.
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java (line 100)
<https://reviews.apache.org/r/42642/#comment177520>
My understanding from the code is that there can be one read and one write interface. The xsd for datasource does not seem to reflect that. Please help me understand if I am wrong.
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java (line 104)
<https://reviews.apache.org/r/42642/#comment177523>
Can we add a unit test which will test this codepath. Have a test with input datasource where there is no credential for read and write interface, but there exists default credential and this default credential is used.
common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java (line 126)
<https://reviews.apache.org/r/42642/#comment177521>
Nit : why is this called fetchPasswordInfo instead of fetchPassword?
oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java (line 128)
<https://reviews.apache.org/r/42642/#comment177522>
There is code repetition here and DatabaseImportWorkflowBuilder.java, It might help to have a class DatabaseWorkflowBuilder.java which is extended by import and export classes.
- Balu Vellanki
On Jan. 22, 2016, 3:27 a.m., Venkatesan Ramachandran wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42642/
> -----------------------------------------------------------
>
> (Updated Jan. 22, 2016, 3:27 a.m.)
>
>
> Review request for Falcon.
>
>
> Repository: falcon-git
>
>
> Description
> -------
>
> Support password alias for database import and export using java keystore
>
>
> Diffs
> -----
>
> client/src/main/resources/datasource-0.1.xsd beb82cc
> common/src/main/java/org/apache/falcon/entity/DatasourceHelper.java 1f1a193
> common/src/main/java/org/apache/falcon/entity/parser/DatasourceEntityParser.java e58b1e9
> common/src/main/java/org/apache/falcon/security/CredentialProviderHelper.java PRE-CREATION
> common/src/main/java/org/apache/falcon/util/HdfsClassLoader.java 786ffea
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseExportWorkflowBuilder.java f1fb337
> oozie/src/main/java/org/apache/falcon/oozie/DatabaseImportWorkflowBuilder.java 19fa931
> webapp/src/test/java/org/apache/falcon/lifecycle/FeedImportIT.java 1fbad88
> webapp/src/test/java/org/apache/falcon/resource/AbstractTestContext.java 413dfde
> webapp/src/test/java/org/apache/falcon/resource/TestContext.java 321a5cf
> webapp/src/test/resources/datasource-template.xml fb7a329
> webapp/src/test/resources/datasource-template1.xml PRE-CREATION
> webapp/src/test/resources/datasource-template2.xml PRE-CREATION
> webapp/src/test/resources/feed-template3.xml a6c1d6b
> webapp/src/test/resources/feed-template4.xml PRE-CREATION
>
> Diff: https://reviews.apache.org/r/42642/diff/
>
>
> Testing
> -------
>
> Unit tests and Manual end to end testing both on regular and secure cluster.
>
>
> Thanks,
>
> Venkatesan Ramachandran
>
>