You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ranger.apache.org by Qiang Zhang <zh...@zte.com.cn> on 2017/09/30 07:19:42 UTC
Review Request 62710: RANGER-1810:Ranger supports plugin to enable,
monitor and manage apache Sqoop2
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
-----------------------------------------------------------
Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810
Repository: ranger
Description
-------
Apache Sqoop2 is a tool designed for efficiently transferring data betweeen structured, semi-structured and unstructured data sources. Relational databases are examples of structured data sources with well defined schema for the data they store. Cassandra, Hbase are examples of semi-structured data sources and HDFS is an example of unstructured data source that Sqoop can support.
Ranger should support plugin to enable, monitor and manage apache Sqoop2
Diffs
-----
agents-common/scripts/enable-agent.sh d31a264
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 58cdd35
agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
plugin-sqoop/.gitignore PRE-CREATION
plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
plugin-sqoop/pom.xml PRE-CREATION
plugin-sqoop/scripts/install.properties PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
pom.xml 3958014
ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
src/main/assembly/admin-web.xml 0e97818
src/main/assembly/plugin-sqoop.xml PRE-CREATION
Diff: https://reviews.apache.org/r/62710/diff/1/
Testing
-------
Tested
Thanks,
Qiang Zhang
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Qiang Zhang <zh...@zte.com.cn>.
> On 十月 2, 2017, 6:46 p.m., Ramesh Mani wrote:
> > plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
> > Lines 99 (patched)
> > <https://reviews.apache.org/r/62710/diff/1/?file=1841282#file1841282line99>
> >
> > can privilege be null? Is it test for this case?
> > Please add some unit test also for various scenarios.
OK,I will submit another patch for some unit test.
And according to Sqoop AuthorizationEngine.checkPrivilege method,
the privilege would not be null, but privileges would be an empty list.
Sqoop code reference, please notice SQOOP-2256 commont:
org.apache.sqoop.security.authorization.AuthorizationEngine.checkPrivilege(MPrivilege...)
private static void checkPrivilege(MPrivilege... privileges) {
AuthorizationHandler handler = AuthorizationManager.getAuthorizationHandler();
UserGroupInformation user = HttpUserGroupInformation.get();
String user_name = user == null ? StringUtils.EMPTY : user.getShortUserName();
MPrincipal principal = new MPrincipal(user_name, MPrincipal.TYPE.USER);
// SQOOP-2256: Hack code, do not check privilege when the user is the creator
// If the user is the owner/creator of this resource, then privilege will
// not be checked. It is a hack code for the time being. The concept of
// "Owner" will be added in the future and this code will be removed.
ArrayList<MPrivilege> privilegesNeedCheck = new ArrayList<MPrivilege>();
for (MPrivilege privilege : privileges) {
Repository repository = RepositoryManager.getInstance().getRepository();
if (MResource.TYPE.LINK.name().equalsIgnoreCase(privilege.getResource().getType())) {
MLink link = repository.findLink(Long.valueOf(privilege.getResource().getName()));
if (!user_name.equals(link.getCreationUser())) {
privilegesNeedCheck.add(privilege);
}
} else if (MResource.TYPE.JOB.name().equalsIgnoreCase(privilege.getResource().getType())) {
MJob job = repository.findJob(Long.valueOf(privilege.getResource().getName()));
if (!user_name.equals(job.getCreationUser())) {
privilegesNeedCheck.add(privilege);
}
} else {
privilegesNeedCheck.add(privilege);
}
}
handler.checkPrivileges(principal, privilegesNeedCheck);
}
}
- Qiang
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186865
-----------------------------------------------------------
On 十月 12, 2017, 9:02 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> -----------------------------------------------------------
>
> (Updated 十月 12, 2017, 9:02 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.It successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it supports sqoop authorization. So we should develop sqoop plugin to enable, monitor and manage apache Sqoop2.
>
> Our test specialists have rigorously tested this feature.
>
>
> Diffs
> -----
>
> agents-common/scripts/enable-agent.sh d31a264
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
> agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
> plugin-sqoop/.gitignore PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
> plugin-sqoop/pom.xml PRE-CREATION
> plugin-sqoop/scripts/install.properties PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
> pom.xml 3958014
> ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
> ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> src/main/assembly/admin-web.xml 0e97818
> src/main/assembly/plugin-sqoop.xml PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62710/diff/2/
>
>
> Testing
> -------
>
> Our test specialists have rigorously tested this feature.
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Ramesh Mani <rm...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186865
-----------------------------------------------------------
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java
Lines 99 (patched)
<https://reviews.apache.org/r/62710/#comment263728>
can privilege be null? Is it test for this case?
Please add some unit test also for various scenarios.
- Ramesh Mani
On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 8:34 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.It successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it supports sqoop authorization. So we should develop sqoop plugin to enable, monitor and manage apache Sqoop2.
>
> Our test specialists have rigorously tested this feature.
>
>
> Diffs
> -----
>
> agents-common/scripts/enable-agent.sh d31a264
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 58cdd35
> agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
> plugin-sqoop/.gitignore PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
> plugin-sqoop/pom.xml PRE-CREATION
> plugin-sqoop/scripts/install.properties PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
> pom.xml 3958014
> ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
> ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> src/main/assembly/admin-web.xml 0e97818
> src/main/assembly/plugin-sqoop.xml PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62710/diff/1/
>
>
> Testing
> -------
>
> Our test specialists have rigorously tested this feature.
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Alejandro Fernandez <af...@hortonworks.com>.
> On Oct. 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> >
Awesome contribution.
- Alejandro
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186860
-----------------------------------------------------------
On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 8:34 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.It successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it supports sqoop authorization. So we should develop sqoop plugin to enable, monitor and manage apache Sqoop2.
>
> Our test specialists have rigorously tested this feature.
>
>
> Diffs
> -----
>
> agents-common/scripts/enable-agent.sh d31a264
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 58cdd35
> agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
> plugin-sqoop/.gitignore PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
> plugin-sqoop/pom.xml PRE-CREATION
> plugin-sqoop/scripts/install.properties PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
> pom.xml 3958014
> ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
> ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> src/main/assembly/admin-web.xml 0e97818
> src/main/assembly/plugin-sqoop.xml PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62710/diff/1/
>
>
> Testing
> -------
>
> Our test specialists have rigorously tested this feature.
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Qiang Zhang <zh...@zte.com.cn>.
> On 十月 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> > plugin-sqoop/scripts/install.properties
> > Lines 39 (patched)
> > <https://reviews.apache.org/r/62710/diff/1/?file=1841281#file1841281line39>
> >
> > Should all of these commented lines be removed?
These commented lines should not be removed, they are useful for user configuration parameters.
> On 十月 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
> > Lines 85 (patched)
> > <https://reviews.apache.org/r/62710/diff/1/?file=1841284#file1841284line85>
> >
> > If any errors are found, should this raise an exception instead of just logging?
There is no need to throw an exception, and it doesn't affect creating policies in Ranger.
Notice:" You can still save the repository and start creating policies, but you would not be able to use autocomplete for resource names. Check ranger_admin.log for more info."
> On 十月 2, 2017, 6:13 p.m., Alejandro Fernandez wrote:
> > plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
> > Lines 103 (patched)
> > <https://reviews.apache.org/r/62710/diff/1/?file=1841284#file1841284line103>
> >
> > Are there any paths in which the response object needs to be .closed()?
The response will be closed in method org.apache.ranger.services.sqoop.client.SqoopClient.getSqoopConnectorResponse(ClientResponse) finally block.
- Qiang
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186860
-----------------------------------------------------------
On 十月 12, 2017, 9:02 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> -----------------------------------------------------------
>
> (Updated 十月 12, 2017, 9:02 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, Venkat Ranganathan, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.It successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it supports sqoop authorization. So we should develop sqoop plugin to enable, monitor and manage apache Sqoop2.
>
> Our test specialists have rigorously tested this feature.
>
>
> Diffs
> -----
>
> agents-common/scripts/enable-agent.sh d31a264
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
> agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
> plugin-sqoop/.gitignore PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
> plugin-sqoop/pom.xml PRE-CREATION
> plugin-sqoop/scripts/install.properties PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
> pom.xml 3958014
> ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
> ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> src/main/assembly/admin-web.xml 0e97818
> src/main/assembly/plugin-sqoop.xml PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62710/diff/2/
>
>
> Testing
> -------
>
> Our test specialists have rigorously tested this feature.
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Alejandro Fernandez <af...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/#review186860
-----------------------------------------------------------
plugin-sqoop/scripts/install.properties
Lines 39 (patched)
<https://reviews.apache.org/r/62710/#comment263707>
Should all of these commented lines be removed?
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
Lines 49 (patched)
<https://reviews.apache.org/r/62710/#comment263704>
Add JavaDoc to all new classes and methods.
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
Lines 85 (patched)
<https://reviews.apache.org/r/62710/#comment263703>
If any errors are found, should this raise an exception instead of just logging?
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java
Lines 103 (patched)
<https://reviews.apache.org/r/62710/#comment263705>
Are there any paths in which the response object needs to be .closed()?
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java
Lines 21 (patched)
<https://reviews.apache.org/r/62710/#comment263706>
Add JavaDoc
- Alejandro Fernandez
On Sept. 30, 2017, 8:34 a.m., Qiang Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62710/
> -----------------------------------------------------------
>
> (Updated Sept. 30, 2017, 8:34 a.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-1810
> https://issues.apache.org/jira/browse/RANGER-1810
>
>
> Repository: ranger
>
>
> Description
> -------
>
> Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.It successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project.
> The Ranger will further expand the influence in the hadoop ecosystem if it supports sqoop authorization. So we should develop sqoop plugin to enable, monitor and manage apache Sqoop2.
>
> Our test specialists have rigorously tested this feature.
>
>
> Diffs
> -----
>
> agents-common/scripts/enable-agent.sh d31a264
> agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
> agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 58cdd35
> agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
> plugin-sqoop/.gitignore PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
> plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
> plugin-sqoop/pom.xml PRE-CREATION
> plugin-sqoop/scripts/install.properties PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
> plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
> pom.xml 3958014
> ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
> ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
> ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
> src/main/assembly/admin-web.xml 0e97818
> src/main/assembly/plugin-sqoop.xml PRE-CREATION
>
>
> Diff: https://reviews.apache.org/r/62710/diff/1/
>
>
> Testing
> -------
>
> Our test specialists have rigorously tested this feature.
>
>
> Thanks,
>
> Qiang Zhang
>
>
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Qiang Zhang <zh...@zte.com.cn>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
-----------------------------------------------------------
(Updated Sept. 30, 2017, 8:34 a.m.)
Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810
Repository: ranger
Description (updated)
-------
Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases. You can use Sqoop to import data from external structured datastores into Hadoop Distributed File System or related systems like Hive and HBase. Conversely, Sqoop can be used to extract data from Hadoop and export it to external structured datastores such as relational databases and enterprise data warehouses.It successfully graduated from the Incubator in March of 2012 and is now a Top-Level Apache project.
The Ranger will further expand the influence in the hadoop ecosystem if it supports sqoop authorization. So we should develop sqoop plugin to enable, monitor and manage apache Sqoop2.
Our test specialists have rigorously tested this feature.
Diffs
-----
agents-common/scripts/enable-agent.sh d31a264
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 58cdd35
agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
plugin-sqoop/.gitignore PRE-CREATION
plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
plugin-sqoop/pom.xml PRE-CREATION
plugin-sqoop/scripts/install.properties PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
pom.xml 3958014
ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
src/main/assembly/admin-web.xml 0e97818
src/main/assembly/plugin-sqoop.xml PRE-CREATION
Diff: https://reviews.apache.org/r/62710/diff/1/
Testing
-------
Our test specialists have rigorously tested this feature.
Thanks,
Qiang Zhang
Re: Review Request 62710: RANGER-1810:Ranger supports plugin to
enable, monitor and manage apache Sqoop2
Posted by Qiang Zhang <zh...@zte.com.cn>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62710/
-----------------------------------------------------------
(Updated Sept. 30, 2017, 8:17 a.m.)
Review request for ranger, Ankita Sinha, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Madhan Neethiraj, pengjianhua, Ramesh Mani, Selvamohan Neethiraj, sam rome, and Velmurugan Periasamy.
Bugs: RANGER-1810
https://issues.apache.org/jira/browse/RANGER-1810
Repository: ranger
Description (updated)
-------
Apache Sqoop2 is a tool designed for efficiently transferring data betweeen structured, semi-structured and unstructured data sources. Relational databases are examples of structured data sources with well defined schema for the data they store. Cassandra, Hbase are examples of semi-structured data sources and HDFS is an example of unstructured data source that Sqoop can support.
Ranger should support plugin to enable, monitor and manage apache Sqoop2
Our test specialists have rigorously tested this feature.
Diffs
-----
agents-common/scripts/enable-agent.sh d31a264
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java 9463ab8
agents-common/src/main/java/org/apache/ranger/plugin/util/PasswordUtils.java 58cdd35
agents-common/src/main/resources/service-defs/ranger-servicedef-sqoop.json PRE-CREATION
plugin-sqoop/.gitignore PRE-CREATION
plugin-sqoop/conf/ranger-policymgr-ssl-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-policymgr-ssl.xml PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-audit-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-audit.xml PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-security-changes.cfg PRE-CREATION
plugin-sqoop/conf/ranger-sqoop-security.xml PRE-CREATION
plugin-sqoop/pom.xml PRE-CREATION
plugin-sqoop/scripts/install.properties PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/RangerServiceSqoop.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopClient.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/SqoopResourceMgr.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorResponse.java PRE-CREATION
plugin-sqoop/src/main/java/org/apache/ranger/services/sqoop/client/json/model/SqoopConnectorsResponse.java PRE-CREATION
pom.xml 3958014
ranger-sqoop-plugin-shim/.gitignore PRE-CREATION
ranger-sqoop-plugin-shim/pom.xml PRE-CREATION
ranger-sqoop-plugin-shim/src/main/java/org/apache/ranger/authorization/sqoop/authorizer/RangerSqoopAuthorizer.java PRE-CREATION
src/main/assembly/admin-web.xml 0e97818
src/main/assembly/plugin-sqoop.xml PRE-CREATION
Diff: https://reviews.apache.org/r/62710/diff/1/
Testing (updated)
-------
Our test specialists have rigorously tested this feature.
Thanks,
Qiang Zhang