You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Dian Fu <di...@gmail.com> on 2015/12/10 09:57:03 UTC
Review Request 41197: Sqoop2: Improve ConnectorClassLoader to support
loading classes from the dependencies inside the connector jar
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41197/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-2635
https://issues.apache.org/jira/browse/SQOOP-2635
Repository: sqoop-sqoop2
Description
-------
As discussed in SQOOP-2634, we need to improve ConnectorClassLoader to support loading classes which are from the dependencies inside the connector jar.
Diffs
-----
common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java PRE-CREATION
common/src/main/java/org/apache/sqoop/classloader/ConnectorURLFactory.java PRE-CREATION
common/src/main/java/org/apache/sqoop/classloader/URLFactory.java PRE-CREATION
common/src/main/java/org/apache/sqoop/utils/ConnectorClassLoader.java 4c42a78
common/src/main/resources/org.apache.sqoop.connector-classloader.properties e2936a9
common/src/test/java/org/apache/sqoop/classloader/TestConnectorClassLoader.java PRE-CREATION
common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java 674ae6a
common/src/test/java/org/apache/sqoop/utils/TestConnectorClassLoader.java 1ec1bb4
common/src/test/resources/TestConnectorClassLoader/A.java PRE-CREATION
common/src/test/resources/TestConnectorClassLoader/lib/B.java PRE-CREATION
common/src/test/resources/TestConnectorClassLoader/lib/resource-dep.txt PRE-CREATION
common/src/test/resources/TestConnectorClassLoader/lib/resource.txt PRE-CREATION
common/src/test/resources/TestConnectorClassLoader/resource.txt PRE-CREATION
Diff: https://reviews.apache.org/r/41197/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 41197: Sqoop2: Improve ConnectorClassLoader to
support loading classes from the dependencies inside the connector jar
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41197/#review110735
-----------------------------------------------------------
Ship it!
Ship It!
- Jarek Cecho
On Dec. 16, 2015, 1:26 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41197/
> -----------------------------------------------------------
>
> (Updated Dec. 16, 2015, 1:26 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2635
> https://issues.apache.org/jira/browse/SQOOP-2635
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> As discussed in SQOOP-2634, we need to improve ConnectorClassLoader to support loading classes which are from the dependencies inside the connector jar.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/classloader/ConnectorURLFactory.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/classloader/URLFactory.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/utils/ConnectorClassLoader.java 4c42a78
> common/src/main/resources/org.apache.sqoop.connector-classloader.properties e2936a9
> common/src/test/java/org/apache/sqoop/classloader/TestConnectorClassLoader.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java 674ae6a
> common/src/test/java/org/apache/sqoop/utils/TestConnectorClassLoader.java 1ec1bb4
> common/src/test/resources/TestConnectorClassLoader/A.java PRE-CREATION
> common/src/test/resources/TestConnectorClassLoader/lib/B.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/41197/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 41197: Sqoop2: Improve ConnectorClassLoader to
support loading classes from the dependencies inside the connector jar
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41197/
-----------------------------------------------------------
(Updated Dec. 16, 2015, 1:26 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2635
https://issues.apache.org/jira/browse/SQOOP-2635
Repository: sqoop-sqoop2
Description
-------
As discussed in SQOOP-2634, we need to improve ConnectorClassLoader to support loading classes which are from the dependencies inside the connector jar.
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java PRE-CREATION
common/src/main/java/org/apache/sqoop/classloader/ConnectorURLFactory.java PRE-CREATION
common/src/main/java/org/apache/sqoop/classloader/URLFactory.java PRE-CREATION
common/src/main/java/org/apache/sqoop/utils/ConnectorClassLoader.java 4c42a78
common/src/main/resources/org.apache.sqoop.connector-classloader.properties e2936a9
common/src/test/java/org/apache/sqoop/classloader/TestConnectorClassLoader.java PRE-CREATION
common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java 674ae6a
common/src/test/java/org/apache/sqoop/utils/TestConnectorClassLoader.java 1ec1bb4
common/src/test/resources/TestConnectorClassLoader/A.java PRE-CREATION
common/src/test/resources/TestConnectorClassLoader/lib/B.java PRE-CREATION
Diff: https://reviews.apache.org/r/41197/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 41197: Sqoop2: Improve ConnectorClassLoader to
support loading classes from the dependencies inside the connector jar
Posted by Dian Fu <di...@gmail.com>.
> On Dec. 15, 2015, 2:25 p.m., Jarek Cecho wrote:
> > Quite advanced patch Dian! I read it through and I like the approach of not spilling the jars to the disk and rather keep them in memory. I have to say that I'm not 100% sure on all the code but it seems in the right direction, so I'm happy to +1 and commit it.
> >
> > One thing that I've noticed is that compilation on JDK8 is failing with:
> >
> > [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile (default-compile) on project sqoop-common: Compilation failure
> > [ERROR] /Users/jarcec/apache/sqoop2/common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java:[228,11] error: cannot infer type arguments for CompoundEnumeration<>
> >
> > Could you take a look?
Hi Jarcec,
Thanks a lot for the review and +1 of the patch.:) The functionality of this ConnectorClassLoader will be fully tested in SQOOP-2634 which dependens on this ConnectorClassLoader. Actually I have tested locally that all the integration tests passed for SQOOP-2634 based on this patch.
Good catch of the compilation failure. Fixed in the new patch. :)
- Dian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41197/#review110493
-----------------------------------------------------------
On Dec. 16, 2015, 1:26 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41197/
> -----------------------------------------------------------
>
> (Updated Dec. 16, 2015, 1:26 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2635
> https://issues.apache.org/jira/browse/SQOOP-2635
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> As discussed in SQOOP-2634, we need to improve ConnectorClassLoader to support loading classes which are from the dependencies inside the connector jar.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/classloader/ConnectorURLFactory.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/classloader/URLFactory.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/utils/ConnectorClassLoader.java 4c42a78
> common/src/main/resources/org.apache.sqoop.connector-classloader.properties e2936a9
> common/src/test/java/org/apache/sqoop/classloader/TestConnectorClassLoader.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java 674ae6a
> common/src/test/java/org/apache/sqoop/utils/TestConnectorClassLoader.java 1ec1bb4
> common/src/test/resources/TestConnectorClassLoader/A.java PRE-CREATION
> common/src/test/resources/TestConnectorClassLoader/lib/B.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/41197/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 41197: Sqoop2: Improve ConnectorClassLoader to
support loading classes from the dependencies inside the connector jar
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41197/#review110493
-----------------------------------------------------------
Quite advanced patch Dian! I read it through and I like the approach of not spilling the jars to the disk and rather keep them in memory. I have to say that I'm not 100% sure on all the code but it seems in the right direction, so I'm happy to +1 and commit it.
One thing that I've noticed is that compilation on JDK8 is failing with:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile (default-compile) on project sqoop-common: Compilation failure
[ERROR] /Users/jarcec/apache/sqoop2/common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java:[228,11] error: cannot infer type arguments for CompoundEnumeration<>
Could you take a look?
- Jarek Cecho
On Dec. 10, 2015, 11:48 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41197/
> -----------------------------------------------------------
>
> (Updated Dec. 10, 2015, 11:48 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2635
> https://issues.apache.org/jira/browse/SQOOP-2635
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> As discussed in SQOOP-2634, we need to improve ConnectorClassLoader to support loading classes which are from the dependencies inside the connector jar.
>
>
> Diffs
> -----
>
> common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/classloader/ConnectorURLFactory.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/classloader/URLFactory.java PRE-CREATION
> common/src/main/java/org/apache/sqoop/utils/ConnectorClassLoader.java 4c42a78
> common/src/main/resources/org.apache.sqoop.connector-classloader.properties e2936a9
> common/src/test/java/org/apache/sqoop/classloader/TestConnectorClassLoader.java PRE-CREATION
> common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java 674ae6a
> common/src/test/java/org/apache/sqoop/utils/TestConnectorClassLoader.java 1ec1bb4
> common/src/test/resources/TestConnectorClassLoader/A.java PRE-CREATION
> common/src/test/resources/TestConnectorClassLoader/lib/B.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/41197/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 41197: Sqoop2: Improve ConnectorClassLoader to
support loading classes from the dependencies inside the connector jar
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41197/
-----------------------------------------------------------
(Updated Dec. 10, 2015, 11:48 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2635
https://issues.apache.org/jira/browse/SQOOP-2635
Repository: sqoop-sqoop2
Description
-------
As discussed in SQOOP-2634, we need to improve ConnectorClassLoader to support loading classes which are from the dependencies inside the connector jar.
Diffs (updated)
-----
common/src/main/java/org/apache/sqoop/classloader/ConnectorClassLoader.java PRE-CREATION
common/src/main/java/org/apache/sqoop/classloader/ConnectorURLFactory.java PRE-CREATION
common/src/main/java/org/apache/sqoop/classloader/URLFactory.java PRE-CREATION
common/src/main/java/org/apache/sqoop/utils/ConnectorClassLoader.java 4c42a78
common/src/main/resources/org.apache.sqoop.connector-classloader.properties e2936a9
common/src/test/java/org/apache/sqoop/classloader/TestConnectorClassLoader.java PRE-CREATION
common/src/test/java/org/apache/sqoop/utils/TestClassUtils.java 674ae6a
common/src/test/java/org/apache/sqoop/utils/TestConnectorClassLoader.java 1ec1bb4
common/src/test/resources/TestConnectorClassLoader/A.java PRE-CREATION
common/src/test/resources/TestConnectorClassLoader/lib/B.java PRE-CREATION
Diff: https://reviews.apache.org/r/41197/diff/
Testing
-------
Thanks,
Dian Fu