You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Dian Fu <di...@gmail.com> on 2015/11/05 10:50:51 UTC
Review Request 39970: Sqoop2: Remove unnecessary dependencies of sqoop
server module
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/
-----------------------------------------------------------
Review request for Sqoop.
Bugs: SQOOP-2662
https://issues.apache.org/jira/browse/SQOOP-2662
Repository: sqoop-sqoop2
Description
-------
Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
Diffs
-----
connector/connector-ftp/pom.xml 9a70dfc
connector/connector-generic-jdbc/pom.xml 7bdeddb
connector/connector-hdfs/pom.xml a28989c
connector/connector-kafka/pom.xml e0f0684
connector/connector-kite/pom.xml 41441f5
connector/connector-sftp/pom.xml 44e64b1
dist/pom.xml 315a018
pom.xml ab505f4
repository/repository-common/pom.xml c740117
repository/repository-derby/pom.xml a28b036
repository/repository-mysql/pom.xml 066d76b
repository/repository-postgresql/pom.xml 6550bc2
server/pom.xml ca068e0
submission/mapreduce/pom.xml f56437b
Diff: https://reviews.apache.org/r/39970/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Dian Fu <di...@gmail.com>.
> On Nov. 5, 2015, 5 p.m., Jarek Cecho wrote:
> > Hi Dian,
> > you're indeed correct that the server module depends on all other modules just for the sake of packaging - there are no code dependencies on any of the connectors or repository implementations.
> >
> > The reason we did it this way is that it's easier to maintain few dependencies then copy huge blocks of XML code around to a) copy dependencies to target/ and b) copying them to the distribution tarball.
> >
> > I would prefer not to overly optimize this portion of the packaging code because it's very old and if we want to make major changes, then we should migrate to maven assembly plugin [1] instead (which is the recommended way of doing packaging in maven).
> >
> > Jarcec
> >
> > Links:
> > 1: http://maven.apache.org/plugins/maven-assembly-plugin/
>
> Dian Fu wrote:
> Hi Jarcec,
> Thanks for the comments and suggestions. According to the latest design of SQOOP-2634, it's necessary to separate the dependencies of each connector. While the current packaging approach makes it difficult to do it. Thanks for the suggestion of maven assembly plugin. I'll update the patch with maven assembly plugin.
>
> Jarek Cecho wrote:
> Let's not push breaking changes because of SQOOP-2634 yet. The classpath isolation design document hasn't been accepted so we shouldn't execute such changes yet.
SQOOP-2634 is just a motivation of the changes here. Even if we put SQOOP-2634 aside, the changes here are still necessary.
- Dian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/#review105259
-----------------------------------------------------------
On Nov. 7, 2015, 2:47 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39970/
> -----------------------------------------------------------
>
> (Updated Nov. 7, 2015, 2:47 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2662
> https://issues.apache.org/jira/browse/SQOOP-2662
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
>
>
> Diffs
> -----
>
> connector/connector-ftp/pom.xml 9a70dfc
> connector/connector-generic-jdbc/pom.xml 7bdeddb
> connector/connector-hdfs/pom.xml a28989c
> connector/connector-kafka/pom.xml e0f0684
> connector/connector-kite/pom.xml 41441f5
> connector/connector-oracle-jdbc/pom.xml 325790d
> connector/connector-sftp/pom.xml 44e64b1
> dist/pom.xml 315a018
> dist/src/main/assembly/bin.xml PRE-CREATION
> dist/src/main/assembly/src.xml PRE-CREATION
> pom.xml f33958c
> repository/repository-common/pom.xml c740117
> repository/repository-derby/pom.xml a28b036
> repository/repository-mysql/pom.xml 066d76b
> repository/repository-postgresql/pom.xml 6550bc2
> server/pom.xml 370a6a2
> submission/mapreduce/pom.xml f56437b
> test/pom.xml 4e1e197
>
> Diff: https://reviews.apache.org/r/39970/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Jarek Cecho <ja...@apache.org>.
> On Nov. 5, 2015, 5 p.m., Jarek Cecho wrote:
> > Hi Dian,
> > you're indeed correct that the server module depends on all other modules just for the sake of packaging - there are no code dependencies on any of the connectors or repository implementations.
> >
> > The reason we did it this way is that it's easier to maintain few dependencies then copy huge blocks of XML code around to a) copy dependencies to target/ and b) copying them to the distribution tarball.
> >
> > I would prefer not to overly optimize this portion of the packaging code because it's very old and if we want to make major changes, then we should migrate to maven assembly plugin [1] instead (which is the recommended way of doing packaging in maven).
> >
> > Jarcec
> >
> > Links:
> > 1: http://maven.apache.org/plugins/maven-assembly-plugin/
>
> Dian Fu wrote:
> Hi Jarcec,
> Thanks for the comments and suggestions. According to the latest design of SQOOP-2634, it's necessary to separate the dependencies of each connector. While the current packaging approach makes it difficult to do it. Thanks for the suggestion of maven assembly plugin. I'll update the patch with maven assembly plugin.
Let's not push breaking changes because of SQOOP-2634 yet. The classpath isolation design document hasn't been accepted so we shouldn't execute such changes yet.
- Jarek
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/#review105259
-----------------------------------------------------------
On Nov. 6, 2015, 2:34 p.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39970/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2015, 2:34 p.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2662
> https://issues.apache.org/jira/browse/SQOOP-2662
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
>
>
> Diffs
> -----
>
> connector/connector-ftp/pom.xml 9a70dfc
> connector/connector-generic-jdbc/pom.xml 7bdeddb
> connector/connector-hdfs/pom.xml a28989c
> connector/connector-kafka/pom.xml e0f0684
> connector/connector-kite/pom.xml 41441f5
> connector/connector-oracle-jdbc/pom.xml 325790d
> connector/connector-sftp/pom.xml 44e64b1
> dist/pom.xml 315a018
> dist/src/main/assembly/bin.xml PRE-CREATION
> dist/src/main/assembly/src.xml PRE-CREATION
> pom.xml f33958c
> repository/repository-common/pom.xml c740117
> repository/repository-derby/pom.xml a28b036
> repository/repository-mysql/pom.xml 066d76b
> repository/repository-postgresql/pom.xml 6550bc2
> server/pom.xml 370a6a2
> submission/mapreduce/pom.xml f56437b
> test/pom.xml 4e1e197
>
> Diff: https://reviews.apache.org/r/39970/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Dian Fu <di...@gmail.com>.
> On Nov. 5, 2015, 5 p.m., Jarek Cecho wrote:
> > Hi Dian,
> > you're indeed correct that the server module depends on all other modules just for the sake of packaging - there are no code dependencies on any of the connectors or repository implementations.
> >
> > The reason we did it this way is that it's easier to maintain few dependencies then copy huge blocks of XML code around to a) copy dependencies to target/ and b) copying them to the distribution tarball.
> >
> > I would prefer not to overly optimize this portion of the packaging code because it's very old and if we want to make major changes, then we should migrate to maven assembly plugin [1] instead (which is the recommended way of doing packaging in maven).
> >
> > Jarcec
> >
> > Links:
> > 1: http://maven.apache.org/plugins/maven-assembly-plugin/
Hi Jarcec,
Thanks for the comments and suggestions. According to the latest design of SQOOP-2634, it's necessary to separate the dependencies of each connector. While the current packaging approach makes it difficult to do it. Thanks for the suggestion of maven assembly plugin. I'll update the patch with maven assembly plugin.
- Dian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/#review105259
-----------------------------------------------------------
On Nov. 5, 2015, 11:47 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39970/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2015, 11:47 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2662
> https://issues.apache.org/jira/browse/SQOOP-2662
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
>
>
> Diffs
> -----
>
> connector/connector-ftp/pom.xml 9a70dfc
> connector/connector-generic-jdbc/pom.xml 7bdeddb
> connector/connector-hdfs/pom.xml a28989c
> connector/connector-kafka/pom.xml e0f0684
> connector/connector-kite/pom.xml 41441f5
> connector/connector-sftp/pom.xml 44e64b1
> dist/pom.xml 315a018
> pom.xml ab505f4
> repository/repository-common/pom.xml c740117
> repository/repository-derby/pom.xml a28b036
> repository/repository-mysql/pom.xml 066d76b
> repository/repository-postgresql/pom.xml 6550bc2
> server/pom.xml ca068e0
> submission/mapreduce/pom.xml f56437b
> test/pom.xml 4e1e197
>
> Diff: https://reviews.apache.org/r/39970/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Jarek Cecho <ja...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/#review105259
-----------------------------------------------------------
Hi Dian,
you're indeed correct that the server module depends on all other modules just for the sake of packaging - there are no code dependencies on any of the connectors or repository implementations.
The reason we did it this way is that it's easier to maintain few dependencies then copy huge blocks of XML code around to a) copy dependencies to target/ and b) copying them to the distribution tarball.
I would prefer not to overly optimize this portion of the packaging code because it's very old and if we want to make major changes, then we should migrate to maven assembly plugin [1] instead (which is the recommended way of doing packaging in maven).
Jarcec
Links:
1: http://maven.apache.org/plugins/maven-assembly-plugin/
- Jarek Cecho
On Nov. 5, 2015, 11:47 a.m., Dian Fu wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39970/
> -----------------------------------------------------------
>
> (Updated Nov. 5, 2015, 11:47 a.m.)
>
>
> Review request for Sqoop.
>
>
> Bugs: SQOOP-2662
> https://issues.apache.org/jira/browse/SQOOP-2662
>
>
> Repository: sqoop-sqoop2
>
>
> Description
> -------
>
> Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
>
>
> Diffs
> -----
>
> connector/connector-ftp/pom.xml 9a70dfc
> connector/connector-generic-jdbc/pom.xml 7bdeddb
> connector/connector-hdfs/pom.xml a28989c
> connector/connector-kafka/pom.xml e0f0684
> connector/connector-kite/pom.xml 41441f5
> connector/connector-sftp/pom.xml 44e64b1
> dist/pom.xml 315a018
> pom.xml ab505f4
> repository/repository-common/pom.xml c740117
> repository/repository-derby/pom.xml a28b036
> repository/repository-mysql/pom.xml 066d76b
> repository/repository-postgresql/pom.xml 6550bc2
> server/pom.xml ca068e0
> submission/mapreduce/pom.xml f56437b
> test/pom.xml 4e1e197
>
> Diff: https://reviews.apache.org/r/39970/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Dian Fu
>
>
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/
-----------------------------------------------------------
(Updated Nov. 7, 2015, 3:53 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2662
https://issues.apache.org/jira/browse/SQOOP-2662
Repository: sqoop-sqoop2
Description
-------
Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
Diffs (updated)
-----
connector/connector-ftp/pom.xml 9a70dfc
connector/connector-generic-jdbc/pom.xml 7bdeddb
connector/connector-hdfs/pom.xml a28989c
connector/connector-kafka/pom.xml e0f0684
connector/connector-kite/pom.xml 41441f5
connector/connector-oracle-jdbc/pom.xml 325790d
connector/connector-sftp/pom.xml 44e64b1
dist/pom.xml 315a018
dist/src/main/assembly/bin.xml PRE-CREATION
dist/src/main/assembly/src.xml PRE-CREATION
pom.xml f33958c
repository/repository-common/pom.xml c740117
repository/repository-derby/pom.xml a28b036
repository/repository-mysql/pom.xml 066d76b
repository/repository-postgresql/pom.xml 6550bc2
server/pom.xml 370a6a2
submission/mapreduce/pom.xml f56437b
test/pom.xml 4e1e197
Diff: https://reviews.apache.org/r/39970/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/
-----------------------------------------------------------
(Updated Nov. 7, 2015, 2:47 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2662
https://issues.apache.org/jira/browse/SQOOP-2662
Repository: sqoop-sqoop2
Description
-------
Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
Diffs (updated)
-----
connector/connector-ftp/pom.xml 9a70dfc
connector/connector-generic-jdbc/pom.xml 7bdeddb
connector/connector-hdfs/pom.xml a28989c
connector/connector-kafka/pom.xml e0f0684
connector/connector-kite/pom.xml 41441f5
connector/connector-oracle-jdbc/pom.xml 325790d
connector/connector-sftp/pom.xml 44e64b1
dist/pom.xml 315a018
dist/src/main/assembly/bin.xml PRE-CREATION
dist/src/main/assembly/src.xml PRE-CREATION
pom.xml f33958c
repository/repository-common/pom.xml c740117
repository/repository-derby/pom.xml a28b036
repository/repository-mysql/pom.xml 066d76b
repository/repository-postgresql/pom.xml 6550bc2
server/pom.xml 370a6a2
submission/mapreduce/pom.xml f56437b
test/pom.xml 4e1e197
Diff: https://reviews.apache.org/r/39970/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/
-----------------------------------------------------------
(Updated Nov. 6, 2015, 2:34 p.m.)
Review request for Sqoop.
Bugs: SQOOP-2662
https://issues.apache.org/jira/browse/SQOOP-2662
Repository: sqoop-sqoop2
Description
-------
Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
Diffs (updated)
-----
connector/connector-ftp/pom.xml 9a70dfc
connector/connector-generic-jdbc/pom.xml 7bdeddb
connector/connector-hdfs/pom.xml a28989c
connector/connector-kafka/pom.xml e0f0684
connector/connector-kite/pom.xml 41441f5
connector/connector-oracle-jdbc/pom.xml 325790d
connector/connector-sftp/pom.xml 44e64b1
dist/pom.xml 315a018
dist/src/main/assembly/bin.xml PRE-CREATION
dist/src/main/assembly/src.xml PRE-CREATION
pom.xml f33958c
repository/repository-common/pom.xml c740117
repository/repository-derby/pom.xml a28b036
repository/repository-mysql/pom.xml 066d76b
repository/repository-postgresql/pom.xml 6550bc2
server/pom.xml 370a6a2
submission/mapreduce/pom.xml f56437b
test/pom.xml 4e1e197
Diff: https://reviews.apache.org/r/39970/diff/
Testing
-------
Thanks,
Dian Fu
Re: Review Request 39970: Sqoop2: Remove unnecessary dependencies of
sqoop server module
Posted by Dian Fu <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39970/
-----------------------------------------------------------
(Updated Nov. 5, 2015, 11:47 a.m.)
Review request for Sqoop.
Bugs: SQOOP-2662
https://issues.apache.org/jira/browse/SQOOP-2662
Repository: sqoop-sqoop2
Description
-------
Currently, Sqoop server module has dependencies of all kinds of repositories and connectors in the pom definition. But from the code level, it doesn't depends on these modules at all.
Diffs (updated)
-----
connector/connector-ftp/pom.xml 9a70dfc
connector/connector-generic-jdbc/pom.xml 7bdeddb
connector/connector-hdfs/pom.xml a28989c
connector/connector-kafka/pom.xml e0f0684
connector/connector-kite/pom.xml 41441f5
connector/connector-sftp/pom.xml 44e64b1
dist/pom.xml 315a018
pom.xml ab505f4
repository/repository-common/pom.xml c740117
repository/repository-derby/pom.xml a28b036
repository/repository-mysql/pom.xml 066d76b
repository/repository-postgresql/pom.xml 6550bc2
server/pom.xml ca068e0
submission/mapreduce/pom.xml f56437b
test/pom.xml 4e1e197
Diff: https://reviews.apache.org/r/39970/diff/
Testing
-------
Thanks,
Dian Fu