You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Olivier Lamy <ol...@apache.org> on 2011/08/02 09:49:20 UTC
Maven Build etc...
Hi Folks,
FYI I have started working on a maven build for sqoop (not yet finished :-) ).
I will probably move some unit tests with oracle jdbc driver to a
profile (as this driver is not available in central repo).
No objections if I move classes to package org.apache.sqoop (instead
of com.cloudera.sqoop) ?
I will fix license headers too (all contains : "Licensed to Cloudera,
Inc. .... ")
I have added a jenkins build
(https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
couldn't resist to do it before you :P )
I will add a sonar analysis too.
Thanks,
--
Olivier Lamy
Talend : http://talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Maven Build etc...
Posted by Arvind Prabhakar <ar...@apache.org>.
Sorry - I will go ahead and close 307 as duplicate of 306.
Thanks,
Arvind
On Wed, Aug 3, 2011 at 2:00 PM, Arvind Prabhakar <ar...@apache.org> wrote:
> I took the liberty of creating SQOOP-307 for mavenization of Sqoop,
> and assigned it to Olivier.
>
> Also, +1 on RTC.
>
> Thanks,
> Arvind
>
> On Wed, Aug 3, 2011 at 1:34 PM, Tom White <to...@gmail.com> wrote:
>> +1 for moving to Maven. Is there a JIRA for this? Code changes should
>> have an associated JIRA for discussion and posting patches.
>>
>> Also, I noticed that (some of) these changes have already been
>> committed - I guess we should discuss whether we want to practice
>> review-then-commit or commit-then-review. Personally, I prefer RTC,
>> but it needs to be a group decision.
>>
>> Cheers,
>> Tom
>>
>> On Tue, Aug 2, 2011 at 10:58 AM, Arvind Prabhakar <ar...@apache.org> wrote:
>>> Olivier,
>>>
>>> +1 on Maven migration.
>>>
>>> Thanks for doing this. I would prefer that we maintain the dual build
>>> systems (ant and maven together) until the migration is complete and
>>> all committers have successfully transitioned. Also, I think using
>>> profiles to group tests with a particular database is a good idea. So
>>> mysql profile would include MySQL tests for execution etc. Beyond that
>>> I would also suggest that there be a "full" profile that includes all
>>> the other profiles that are not activated by default.
>>>
>>> Thanks,
>>> Arvind
>>>
>>> On Tue, Aug 2, 2011 at 10:41 AM, Andrew Bayer <an...@gmail.com> wrote:
>>>> What I've been suggesting for at least the tests is, as Olivier mentioned,
>>>> splitting out the non-unit tests into separate modules (a mysql test module,
>>>> an oracle test module, etc) so that they can then be specified in a profile
>>>> in the top-level POM. That'd let us do "builds" without needing to run all
>>>> the tests that require testing against actual DB instances.
>>>>
>>>> The RAT/Findbugs/Cobertura/Checkstyle are all easy, since there are Maven
>>>> plugins for all of 'em. For Eclipse project generation, I think that's
>>>> already in place by default.
>>>>
>>>> The docs will be a little hairy, though.
>>>>
>>>> A.
>>>>
>>>> On Tue, Aug 2, 2011 at 10:32 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>>>>
>>>>> Hey Olivier,
>>>>>
>>>>> Maven is a good thing -- but this is more than just compiling and unit
>>>>> testing the artifact. Let's make sure that we decide which pieces are in
>>>>> scope for the mavenization and at least have a story for if we port or not
>>>>> port certain build features. Having recently gone through (and still going
>>>>> through) an ant -> maven change with Flume, hopefully we can avoid some
>>>>> effort and save some time in the process.
>>>>>
>>>>> Here's a list from perusing the ant build.xml file.
>>>>> * Binary build vs release/source build
>>>>> * Properties or profiles for test vs perf-test
>>>>> * Docs build. Check out this link for how we converted the ant asciidoc
>>>>> build to a maven+ant asciidoc build:
>>>>>
>>>>> https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
>>>>> .
>>>>> This requires external tools.
>>>>> * RAT (release audit tool)?
>>>>> * Findbugs?
>>>>> * Cobertura?
>>>>> * Checkstyle? (This has historically been strongly enforced with sqoop)
>>>>> * Eclipse project? (flume documented around this problem)
>>>>>
>>>>> Jon.
>>>>>
>>>>> On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
>>>>>
>>>>> > BTW First I have to ask if you agree to move to a maven build.
>>>>> > Just chatted with Andrew and he has a good idea to move some tests in
>>>>> > modules activated tru a profile (yup some tests need a running mysql,
>>>>> > oracle instance etc..).
>>>>> >
>>>>> > Let me know before I break everything :-)
>>>>> >
>>>>> > 2011/8/2 Olivier Lamy <ol...@apache.org>:
>>>>> > > Hi Folks,
>>>>> > >
>>>>> > > FYI I have started working on a maven build for sqoop (not yet finished
>>>>> > :-) ).
>>>>> > > I will probably move some unit tests with oracle jdbc driver to a
>>>>> > > profile (as this driver is not available in central repo).
>>>>> > >
>>>>> > > No objections if I move classes to package org.apache.sqoop (instead
>>>>> > > of com.cloudera.sqoop) ?
>>>>> > > I will fix license headers too (all contains : "Licensed to Cloudera,
>>>>> > > Inc. .... ")
>>>>> > >
>>>>> > > I have added a jenkins build
>>>>> > > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
>>>>> > > couldn't resist to do it before you :P )
>>>>> > > I will add a sonar analysis too.
>>>>> > >
>>>>> > > Thanks,
>>>>> > > --
>>>>> > > Olivier Lamy
>>>>> > > Talend : http://talend.com
>>>>> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>> > >
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Olivier Lamy
>>>>> > Talend : http://talend.com
>>>>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> // Jonathan Hsieh (shay)
>>>>> // Software Engineer, Cloudera
>>>>> // jon@cloudera.com
>>>>>
>>>>
>>>
>>
>
Re: Maven Build etc...
Posted by Arvind Prabhakar <ar...@apache.org>.
I took the liberty of creating SQOOP-307 for mavenization of Sqoop,
and assigned it to Olivier.
Also, +1 on RTC.
Thanks,
Arvind
On Wed, Aug 3, 2011 at 1:34 PM, Tom White <to...@gmail.com> wrote:
> +1 for moving to Maven. Is there a JIRA for this? Code changes should
> have an associated JIRA for discussion and posting patches.
>
> Also, I noticed that (some of) these changes have already been
> committed - I guess we should discuss whether we want to practice
> review-then-commit or commit-then-review. Personally, I prefer RTC,
> but it needs to be a group decision.
>
> Cheers,
> Tom
>
> On Tue, Aug 2, 2011 at 10:58 AM, Arvind Prabhakar <ar...@apache.org> wrote:
>> Olivier,
>>
>> +1 on Maven migration.
>>
>> Thanks for doing this. I would prefer that we maintain the dual build
>> systems (ant and maven together) until the migration is complete and
>> all committers have successfully transitioned. Also, I think using
>> profiles to group tests with a particular database is a good idea. So
>> mysql profile would include MySQL tests for execution etc. Beyond that
>> I would also suggest that there be a "full" profile that includes all
>> the other profiles that are not activated by default.
>>
>> Thanks,
>> Arvind
>>
>> On Tue, Aug 2, 2011 at 10:41 AM, Andrew Bayer <an...@gmail.com> wrote:
>>> What I've been suggesting for at least the tests is, as Olivier mentioned,
>>> splitting out the non-unit tests into separate modules (a mysql test module,
>>> an oracle test module, etc) so that they can then be specified in a profile
>>> in the top-level POM. That'd let us do "builds" without needing to run all
>>> the tests that require testing against actual DB instances.
>>>
>>> The RAT/Findbugs/Cobertura/Checkstyle are all easy, since there are Maven
>>> plugins for all of 'em. For Eclipse project generation, I think that's
>>> already in place by default.
>>>
>>> The docs will be a little hairy, though.
>>>
>>> A.
>>>
>>> On Tue, Aug 2, 2011 at 10:32 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>>>
>>>> Hey Olivier,
>>>>
>>>> Maven is a good thing -- but this is more than just compiling and unit
>>>> testing the artifact. Let's make sure that we decide which pieces are in
>>>> scope for the mavenization and at least have a story for if we port or not
>>>> port certain build features. Having recently gone through (and still going
>>>> through) an ant -> maven change with Flume, hopefully we can avoid some
>>>> effort and save some time in the process.
>>>>
>>>> Here's a list from perusing the ant build.xml file.
>>>> * Binary build vs release/source build
>>>> * Properties or profiles for test vs perf-test
>>>> * Docs build. Check out this link for how we converted the ant asciidoc
>>>> build to a maven+ant asciidoc build:
>>>>
>>>> https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
>>>> .
>>>> This requires external tools.
>>>> * RAT (release audit tool)?
>>>> * Findbugs?
>>>> * Cobertura?
>>>> * Checkstyle? (This has historically been strongly enforced with sqoop)
>>>> * Eclipse project? (flume documented around this problem)
>>>>
>>>> Jon.
>>>>
>>>> On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
>>>>
>>>> > BTW First I have to ask if you agree to move to a maven build.
>>>> > Just chatted with Andrew and he has a good idea to move some tests in
>>>> > modules activated tru a profile (yup some tests need a running mysql,
>>>> > oracle instance etc..).
>>>> >
>>>> > Let me know before I break everything :-)
>>>> >
>>>> > 2011/8/2 Olivier Lamy <ol...@apache.org>:
>>>> > > Hi Folks,
>>>> > >
>>>> > > FYI I have started working on a maven build for sqoop (not yet finished
>>>> > :-) ).
>>>> > > I will probably move some unit tests with oracle jdbc driver to a
>>>> > > profile (as this driver is not available in central repo).
>>>> > >
>>>> > > No objections if I move classes to package org.apache.sqoop (instead
>>>> > > of com.cloudera.sqoop) ?
>>>> > > I will fix license headers too (all contains : "Licensed to Cloudera,
>>>> > > Inc. .... ")
>>>> > >
>>>> > > I have added a jenkins build
>>>> > > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
>>>> > > couldn't resist to do it before you :P )
>>>> > > I will add a sonar analysis too.
>>>> > >
>>>> > > Thanks,
>>>> > > --
>>>> > > Olivier Lamy
>>>> > > Talend : http://talend.com
>>>> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Olivier Lamy
>>>> > Talend : http://talend.com
>>>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> // Jonathan Hsieh (shay)
>>>> // Software Engineer, Cloudera
>>>> // jon@cloudera.com
>>>>
>>>
>>
>
Re: Maven Build etc...
Posted by Olivier Lamy <ol...@apache.org>.
2011/8/3 Tom White <to...@gmail.com>:
> +1 for moving to Maven. Is there a JIRA for this? Code changes should
> have an associated JIRA for discussion and posting patches.
Just created https://issues.apache.org/jira/browse/SQOOP-306
>
> Also, I noticed that (some of) these changes have already been
> committed - I guess we should discuss whether we want to practice
> review-then-commit or commit-then-review. Personally, I prefer RTC,
> but it needs to be a group decision.
IMHO it depends on the modification type. Perso I do it (RTC) when I
have no enough knowledge on what I change :-)
>
> Cheers,
> Tom
>
> On Tue, Aug 2, 2011 at 10:58 AM, Arvind Prabhakar <ar...@apache.org> wrote:
>> Olivier,
>>
>> +1 on Maven migration.
>>
>> Thanks for doing this. I would prefer that we maintain the dual build
>> systems (ant and maven together) until the migration is complete and
>> all committers have successfully transitioned. Also, I think using
>> profiles to group tests with a particular database is a good idea. So
>> mysql profile would include MySQL tests for execution etc. Beyond that
>> I would also suggest that there be a "full" profile that includes all
>> the other profiles that are not activated by default.
>>
>> Thanks,
>> Arvind
>>
>> On Tue, Aug 2, 2011 at 10:41 AM, Andrew Bayer <an...@gmail.com> wrote:
>>> What I've been suggesting for at least the tests is, as Olivier mentioned,
>>> splitting out the non-unit tests into separate modules (a mysql test module,
>>> an oracle test module, etc) so that they can then be specified in a profile
>>> in the top-level POM. That'd let us do "builds" without needing to run all
>>> the tests that require testing against actual DB instances.
>>>
>>> The RAT/Findbugs/Cobertura/Checkstyle are all easy, since there are Maven
>>> plugins for all of 'em. For Eclipse project generation, I think that's
>>> already in place by default.
>>>
>>> The docs will be a little hairy, though.
>>>
>>> A.
>>>
>>> On Tue, Aug 2, 2011 at 10:32 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>>>
>>>> Hey Olivier,
>>>>
>>>> Maven is a good thing -- but this is more than just compiling and unit
>>>> testing the artifact. Let's make sure that we decide which pieces are in
>>>> scope for the mavenization and at least have a story for if we port or not
>>>> port certain build features. Having recently gone through (and still going
>>>> through) an ant -> maven change with Flume, hopefully we can avoid some
>>>> effort and save some time in the process.
>>>>
>>>> Here's a list from perusing the ant build.xml file.
>>>> * Binary build vs release/source build
>>>> * Properties or profiles for test vs perf-test
>>>> * Docs build. Check out this link for how we converted the ant asciidoc
>>>> build to a maven+ant asciidoc build:
>>>>
>>>> https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
>>>> .
>>>> This requires external tools.
>>>> * RAT (release audit tool)?
>>>> * Findbugs?
>>>> * Cobertura?
>>>> * Checkstyle? (This has historically been strongly enforced with sqoop)
>>>> * Eclipse project? (flume documented around this problem)
>>>>
>>>> Jon.
>>>>
>>>> On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
>>>>
>>>> > BTW First I have to ask if you agree to move to a maven build.
>>>> > Just chatted with Andrew and he has a good idea to move some tests in
>>>> > modules activated tru a profile (yup some tests need a running mysql,
>>>> > oracle instance etc..).
>>>> >
>>>> > Let me know before I break everything :-)
>>>> >
>>>> > 2011/8/2 Olivier Lamy <ol...@apache.org>:
>>>> > > Hi Folks,
>>>> > >
>>>> > > FYI I have started working on a maven build for sqoop (not yet finished
>>>> > :-) ).
>>>> > > I will probably move some unit tests with oracle jdbc driver to a
>>>> > > profile (as this driver is not available in central repo).
>>>> > >
>>>> > > No objections if I move classes to package org.apache.sqoop (instead
>>>> > > of com.cloudera.sqoop) ?
>>>> > > I will fix license headers too (all contains : "Licensed to Cloudera,
>>>> > > Inc. .... ")
>>>> > >
>>>> > > I have added a jenkins build
>>>> > > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
>>>> > > couldn't resist to do it before you :P )
>>>> > > I will add a sonar analysis too.
>>>> > >
>>>> > > Thanks,
>>>> > > --
>>>> > > Olivier Lamy
>>>> > > Talend : http://talend.com
>>>> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> > >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Olivier Lamy
>>>> > Talend : http://talend.com
>>>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> // Jonathan Hsieh (shay)
>>>> // Software Engineer, Cloudera
>>>> // jon@cloudera.com
>>>>
>>>
>>
>
--
Olivier Lamy
Talend : http://talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Maven Build etc...
Posted by Tom White <to...@gmail.com>.
+1 for moving to Maven. Is there a JIRA for this? Code changes should
have an associated JIRA for discussion and posting patches.
Also, I noticed that (some of) these changes have already been
committed - I guess we should discuss whether we want to practice
review-then-commit or commit-then-review. Personally, I prefer RTC,
but it needs to be a group decision.
Cheers,
Tom
On Tue, Aug 2, 2011 at 10:58 AM, Arvind Prabhakar <ar...@apache.org> wrote:
> Olivier,
>
> +1 on Maven migration.
>
> Thanks for doing this. I would prefer that we maintain the dual build
> systems (ant and maven together) until the migration is complete and
> all committers have successfully transitioned. Also, I think using
> profiles to group tests with a particular database is a good idea. So
> mysql profile would include MySQL tests for execution etc. Beyond that
> I would also suggest that there be a "full" profile that includes all
> the other profiles that are not activated by default.
>
> Thanks,
> Arvind
>
> On Tue, Aug 2, 2011 at 10:41 AM, Andrew Bayer <an...@gmail.com> wrote:
>> What I've been suggesting for at least the tests is, as Olivier mentioned,
>> splitting out the non-unit tests into separate modules (a mysql test module,
>> an oracle test module, etc) so that they can then be specified in a profile
>> in the top-level POM. That'd let us do "builds" without needing to run all
>> the tests that require testing against actual DB instances.
>>
>> The RAT/Findbugs/Cobertura/Checkstyle are all easy, since there are Maven
>> plugins for all of 'em. For Eclipse project generation, I think that's
>> already in place by default.
>>
>> The docs will be a little hairy, though.
>>
>> A.
>>
>> On Tue, Aug 2, 2011 at 10:32 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>>
>>> Hey Olivier,
>>>
>>> Maven is a good thing -- but this is more than just compiling and unit
>>> testing the artifact. Let's make sure that we decide which pieces are in
>>> scope for the mavenization and at least have a story for if we port or not
>>> port certain build features. Having recently gone through (and still going
>>> through) an ant -> maven change with Flume, hopefully we can avoid some
>>> effort and save some time in the process.
>>>
>>> Here's a list from perusing the ant build.xml file.
>>> * Binary build vs release/source build
>>> * Properties or profiles for test vs perf-test
>>> * Docs build. Check out this link for how we converted the ant asciidoc
>>> build to a maven+ant asciidoc build:
>>>
>>> https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
>>> .
>>> This requires external tools.
>>> * RAT (release audit tool)?
>>> * Findbugs?
>>> * Cobertura?
>>> * Checkstyle? (This has historically been strongly enforced with sqoop)
>>> * Eclipse project? (flume documented around this problem)
>>>
>>> Jon.
>>>
>>> On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
>>>
>>> > BTW First I have to ask if you agree to move to a maven build.
>>> > Just chatted with Andrew and he has a good idea to move some tests in
>>> > modules activated tru a profile (yup some tests need a running mysql,
>>> > oracle instance etc..).
>>> >
>>> > Let me know before I break everything :-)
>>> >
>>> > 2011/8/2 Olivier Lamy <ol...@apache.org>:
>>> > > Hi Folks,
>>> > >
>>> > > FYI I have started working on a maven build for sqoop (not yet finished
>>> > :-) ).
>>> > > I will probably move some unit tests with oracle jdbc driver to a
>>> > > profile (as this driver is not available in central repo).
>>> > >
>>> > > No objections if I move classes to package org.apache.sqoop (instead
>>> > > of com.cloudera.sqoop) ?
>>> > > I will fix license headers too (all contains : "Licensed to Cloudera,
>>> > > Inc. .... ")
>>> > >
>>> > > I have added a jenkins build
>>> > > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
>>> > > couldn't resist to do it before you :P )
>>> > > I will add a sonar analysis too.
>>> > >
>>> > > Thanks,
>>> > > --
>>> > > Olivier Lamy
>>> > > Talend : http://talend.com
>>> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Olivier Lamy
>>> > Talend : http://talend.com
>>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >
>>>
>>>
>>>
>>> --
>>> // Jonathan Hsieh (shay)
>>> // Software Engineer, Cloudera
>>> // jon@cloudera.com
>>>
>>
>
Re: Maven Build etc...
Posted by Arvind Prabhakar <ar...@apache.org>.
Olivier,
+1 on Maven migration.
Thanks for doing this. I would prefer that we maintain the dual build
systems (ant and maven together) until the migration is complete and
all committers have successfully transitioned. Also, I think using
profiles to group tests with a particular database is a good idea. So
mysql profile would include MySQL tests for execution etc. Beyond that
I would also suggest that there be a "full" profile that includes all
the other profiles that are not activated by default.
Thanks,
Arvind
On Tue, Aug 2, 2011 at 10:41 AM, Andrew Bayer <an...@gmail.com> wrote:
> What I've been suggesting for at least the tests is, as Olivier mentioned,
> splitting out the non-unit tests into separate modules (a mysql test module,
> an oracle test module, etc) so that they can then be specified in a profile
> in the top-level POM. That'd let us do "builds" without needing to run all
> the tests that require testing against actual DB instances.
>
> The RAT/Findbugs/Cobertura/Checkstyle are all easy, since there are Maven
> plugins for all of 'em. For Eclipse project generation, I think that's
> already in place by default.
>
> The docs will be a little hairy, though.
>
> A.
>
> On Tue, Aug 2, 2011 at 10:32 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
>> Hey Olivier,
>>
>> Maven is a good thing -- but this is more than just compiling and unit
>> testing the artifact. Let's make sure that we decide which pieces are in
>> scope for the mavenization and at least have a story for if we port or not
>> port certain build features. Having recently gone through (and still going
>> through) an ant -> maven change with Flume, hopefully we can avoid some
>> effort and save some time in the process.
>>
>> Here's a list from perusing the ant build.xml file.
>> * Binary build vs release/source build
>> * Properties or profiles for test vs perf-test
>> * Docs build. Check out this link for how we converted the ant asciidoc
>> build to a maven+ant asciidoc build:
>>
>> https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
>> .
>> This requires external tools.
>> * RAT (release audit tool)?
>> * Findbugs?
>> * Cobertura?
>> * Checkstyle? (This has historically been strongly enforced with sqoop)
>> * Eclipse project? (flume documented around this problem)
>>
>> Jon.
>>
>> On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
>>
>> > BTW First I have to ask if you agree to move to a maven build.
>> > Just chatted with Andrew and he has a good idea to move some tests in
>> > modules activated tru a profile (yup some tests need a running mysql,
>> > oracle instance etc..).
>> >
>> > Let me know before I break everything :-)
>> >
>> > 2011/8/2 Olivier Lamy <ol...@apache.org>:
>> > > Hi Folks,
>> > >
>> > > FYI I have started working on a maven build for sqoop (not yet finished
>> > :-) ).
>> > > I will probably move some unit tests with oracle jdbc driver to a
>> > > profile (as this driver is not available in central repo).
>> > >
>> > > No objections if I move classes to package org.apache.sqoop (instead
>> > > of com.cloudera.sqoop) ?
>> > > I will fix license headers too (all contains : "Licensed to Cloudera,
>> > > Inc. .... ")
>> > >
>> > > I have added a jenkins build
>> > > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
>> > > couldn't resist to do it before you :P )
>> > > I will add a sonar analysis too.
>> > >
>> > > Thanks,
>> > > --
>> > > Olivier Lamy
>> > > Talend : http://talend.com
>> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
>> > >
>> >
>> >
>> >
>> > --
>> > Olivier Lamy
>> > Talend : http://talend.com
>> > http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >
>>
>>
>>
>> --
>> // Jonathan Hsieh (shay)
>> // Software Engineer, Cloudera
>> // jon@cloudera.com
>>
>
Re: Maven Build etc...
Posted by Andrew Bayer <an...@gmail.com>.
What I've been suggesting for at least the tests is, as Olivier mentioned,
splitting out the non-unit tests into separate modules (a mysql test module,
an oracle test module, etc) so that they can then be specified in a profile
in the top-level POM. That'd let us do "builds" without needing to run all
the tests that require testing against actual DB instances.
The RAT/Findbugs/Cobertura/Checkstyle are all easy, since there are Maven
plugins for all of 'em. For Eclipse project generation, I think that's
already in place by default.
The docs will be a little hairy, though.
A.
On Tue, Aug 2, 2011 at 10:32 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> Hey Olivier,
>
> Maven is a good thing -- but this is more than just compiling and unit
> testing the artifact. Let's make sure that we decide which pieces are in
> scope for the mavenization and at least have a story for if we port or not
> port certain build features. Having recently gone through (and still going
> through) an ant -> maven change with Flume, hopefully we can avoid some
> effort and save some time in the process.
>
> Here's a list from perusing the ant build.xml file.
> * Binary build vs release/source build
> * Properties or profiles for test vs perf-test
> * Docs build. Check out this link for how we converted the ant asciidoc
> build to a maven+ant asciidoc build:
>
> https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
> .
> This requires external tools.
> * RAT (release audit tool)?
> * Findbugs?
> * Cobertura?
> * Checkstyle? (This has historically been strongly enforced with sqoop)
> * Eclipse project? (flume documented around this problem)
>
> Jon.
>
> On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
>
> > BTW First I have to ask if you agree to move to a maven build.
> > Just chatted with Andrew and he has a good idea to move some tests in
> > modules activated tru a profile (yup some tests need a running mysql,
> > oracle instance etc..).
> >
> > Let me know before I break everything :-)
> >
> > 2011/8/2 Olivier Lamy <ol...@apache.org>:
> > > Hi Folks,
> > >
> > > FYI I have started working on a maven build for sqoop (not yet finished
> > :-) ).
> > > I will probably move some unit tests with oracle jdbc driver to a
> > > profile (as this driver is not available in central repo).
> > >
> > > No objections if I move classes to package org.apache.sqoop (instead
> > > of com.cloudera.sqoop) ?
> > > I will fix license headers too (all contains : "Licensed to Cloudera,
> > > Inc. .... ")
> > >
> > > I have added a jenkins build
> > > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
> > > couldn't resist to do it before you :P )
> > > I will add a sonar analysis too.
> > >
> > > Thanks,
> > > --
> > > Olivier Lamy
> > > Talend : http://talend.com
> > > http://twitter.com/olamy | http://linkedin.com/in/olamy
> > >
> >
> >
> >
> > --
> > Olivier Lamy
> > Talend : http://talend.com
> > http://twitter.com/olamy | http://linkedin.com/in/olamy
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>
Re: Maven Build etc...
Posted by Jonathan Hsieh <jo...@cloudera.com>.
Hey Olivier,
Maven is a good thing -- but this is more than just compiling and unit
testing the artifact. Let's make sure that we decide which pieces are in
scope for the mavenization and at least have a story for if we port or not
port certain build features. Having recently gone through (and still going
through) an ant -> maven change with Flume, hopefully we can avoid some
effort and save some time in the process.
Here's a list from perusing the ant build.xml file.
* Binary build vs release/source build
* Properties or profiles for test vs perf-test
* Docs build. Check out this link for how we converted the ant asciidoc
build to a maven+ant asciidoc build:
https://github.com/cloudera/flume/commit/7cb94837beedf2bdfd0245e0404f8855b259a274
.
This requires external tools.
* RAT (release audit tool)?
* Findbugs?
* Cobertura?
* Checkstyle? (This has historically been strongly enforced with sqoop)
* Eclipse project? (flume documented around this problem)
Jon.
On Tue, Aug 2, 2011 at 9:36 AM, Olivier Lamy <ol...@apache.org> wrote:
> BTW First I have to ask if you agree to move to a maven build.
> Just chatted with Andrew and he has a good idea to move some tests in
> modules activated tru a profile (yup some tests need a running mysql,
> oracle instance etc..).
>
> Let me know before I break everything :-)
>
> 2011/8/2 Olivier Lamy <ol...@apache.org>:
> > Hi Folks,
> >
> > FYI I have started working on a maven build for sqoop (not yet finished
> :-) ).
> > I will probably move some unit tests with oracle jdbc driver to a
> > profile (as this driver is not available in central repo).
> >
> > No objections if I move classes to package org.apache.sqoop (instead
> > of com.cloudera.sqoop) ?
> > I will fix license headers too (all contains : "Licensed to Cloudera,
> > Inc. .... ")
> >
> > I have added a jenkins build
> > (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
> > couldn't resist to do it before you :P )
> > I will add a sonar analysis too.
> >
> > Thanks,
> > --
> > Olivier Lamy
> > Talend : http://talend.com
> > http://twitter.com/olamy | http://linkedin.com/in/olamy
> >
>
>
>
> --
> Olivier Lamy
> Talend : http://talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com
Re: Maven Build etc...
Posted by Olivier Lamy <ol...@apache.org>.
BTW First I have to ask if you agree to move to a maven build.
Just chatted with Andrew and he has a good idea to move some tests in
modules activated tru a profile (yup some tests need a running mysql,
oracle instance etc..).
Let me know before I break everything :-)
2011/8/2 Olivier Lamy <ol...@apache.org>:
> Hi Folks,
>
> FYI I have started working on a maven build for sqoop (not yet finished :-) ).
> I will probably move some unit tests with oracle jdbc driver to a
> profile (as this driver is not available in central repo).
>
> No objections if I move classes to package org.apache.sqoop (instead
> of com.cloudera.sqoop) ?
> I will fix license headers too (all contains : "Licensed to Cloudera,
> Inc. .... ")
>
> I have added a jenkins build
> (https://builds.apache.org/job/Sqoop-jdk-1.6/ ) (hehe @abayer I
> couldn't resist to do it before you :P )
> I will add a sonar analysis too.
>
> Thanks,
> --
> Olivier Lamy
> Talend : http://talend.com
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
--
Olivier Lamy
Talend : http://talend.com
http://twitter.com/olamy | http://linkedin.com/in/olamy