You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Andrew Wang <an...@cloudera.com> on 2015/09/11 01:16:57 UTC
Using dependency plugin to build minimal local repository to run unit tests
Hi Maven experts,
I'm trying to get a minimal set of local repository contents to be able to
run unit tests for a project in offline mode. The dependency plugin
documentation indicates that by default it's test scope, which is just what
I want, so I did something like this (Maven 3.0.4):
# Copy what I can from my existing local repository to my temp repo
mvn dependency:copy-dependencies -Dmdep.useRepositoryLayout=true
-DoutputDirectory=/tmp/hadoop-deps -Dmdep.copyPom
# Download things missed by copy:dependencies
mvn dependency:go-offline -Dmaven.repo.local=/tmp/hadoop-deps
# Try running offline unit test
mvn -o surefire:test -Dtest=TestFoo -Dmaven.repo.local=/tmp/hadoop-deps
This was still missing some dependencies, so I ran this which downloaded
some more:
mvn surefire:test -DskipTests -Dmaven.repo.local=/tmp/hadoop-deps
Invoking offline maven to run TestFoo still failed though, this time
missing Surefire. After doing dependency:get on that, my offline test
finally worked.
Is this expected behavior? I expected dependency:go-offline to be
sufficient for this purpose. I'm was also very surprised that running `mvn
surefire:test -DskipTests` did not download all the required dependencies.
If you have ideas on how better to do this, I'm also all ears :) This is
related to some distributed testing experiments I'm doing for Apache Hadoop
and HBase, which will hopefully be put into broader use upstream at some
point.
Thanks,
Andrew
Re: Using dependency plugin to build minimal local repository to run
unit tests
Posted by Andrew Wang <an...@cloudera.com>.
Hi all, thanks for prompt replies!
A common theme I'm hearing is to point to a clean repository and then do
the build. The thing is, there are additional dependencies that are only
pulled down when tests are actually run. Even doing `mvn test -DskipTests`
still left me missing a surefire-junit4 dependency. I can hack around that
by making my users specify these additional "test runtime" dependencies,
but that's kind of unfortunate.
Is this usecase supposed to be handled by the dependency plugin? It seems
like what copy-dependencies and go-offline are intended for, and they even
mention test scope, but after running them I'm still left missing many
dependencies.
I also already have all the deps in ~/.m2/repository, so it'd be really
great to copy from there (or better, hardlink) rather than re-download from
the internet. I could work on some of this if it sounds reasonable.
Basically, a copy-dependency mode that also pulls in everything pulled by
go-offline, and hardlinks to ~/.m2/repository rather than copying.
Thanks,
Andrew
On Thu, Sep 10, 2015 at 4:35 PM, Laird Nelson <lj...@gmail.com> wrote:
> Hello; you might also want to look at
>
> https://maven.apache.org/plugins/maven-dependency-plugin/go-offline-mojo.html
> .
>
> Best,
> Laird
> --
> http://about.me/lairdnelson
>
> On Thu, Sep 10, 2015 at 4:29 PM Mark Eggers <its_toasted@yahoo.com.invalid
> >
> wrote:
>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Andrew,
> >
> > I don't know if this would help:
> >
> > 1. Set up a repository on your local machine
> > 2. Configure that repository to proxy remote repositories
> > 3. Set your settings.xml to point to that local repository
> > 4. Build normally
> >
> > Then when you're off-line, you're not really off-line. Provided that
> > your local repository has pulled down all the dependencies, builds
> > should work as expected.
> >
> > The downside of this is that it's a bit piggish on disk space.
> >
> > I'm also not sure how to bounce back and forth between various
> > repositories, ie;
> >
> > 1. Connected to $work - you'll want to pull everything from that
> > repository into your local repository
> > 2. Connected to notwork - you'll probably want to use Central through
> > your local repository
> >
> > Almost sounds like you would need two different environments ($work
> > and notwork). This is probably best in any case.
> >
> > . . . just my two cents
> > /mde/
> >
> > On 9/10/2015 4:16 PM, Andrew Wang wrote:
> > > Hi Maven experts,
> > >
> > > I'm trying to get a minimal set of local repository contents to be
> > > able to run unit tests for a project in offline mode. The
> > > dependency plugin documentation indicates that by default it's test
> > > scope, which is just what I want, so I did something like this
> > > (Maven 3.0.4):
> > >
> > > # Copy what I can from my existing local repository to my temp
> > > repo mvn dependency:copy-dependencies
> > > -Dmdep.useRepositoryLayout=true -DoutputDirectory=/tmp/hadoop-deps
> > > -Dmdep.copyPom # Download things missed by copy:dependencies mvn
> > > dependency:go-offline -Dmaven.repo.local=/tmp/hadoop-deps # Try
> > > running offline unit test mvn -o surefire:test -Dtest=TestFoo
> > > -Dmaven.repo.local=/tmp/hadoop-deps
> > >
> > > This was still missing some dependencies, so I ran this which
> > > downloaded some more:
> > >
> > > mvn surefire:test -DskipTests -Dmaven.repo.local=/tmp/hadoop-deps
> > >
> > > Invoking offline maven to run TestFoo still failed though, this
> > > time missing Surefire. After doing dependency:get on that, my
> > > offline test finally worked.
> > >
> > > Is this expected behavior? I expected dependency:go-offline to be
> > > sufficient for this purpose. I'm was also very surprised that
> > > running `mvn surefire:test -DskipTests` did not download all the
> > > required dependencies.
> > >
> > > If you have ideas on how better to do this, I'm also all ears :)
> > > This is related to some distributed testing experiments I'm doing
> > > for Apache Hadoop and HBase, which will hopefully be put into
> > > broader use upstream at some point.
> > >
> > > Thanks, Andrew
> > >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v2
> >
> > iQEcBAEBAgAGBQJV8hJDAAoJEEFGbsYNeTwtRlwH/2Yh7Xz5PwKDX+fBVhT+WAaK
> > dihjGnwfs0pIXrwa2jh3Vnw2t79CR5FiJf2f9+AGK75p8N47pJQiUpN7ioADmLuJ
> > dox/CloSRl0XGNME1rhp5B+wgfafjYBE5tAY0DTVfIODYAjAKigXQvFDeAlCihHs
> > HI/OMg0+Q+A9UTKDU/dcd15MnRDktmX+9iCPO+KOeEEOywYCIYclW1Pg95PEJKkA
> > LaJ9WnckDhWsghAO5zj4j/+V4ByBm4RKO8mX0eQwSES7Fq4R2O0Mb6Q/mcz/Ap7v
> > lvHbHbr5cGoDm6Ru7tYy8If6CkoirjUEqz+f0v0NXbY5N2s6l0rHnjz2gkydPTY=
> > =NU1A
> > -----END PGP SIGNATURE-----
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> > For additional commands, e-mail: users-help@maven.apache.org
> >
> >
>
Re: Using dependency plugin to build minimal local repository to run
unit tests
Posted by Laird Nelson <lj...@gmail.com>.
Hello; you might also want to look at
https://maven.apache.org/plugins/maven-dependency-plugin/go-offline-mojo.html
.
Best,
Laird
--
http://about.me/lairdnelson
On Thu, Sep 10, 2015 at 4:29 PM Mark Eggers <it...@yahoo.com.invalid>
wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Andrew,
>
> I don't know if this would help:
>
> 1. Set up a repository on your local machine
> 2. Configure that repository to proxy remote repositories
> 3. Set your settings.xml to point to that local repository
> 4. Build normally
>
> Then when you're off-line, you're not really off-line. Provided that
> your local repository has pulled down all the dependencies, builds
> should work as expected.
>
> The downside of this is that it's a bit piggish on disk space.
>
> I'm also not sure how to bounce back and forth between various
> repositories, ie;
>
> 1. Connected to $work - you'll want to pull everything from that
> repository into your local repository
> 2. Connected to notwork - you'll probably want to use Central through
> your local repository
>
> Almost sounds like you would need two different environments ($work
> and notwork). This is probably best in any case.
>
> . . . just my two cents
> /mde/
>
> On 9/10/2015 4:16 PM, Andrew Wang wrote:
> > Hi Maven experts,
> >
> > I'm trying to get a minimal set of local repository contents to be
> > able to run unit tests for a project in offline mode. The
> > dependency plugin documentation indicates that by default it's test
> > scope, which is just what I want, so I did something like this
> > (Maven 3.0.4):
> >
> > # Copy what I can from my existing local repository to my temp
> > repo mvn dependency:copy-dependencies
> > -Dmdep.useRepositoryLayout=true -DoutputDirectory=/tmp/hadoop-deps
> > -Dmdep.copyPom # Download things missed by copy:dependencies mvn
> > dependency:go-offline -Dmaven.repo.local=/tmp/hadoop-deps # Try
> > running offline unit test mvn -o surefire:test -Dtest=TestFoo
> > -Dmaven.repo.local=/tmp/hadoop-deps
> >
> > This was still missing some dependencies, so I ran this which
> > downloaded some more:
> >
> > mvn surefire:test -DskipTests -Dmaven.repo.local=/tmp/hadoop-deps
> >
> > Invoking offline maven to run TestFoo still failed though, this
> > time missing Surefire. After doing dependency:get on that, my
> > offline test finally worked.
> >
> > Is this expected behavior? I expected dependency:go-offline to be
> > sufficient for this purpose. I'm was also very surprised that
> > running `mvn surefire:test -DskipTests` did not download all the
> > required dependencies.
> >
> > If you have ideas on how better to do this, I'm also all ears :)
> > This is related to some distributed testing experiments I'm doing
> > for Apache Hadoop and HBase, which will hopefully be put into
> > broader use upstream at some point.
> >
> > Thanks, Andrew
> >
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQEcBAEBAgAGBQJV8hJDAAoJEEFGbsYNeTwtRlwH/2Yh7Xz5PwKDX+fBVhT+WAaK
> dihjGnwfs0pIXrwa2jh3Vnw2t79CR5FiJf2f9+AGK75p8N47pJQiUpN7ioADmLuJ
> dox/CloSRl0XGNME1rhp5B+wgfafjYBE5tAY0DTVfIODYAjAKigXQvFDeAlCihHs
> HI/OMg0+Q+A9UTKDU/dcd15MnRDktmX+9iCPO+KOeEEOywYCIYclW1Pg95PEJKkA
> LaJ9WnckDhWsghAO5zj4j/+V4ByBm4RKO8mX0eQwSES7Fq4R2O0Mb6Q/mcz/Ap7v
> lvHbHbr5cGoDm6Ru7tYy8If6CkoirjUEqz+f0v0NXbY5N2s6l0rHnjz2gkydPTY=
> =NU1A
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>
Re: Using dependency plugin to build minimal local repository to run
unit tests
Posted by Mark Eggers <it...@yahoo.com.INVALID>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Andrew,
I don't know if this would help:
1. Set up a repository on your local machine
2. Configure that repository to proxy remote repositories
3. Set your settings.xml to point to that local repository
4. Build normally
Then when you're off-line, you're not really off-line. Provided that
your local repository has pulled down all the dependencies, builds
should work as expected.
The downside of this is that it's a bit piggish on disk space.
I'm also not sure how to bounce back and forth between various
repositories, ie;
1. Connected to $work - you'll want to pull everything from that
repository into your local repository
2. Connected to notwork - you'll probably want to use Central through
your local repository
Almost sounds like you would need two different environments ($work
and notwork). This is probably best in any case.
. . . just my two cents
/mde/
On 9/10/2015 4:16 PM, Andrew Wang wrote:
> Hi Maven experts,
>
> I'm trying to get a minimal set of local repository contents to be
> able to run unit tests for a project in offline mode. The
> dependency plugin documentation indicates that by default it's test
> scope, which is just what I want, so I did something like this
> (Maven 3.0.4):
>
> # Copy what I can from my existing local repository to my temp
> repo mvn dependency:copy-dependencies
> -Dmdep.useRepositoryLayout=true -DoutputDirectory=/tmp/hadoop-deps
> -Dmdep.copyPom # Download things missed by copy:dependencies mvn
> dependency:go-offline -Dmaven.repo.local=/tmp/hadoop-deps # Try
> running offline unit test mvn -o surefire:test -Dtest=TestFoo
> -Dmaven.repo.local=/tmp/hadoop-deps
>
> This was still missing some dependencies, so I ran this which
> downloaded some more:
>
> mvn surefire:test -DskipTests -Dmaven.repo.local=/tmp/hadoop-deps
>
> Invoking offline maven to run TestFoo still failed though, this
> time missing Surefire. After doing dependency:get on that, my
> offline test finally worked.
>
> Is this expected behavior? I expected dependency:go-offline to be
> sufficient for this purpose. I'm was also very surprised that
> running `mvn surefire:test -DskipTests` did not download all the
> required dependencies.
>
> If you have ideas on how better to do this, I'm also all ears :)
> This is related to some distributed testing experiments I'm doing
> for Apache Hadoop and HBase, which will hopefully be put into
> broader use upstream at some point.
>
> Thanks, Andrew
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBAgAGBQJV8hJDAAoJEEFGbsYNeTwtRlwH/2Yh7Xz5PwKDX+fBVhT+WAaK
dihjGnwfs0pIXrwa2jh3Vnw2t79CR5FiJf2f9+AGK75p8N47pJQiUpN7ioADmLuJ
dox/CloSRl0XGNME1rhp5B+wgfafjYBE5tAY0DTVfIODYAjAKigXQvFDeAlCihHs
HI/OMg0+Q+A9UTKDU/dcd15MnRDktmX+9iCPO+KOeEEOywYCIYclW1Pg95PEJKkA
LaJ9WnckDhWsghAO5zj4j/+V4ByBm4RKO8mX0eQwSES7Fq4R2O0Mb6Q/mcz/Ap7v
lvHbHbr5cGoDm6Ru7tYy8If6CkoirjUEqz+f0v0NXbY5N2s6l0rHnjz2gkydPTY=
=NU1A
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: Using dependency plugin to build minimal local repository to run unit tests
Posted by Bernd Eckenfels <ec...@zusammenkunft.net>.
Just run the Job you want to run with a fresh/empty local repository.
> Am 11.09.2015 um 01:16 schrieb Andrew Wang <an...@cloudera.com>:
>
> Hi Maven experts,
>
> I'm trying to get a minimal set of local repository contents to be able to
> run unit tests for a project in offline mode. The dependency plugin
> documentation indicates that by default it's test scope, which is just what
> I want, so I did something like this (Maven 3.0.4):
>
> # Copy what I can from my existing local repository to my temp repo
> mvn dependency:copy-dependencies -Dmdep.useRepositoryLayout=true
> -DoutputDirectory=/tmp/hadoop-deps -Dmdep.copyPom
> # Download things missed by copy:dependencies
> mvn dependency:go-offline -Dmaven.repo.local=/tmp/hadoop-deps
> # Try running offline unit test
> mvn -o surefire:test -Dtest=TestFoo -Dmaven.repo.local=/tmp/hadoop-deps
>
> This was still missing some dependencies, so I ran this which downloaded
> some more:
>
> mvn surefire:test -DskipTests -Dmaven.repo.local=/tmp/hadoop-deps
>
> Invoking offline maven to run TestFoo still failed though, this time
> missing Surefire. After doing dependency:get on that, my offline test
> finally worked.
>
> Is this expected behavior? I expected dependency:go-offline to be
> sufficient for this purpose. I'm was also very surprised that running `mvn
> surefire:test -DskipTests` did not download all the required dependencies.
>
> If you have ideas on how better to do this, I'm also all ears :) This is
> related to some distributed testing experiments I'm doing for Apache Hadoop
> and HBase, which will hopefully be put into broader use upstream at some
> point.
>
> Thanks,
> Andrew
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org