You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Tibor Digana <ti...@apache.org> on 2019/10/30 01:03:11 UTC
Docker image with initialized local cache saves 50 seconds in startup
If you use Docker images with Maven with no mapping of cache to the
volumes, you may notice that Maven downloads the plugins for the build
lifecycle.
This slows down the build because a lot of artifacts and plugins are
initially downloaded.
This takes 50 seconds which might be even longer than the productive build
itself (compiler, package, ...).
We discussed this topic with Herve and Karl at the Apache CON 2019 the last
time.
Sometime the presentations were funny because the audience had to wait a
minute while the console was black where the Maven was downloading the
plugins in the background.
Nobody was sure what happened that time, whether the console hanged or the
Cloud server hanged, or another issue happened with the network.
I made a test and triggered the default lifecycle on Maven and I realized
that the cache was really very little, cca 12 MB.
So this little cache in the container would save 50 seconds which is the
improvement we are discussing.
From the use point of view, the user would use a new base image `
3.6.2-jdk-14-prefetched` which is my idea.
There are multiple technical solutions (range of plugins, extra pom,
internal Maven plugin versions, etc).
We understood that the best idea would be to have the image with the cache
in new Docker images produced by Carloss Sanchez.
We are discussing this topic in [1] but we do not have a consensus on who
will develop the Docker scripts and how.
We can continue here and we can propose a solution.
[1] https://github.com/carlossg/docker-maven/issues/130
Cheers
Tibor17
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Tibor Digana <ti...@apache.org>.
Thx Herve for the hint. The size of the local cahe is 40 MB.
All core plugins [1] and Packaging (ear, ejb, jar, rar and war) are
included in the range [3.0, 4.0).
Two plugins (verifier:2.0+ and rar:1.0+) do not have this range.
[1] https://maven.apache.org/plugins/
On Wed, Oct 30, 2019 at 10:47 PM Hervé Boutemy <hb...@apache.org> wrote:
> Le mercredi 30 octobre 2019 14:55:46 CET, vous avez écrit :
> > Agree we should publish with an asf account and make it part of the
> release
> > process.
> > That said I still fail to see how you can add a relevant cache. Maybe
> take
> > the time to review plugin version in a few asf projects (let say
> > maven-surefire, geronimo-openapi and spark) and check out if it works to
> > cache anything.
> if we choose one precise version of each plugin, we will never get the
> version
> used by a project: IMHO, we need to choose a range for each plugin
> https://maven.apache.org/plugins/
> Something like dependency:resolve-plugins should be able to do the job.
>
> With a range per plugin, I'd love to see the size of the initial local
> repository
> At least, using such an image would require minimal efforts from users,
> particularly on CI servers
>
> Regards,
>
> Hervé
>
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau>
> > | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le mer. 30 oct. 2019 à 14:31, Tibor Digana <ti...@apache.org> a
> >
> > écrit :
> > > Stephen, yeah something like you do in your scrip but it must not be a
> > > personal owner. Even Carloss is person who makes this deployment to
> > > DockerHub but his images are used by the entire world and we should
> decide
> > > whether we would agree with him to have such images under his
> > > responsibility or our responsibility as the Apache group. Then the
> script
> > > would be official and we can cut the release of Maven and the Docker
> image
> > > will be ready for the downloads together with the Maven distribution.
> So
> > > the users will always know that it is consistent deployment and they
> > > wouldn't expect that the image is missing for the new version.
> > >
> > > So here we should decide on who will be the deployer of these images
> with
> > > the cache. And the technical solution is smaller problem I would say.
> > >
> > > On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> > >
> > > stephen.alan.connolly@gmail.com> wrote:
> > > > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> > >
> > > wrote:
> > > > > It's the situation when you have maven plugins in repo and it means
> > >
> > > that
> > >
> > > > > all custom plugins/deps can be still downloaded as before.
> > > > > Nothing exists like this in the world and we are talking about the
> > > > > approaches.
> > > >
> > > > Cough cough cough
> > >
> > >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae41
> > > 8ca3fd63c63038e896255c6fe8/Dockerfile#L50>
> > > > There are issues, but it does shave a bit of time... though we end up
> > > > adding our common dependencies into the seed pom so that it is an
> > >
> > > effective
> > >
> > > > speed up
> > > >
> > > > > I added Karl, Herve and Stephen in CC because we talked about this
> > >
> > > issue
> > >
> > > > > in ASF CON and Twitter.
> > > > >
> > > > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > > >
> > > > rmannibucau@gmail.com>
> > > >
> > > > > wrote:
> > > > >> Hi Tibor,
> > > > >>
> > > > >> It has two issues:
> > > > >>
> > > > >> 1. It will not be the right plugin versions in 90% of the cases
> > >
> > > (except
> > >
> > > > >> demo ;))
> > > > >> 2. It will miss all custom plugins
> > > > >>
> > > > >> Now question is: what happens if you mount your local repo when
> > >
> > > running
> > >
> > > > >> docker? It works as expected. Means we could use a custom
> entrypoint
> > > > >> printing a warning banner if it is not done probably instead of
> > > >
> > > > incrasing
> > > >
> > > > >> the image size without being sure to reach the original goal.
> > > > >>
> > > > >> Wdyt?
> > > > >>
> > > > >> Romain
> > > > >>
> > > > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <
> tibordigana@apache.org> a
> > > > >>
> > > > >> écrit :
> > > > >> > If you use Docker images with Maven with no mapping of cache to
> the
> > > > >> > volumes, you may notice that Maven downloads the plugins for the
> > >
> > > build
> > >
> > > > >> > lifecycle.
> > > > >> >
> > > > >> > This slows down the build because a lot of artifacts and plugins
> > > > >> > are
> > > > >> > initially downloaded.
> > > > >> > This takes 50 seconds which might be even longer than the
> > > > >> > productive
> > > > >>
> > > > >> build
> > > > >>
> > > > >> > itself (compiler, package, ...).
> > > > >> >
> > > > >> > We discussed this topic with Herve and Karl at the Apache CON
> 2019
> > >
> > > the
> > >
> > > > >> last
> > > > >>
> > > > >> > time.
> > > > >> >
> > > > >> > Sometime the presentations were funny because the audience had
> to
> > > >
> > > > wait a
> > > >
> > > > >> > minute while the console was black where the Maven was
> downloading
> > >
> > > the
> > >
> > > > >> > plugins in the background.
> > > > >> > Nobody was sure what happened that time, whether the console
> hanged
> > >
> > > or
> > >
> > > > >> the
> > > > >>
> > > > >> > Cloud server hanged, or another issue happened with the network.
> > > > >> >
> > > > >> > I made a test and triggered the default lifecycle on Maven and I
> > > > >>
> > > > >> realized
> > > > >>
> > > > >> > that the cache was really very little, cca 12 MB.
> > > > >> > So this little cache in the container would save 50 seconds
> which
> > > > >> > is
> > > >
> > > > the
> > > >
> > > > >> > improvement we are discussing.
> > > > >> >
> > > > >> > From the use point of view, the user would use a new base image
> `
> > > > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > > > >> >
> > > > >> > There are multiple technical solutions (range of plugins, extra
> > > > >> > pom,
> > > > >> > internal Maven plugin versions, etc).
> > > > >> >
> > > > >> > We understood that the best idea would be to have the image with
> > > > >> > the
> > > > >>
> > > > >> cache
> > > > >>
> > > > >> > in new Docker images produced by Carloss Sanchez.
> > > > >> >
> > > > >> > We are discussing this topic in [1] but we do not have a
> consensus
> > >
> > > on
> > >
> > > > >> who
> > > > >>
> > > > >> > will develop the Docker scripts and how.
> > > > >> >
> > > > >> > We can continue here and we can propose a solution.
> > > > >> >
> > > > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > > > >> >
> > > > >> >
> > > > >> > Cheers
> > > > >> > Tibor17
> > > > >
> > > > > --
> > > >
> > > > Sent from my phone
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Romain Manni-Bucau <rm...@gmail.com>.
What is the usage of this first group outside demos (it is trivial to build
a dedicated image for demos so i drop it from any asf work for now)?
Le mer. 30 oct. 2019 à 23:18, Tibor Digana <ti...@apache.org> a
écrit :
> Romain, of course we know this but there are two groups of users.
> First those who use the Maven in docker without CI.
> And second of those who integrate the docker container with the CI and they
> use the volumes.
>
> The first group of people may freely use the image we are discussing here
> because this makes the default build lifecycle faster for them.
> For us it means that we will not be seens as "slow Maven", you know what i
> mean.
>
> The second group already use the maven images from Carloss.
>
> On Wed, Oct 30, 2019 at 11:12 PM Romain Manni-Bucau <rmannibucau@gmail.com
> >
> wrote:
>
> > Ci servers have all cache so it is useless to make the docker pull slow
> > with useless versions to then download or not plugins IMHO.
> >
> > Le mer. 30 oct. 2019 à 22:47, Hervé Boutemy <hb...@apache.org> a
> écrit
> > :
> >
> > > Le mercredi 30 octobre 2019 14:55:46 CET, vous avez écrit :
> > > > Agree we should publish with an asf account and make it part of the
> > > release
> > > > process.
> > > > That said I still fail to see how you can add a relevant cache. Maybe
> > > take
> > > > the time to review plugin version in a few asf projects (let say
> > > > maven-surefire, geronimo-openapi and spark) and check out if it works
> > to
> > > > cache anything.
> > > if we choose one precise version of each plugin, we will never get the
> > > version
> > > used by a project: IMHO, we need to choose a range for each plugin
> > > https://maven.apache.org/plugins/
> > > Something like dependency:resolve-plugins should be able to do the job.
> > >
> > > With a range per plugin, I'd love to see the size of the initial local
> > > repository
> > > At least, using such an image would require minimal efforts from users,
> > > particularly on CI servers
> > >
> > > Regards,
> > >
> > > Hervé
> > >
> > > >
> > > > Romain Manni-Bucau
> > > > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > > <http://rmannibucau.wordpress.com> | Github <
> > > https://github.com/rmannibucau>
> > > > | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > > <
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > >
> > > >
> > > >
> > > > Le mer. 30 oct. 2019 à 14:31, Tibor Digana <ti...@apache.org>
> a
> > > >
> > > > écrit :
> > > > > Stephen, yeah something like you do in your scrip but it must not
> be
> > a
> > > > > personal owner. Even Carloss is person who makes this deployment to
> > > > > DockerHub but his images are used by the entire world and we should
> > > decide
> > > > > whether we would agree with him to have such images under his
> > > > > responsibility or our responsibility as the Apache group. Then the
> > > script
> > > > > would be official and we can cut the release of Maven and the
> Docker
> > > image
> > > > > will be ready for the downloads together with the Maven
> distribution.
> > > So
> > > > > the users will always know that it is consistent deployment and
> they
> > > > > wouldn't expect that the image is missing for the new version.
> > > > >
> > > > > So here we should decide on who will be the deployer of these
> images
> > > with
> > > > > the cache. And the technical solution is smaller problem I would
> say.
> > > > >
> > > > > On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> > > > >
> > > > > stephen.alan.connolly@gmail.com> wrote:
> > > > > > On Wed 30 Oct 2019 at 08:21, Tibor Digana <
> tibordigana@apache.org>
> > > > >
> > > > > wrote:
> > > > > > > It's the situation when you have maven plugins in repo and it
> > means
> > > > >
> > > > > that
> > > > >
> > > > > > > all custom plugins/deps can be still downloaded as before.
> > > > > > > Nothing exists like this in the world and we are talking about
> > the
> > > > > > > approaches.
> > > > > >
> > > > > > Cough cough cough
> > > > >
> > > > >
> > >
> >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae41
> > > > > 8ca3fd63c63038e896255c6fe8/Dockerfile#L50>
> > > > > > There are issues, but it does shave a bit of time... though we
> end
> > up
> > > > > > adding our common dependencies into the seed pom so that it is an
> > > > >
> > > > > effective
> > > > >
> > > > > > speed up
> > > > > >
> > > > > > > I added Karl, Herve and Stephen in CC because we talked about
> > this
> > > > >
> > > > > issue
> > > > >
> > > > > > > in ASF CON and Twitter.
> > > > > > >
> > > > > > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > > > > >
> > > > > > rmannibucau@gmail.com>
> > > > > >
> > > > > > > wrote:
> > > > > > >> Hi Tibor,
> > > > > > >>
> > > > > > >> It has two issues:
> > > > > > >>
> > > > > > >> 1. It will not be the right plugin versions in 90% of the
> cases
> > > > >
> > > > > (except
> > > > >
> > > > > > >> demo ;))
> > > > > > >> 2. It will miss all custom plugins
> > > > > > >>
> > > > > > >> Now question is: what happens if you mount your local repo
> when
> > > > >
> > > > > running
> > > > >
> > > > > > >> docker? It works as expected. Means we could use a custom
> > > entrypoint
> > > > > > >> printing a warning banner if it is not done probably instead
> of
> > > > > >
> > > > > > incrasing
> > > > > >
> > > > > > >> the image size without being sure to reach the original goal.
> > > > > > >>
> > > > > > >> Wdyt?
> > > > > > >>
> > > > > > >> Romain
> > > > > > >>
> > > > > > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <
> > > tibordigana@apache.org> a
> > > > > > >>
> > > > > > >> écrit :
> > > > > > >> > If you use Docker images with Maven with no mapping of cache
> > to
> > > the
> > > > > > >> > volumes, you may notice that Maven downloads the plugins for
> > the
> > > > >
> > > > > build
> > > > >
> > > > > > >> > lifecycle.
> > > > > > >> >
> > > > > > >> > This slows down the build because a lot of artifacts and
> > plugins
> > > > > > >> > are
> > > > > > >> > initially downloaded.
> > > > > > >> > This takes 50 seconds which might be even longer than the
> > > > > > >> > productive
> > > > > > >>
> > > > > > >> build
> > > > > > >>
> > > > > > >> > itself (compiler, package, ...).
> > > > > > >> >
> > > > > > >> > We discussed this topic with Herve and Karl at the Apache
> CON
> > > 2019
> > > > >
> > > > > the
> > > > >
> > > > > > >> last
> > > > > > >>
> > > > > > >> > time.
> > > > > > >> >
> > > > > > >> > Sometime the presentations were funny because the audience
> had
> > > to
> > > > > >
> > > > > > wait a
> > > > > >
> > > > > > >> > minute while the console was black where the Maven was
> > > downloading
> > > > >
> > > > > the
> > > > >
> > > > > > >> > plugins in the background.
> > > > > > >> > Nobody was sure what happened that time, whether the console
> > > hanged
> > > > >
> > > > > or
> > > > >
> > > > > > >> the
> > > > > > >>
> > > > > > >> > Cloud server hanged, or another issue happened with the
> > network.
> > > > > > >> >
> > > > > > >> > I made a test and triggered the default lifecycle on Maven
> > and I
> > > > > > >>
> > > > > > >> realized
> > > > > > >>
> > > > > > >> > that the cache was really very little, cca 12 MB.
> > > > > > >> > So this little cache in the container would save 50 seconds
> > > which
> > > > > > >> > is
> > > > > >
> > > > > > the
> > > > > >
> > > > > > >> > improvement we are discussing.
> > > > > > >> >
> > > > > > >> > From the use point of view, the user would use a new base
> > image
> > > `
> > > > > > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > > > > > >> >
> > > > > > >> > There are multiple technical solutions (range of plugins,
> > extra
> > > > > > >> > pom,
> > > > > > >> > internal Maven plugin versions, etc).
> > > > > > >> >
> > > > > > >> > We understood that the best idea would be to have the image
> > with
> > > > > > >> > the
> > > > > > >>
> > > > > > >> cache
> > > > > > >>
> > > > > > >> > in new Docker images produced by Carloss Sanchez.
> > > > > > >> >
> > > > > > >> > We are discussing this topic in [1] but we do not have a
> > > consensus
> > > > >
> > > > > on
> > > > >
> > > > > > >> who
> > > > > > >>
> > > > > > >> > will develop the Docker scripts and how.
> > > > > > >> >
> > > > > > >> > We can continue here and we can propose a solution.
> > > > > > >> >
> > > > > > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > Cheers
> > > > > > >> > Tibor17
> > > > > > >
> > > > > > > --
> > > > > >
> > > > > > Sent from my phone
> > >
> > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > > For additional commands, e-mail: dev-help@maven.apache.org
> > >
> > >
> >
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Tibor Digana <ti...@apache.org>.
Romain, of course we know this but there are two groups of users.
First those who use the Maven in docker without CI.
And second of those who integrate the docker container with the CI and they
use the volumes.
The first group of people may freely use the image we are discussing here
because this makes the default build lifecycle faster for them.
For us it means that we will not be seens as "slow Maven", you know what i
mean.
The second group already use the maven images from Carloss.
On Wed, Oct 30, 2019 at 11:12 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Ci servers have all cache so it is useless to make the docker pull slow
> with useless versions to then download or not plugins IMHO.
>
> Le mer. 30 oct. 2019 à 22:47, Hervé Boutemy <hb...@apache.org> a écrit
> :
>
> > Le mercredi 30 octobre 2019 14:55:46 CET, vous avez écrit :
> > > Agree we should publish with an asf account and make it part of the
> > release
> > > process.
> > > That said I still fail to see how you can add a relevant cache. Maybe
> > take
> > > the time to review plugin version in a few asf projects (let say
> > > maven-surefire, geronimo-openapi and spark) and check out if it works
> to
> > > cache anything.
> > if we choose one precise version of each plugin, we will never get the
> > version
> > used by a project: IMHO, we need to choose a range for each plugin
> > https://maven.apache.org/plugins/
> > Something like dependency:resolve-plugins should be able to do the job.
> >
> > With a range per plugin, I'd love to see the size of the initial local
> > repository
> > At least, using such an image would require minimal efforts from users,
> > particularly on CI servers
> >
> > Regards,
> >
> > Hervé
> >
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > <http://rmannibucau.wordpress.com> | Github <
> > https://github.com/rmannibucau>
> > > | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > <
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le mer. 30 oct. 2019 à 14:31, Tibor Digana <ti...@apache.org> a
> > >
> > > écrit :
> > > > Stephen, yeah something like you do in your scrip but it must not be
> a
> > > > personal owner. Even Carloss is person who makes this deployment to
> > > > DockerHub but his images are used by the entire world and we should
> > decide
> > > > whether we would agree with him to have such images under his
> > > > responsibility or our responsibility as the Apache group. Then the
> > script
> > > > would be official and we can cut the release of Maven and the Docker
> > image
> > > > will be ready for the downloads together with the Maven distribution.
> > So
> > > > the users will always know that it is consistent deployment and they
> > > > wouldn't expect that the image is missing for the new version.
> > > >
> > > > So here we should decide on who will be the deployer of these images
> > with
> > > > the cache. And the technical solution is smaller problem I would say.
> > > >
> > > > On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> > > >
> > > > stephen.alan.connolly@gmail.com> wrote:
> > > > > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> > > >
> > > > wrote:
> > > > > > It's the situation when you have maven plugins in repo and it
> means
> > > >
> > > > that
> > > >
> > > > > > all custom plugins/deps can be still downloaded as before.
> > > > > > Nothing exists like this in the world and we are talking about
> the
> > > > > > approaches.
> > > > >
> > > > > Cough cough cough
> > > >
> > > >
> >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae41
> > > > 8ca3fd63c63038e896255c6fe8/Dockerfile#L50>
> > > > > There are issues, but it does shave a bit of time... though we end
> up
> > > > > adding our common dependencies into the seed pom so that it is an
> > > >
> > > > effective
> > > >
> > > > > speed up
> > > > >
> > > > > > I added Karl, Herve and Stephen in CC because we talked about
> this
> > > >
> > > > issue
> > > >
> > > > > > in ASF CON and Twitter.
> > > > > >
> > > > > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > > > >
> > > > > rmannibucau@gmail.com>
> > > > >
> > > > > > wrote:
> > > > > >> Hi Tibor,
> > > > > >>
> > > > > >> It has two issues:
> > > > > >>
> > > > > >> 1. It will not be the right plugin versions in 90% of the cases
> > > >
> > > > (except
> > > >
> > > > > >> demo ;))
> > > > > >> 2. It will miss all custom plugins
> > > > > >>
> > > > > >> Now question is: what happens if you mount your local repo when
> > > >
> > > > running
> > > >
> > > > > >> docker? It works as expected. Means we could use a custom
> > entrypoint
> > > > > >> printing a warning banner if it is not done probably instead of
> > > > >
> > > > > incrasing
> > > > >
> > > > > >> the image size without being sure to reach the original goal.
> > > > > >>
> > > > > >> Wdyt?
> > > > > >>
> > > > > >> Romain
> > > > > >>
> > > > > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <
> > tibordigana@apache.org> a
> > > > > >>
> > > > > >> écrit :
> > > > > >> > If you use Docker images with Maven with no mapping of cache
> to
> > the
> > > > > >> > volumes, you may notice that Maven downloads the plugins for
> the
> > > >
> > > > build
> > > >
> > > > > >> > lifecycle.
> > > > > >> >
> > > > > >> > This slows down the build because a lot of artifacts and
> plugins
> > > > > >> > are
> > > > > >> > initially downloaded.
> > > > > >> > This takes 50 seconds which might be even longer than the
> > > > > >> > productive
> > > > > >>
> > > > > >> build
> > > > > >>
> > > > > >> > itself (compiler, package, ...).
> > > > > >> >
> > > > > >> > We discussed this topic with Herve and Karl at the Apache CON
> > 2019
> > > >
> > > > the
> > > >
> > > > > >> last
> > > > > >>
> > > > > >> > time.
> > > > > >> >
> > > > > >> > Sometime the presentations were funny because the audience had
> > to
> > > > >
> > > > > wait a
> > > > >
> > > > > >> > minute while the console was black where the Maven was
> > downloading
> > > >
> > > > the
> > > >
> > > > > >> > plugins in the background.
> > > > > >> > Nobody was sure what happened that time, whether the console
> > hanged
> > > >
> > > > or
> > > >
> > > > > >> the
> > > > > >>
> > > > > >> > Cloud server hanged, or another issue happened with the
> network.
> > > > > >> >
> > > > > >> > I made a test and triggered the default lifecycle on Maven
> and I
> > > > > >>
> > > > > >> realized
> > > > > >>
> > > > > >> > that the cache was really very little, cca 12 MB.
> > > > > >> > So this little cache in the container would save 50 seconds
> > which
> > > > > >> > is
> > > > >
> > > > > the
> > > > >
> > > > > >> > improvement we are discussing.
> > > > > >> >
> > > > > >> > From the use point of view, the user would use a new base
> image
> > `
> > > > > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > > > > >> >
> > > > > >> > There are multiple technical solutions (range of plugins,
> extra
> > > > > >> > pom,
> > > > > >> > internal Maven plugin versions, etc).
> > > > > >> >
> > > > > >> > We understood that the best idea would be to have the image
> with
> > > > > >> > the
> > > > > >>
> > > > > >> cache
> > > > > >>
> > > > > >> > in new Docker images produced by Carloss Sanchez.
> > > > > >> >
> > > > > >> > We are discussing this topic in [1] but we do not have a
> > consensus
> > > >
> > > > on
> > > >
> > > > > >> who
> > > > > >>
> > > > > >> > will develop the Docker scripts and how.
> > > > > >> >
> > > > > >> > We can continue here and we can propose a solution.
> > > > > >> >
> > > > > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > > > > >> >
> > > > > >> >
> > > > > >> > Cheers
> > > > > >> > Tibor17
> > > > > >
> > > > > > --
> > > > >
> > > > > Sent from my phone
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> > For additional commands, e-mail: dev-help@maven.apache.org
> >
> >
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Ci servers have all cache so it is useless to make the docker pull slow
with useless versions to then download or not plugins IMHO.
Le mer. 30 oct. 2019 à 22:47, Hervé Boutemy <hb...@apache.org> a écrit :
> Le mercredi 30 octobre 2019 14:55:46 CET, vous avez écrit :
> > Agree we should publish with an asf account and make it part of the
> release
> > process.
> > That said I still fail to see how you can add a relevant cache. Maybe
> take
> > the time to review plugin version in a few asf projects (let say
> > maven-surefire, geronimo-openapi and spark) and check out if it works to
> > cache anything.
> if we choose one precise version of each plugin, we will never get the
> version
> used by a project: IMHO, we need to choose a range for each plugin
> https://maven.apache.org/plugins/
> Something like dependency:resolve-plugins should be able to do the job.
>
> With a range per plugin, I'd love to see the size of the initial local
> repository
> At least, using such an image would require minimal efforts from users,
> particularly on CI servers
>
> Regards,
>
> Hervé
>
> >
> > Romain Manni-Bucau
> > @rmannibucau <https://twitter.com/rmannibucau> | Blog
> > <https://rmannibucau.metawerx.net/> | Old Blog
> > <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau>
> > | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > >
> >
> >
> > Le mer. 30 oct. 2019 à 14:31, Tibor Digana <ti...@apache.org> a
> >
> > écrit :
> > > Stephen, yeah something like you do in your scrip but it must not be a
> > > personal owner. Even Carloss is person who makes this deployment to
> > > DockerHub but his images are used by the entire world and we should
> decide
> > > whether we would agree with him to have such images under his
> > > responsibility or our responsibility as the Apache group. Then the
> script
> > > would be official and we can cut the release of Maven and the Docker
> image
> > > will be ready for the downloads together with the Maven distribution.
> So
> > > the users will always know that it is consistent deployment and they
> > > wouldn't expect that the image is missing for the new version.
> > >
> > > So here we should decide on who will be the deployer of these images
> with
> > > the cache. And the technical solution is smaller problem I would say.
> > >
> > > On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> > >
> > > stephen.alan.connolly@gmail.com> wrote:
> > > > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> > >
> > > wrote:
> > > > > It's the situation when you have maven plugins in repo and it means
> > >
> > > that
> > >
> > > > > all custom plugins/deps can be still downloaded as before.
> > > > > Nothing exists like this in the world and we are talking about the
> > > > > approaches.
> > > >
> > > > Cough cough cough
> > >
> > >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae41
> > > 8ca3fd63c63038e896255c6fe8/Dockerfile#L50>
> > > > There are issues, but it does shave a bit of time... though we end up
> > > > adding our common dependencies into the seed pom so that it is an
> > >
> > > effective
> > >
> > > > speed up
> > > >
> > > > > I added Karl, Herve and Stephen in CC because we talked about this
> > >
> > > issue
> > >
> > > > > in ASF CON and Twitter.
> > > > >
> > > > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > > >
> > > > rmannibucau@gmail.com>
> > > >
> > > > > wrote:
> > > > >> Hi Tibor,
> > > > >>
> > > > >> It has two issues:
> > > > >>
> > > > >> 1. It will not be the right plugin versions in 90% of the cases
> > >
> > > (except
> > >
> > > > >> demo ;))
> > > > >> 2. It will miss all custom plugins
> > > > >>
> > > > >> Now question is: what happens if you mount your local repo when
> > >
> > > running
> > >
> > > > >> docker? It works as expected. Means we could use a custom
> entrypoint
> > > > >> printing a warning banner if it is not done probably instead of
> > > >
> > > > incrasing
> > > >
> > > > >> the image size without being sure to reach the original goal.
> > > > >>
> > > > >> Wdyt?
> > > > >>
> > > > >> Romain
> > > > >>
> > > > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <
> tibordigana@apache.org> a
> > > > >>
> > > > >> écrit :
> > > > >> > If you use Docker images with Maven with no mapping of cache to
> the
> > > > >> > volumes, you may notice that Maven downloads the plugins for the
> > >
> > > build
> > >
> > > > >> > lifecycle.
> > > > >> >
> > > > >> > This slows down the build because a lot of artifacts and plugins
> > > > >> > are
> > > > >> > initially downloaded.
> > > > >> > This takes 50 seconds which might be even longer than the
> > > > >> > productive
> > > > >>
> > > > >> build
> > > > >>
> > > > >> > itself (compiler, package, ...).
> > > > >> >
> > > > >> > We discussed this topic with Herve and Karl at the Apache CON
> 2019
> > >
> > > the
> > >
> > > > >> last
> > > > >>
> > > > >> > time.
> > > > >> >
> > > > >> > Sometime the presentations were funny because the audience had
> to
> > > >
> > > > wait a
> > > >
> > > > >> > minute while the console was black where the Maven was
> downloading
> > >
> > > the
> > >
> > > > >> > plugins in the background.
> > > > >> > Nobody was sure what happened that time, whether the console
> hanged
> > >
> > > or
> > >
> > > > >> the
> > > > >>
> > > > >> > Cloud server hanged, or another issue happened with the network.
> > > > >> >
> > > > >> > I made a test and triggered the default lifecycle on Maven and I
> > > > >>
> > > > >> realized
> > > > >>
> > > > >> > that the cache was really very little, cca 12 MB.
> > > > >> > So this little cache in the container would save 50 seconds
> which
> > > > >> > is
> > > >
> > > > the
> > > >
> > > > >> > improvement we are discussing.
> > > > >> >
> > > > >> > From the use point of view, the user would use a new base image
> `
> > > > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > > > >> >
> > > > >> > There are multiple technical solutions (range of plugins, extra
> > > > >> > pom,
> > > > >> > internal Maven plugin versions, etc).
> > > > >> >
> > > > >> > We understood that the best idea would be to have the image with
> > > > >> > the
> > > > >>
> > > > >> cache
> > > > >>
> > > > >> > in new Docker images produced by Carloss Sanchez.
> > > > >> >
> > > > >> > We are discussing this topic in [1] but we do not have a
> consensus
> > >
> > > on
> > >
> > > > >> who
> > > > >>
> > > > >> > will develop the Docker scripts and how.
> > > > >> >
> > > > >> > We can continue here and we can propose a solution.
> > > > >> >
> > > > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > > > >> >
> > > > >> >
> > > > >> > Cheers
> > > > >> > Tibor17
> > > > >
> > > > > --
> > > >
> > > > Sent from my phone
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Hervé Boutemy <hb...@apache.org>.
Le mercredi 30 octobre 2019 14:55:46 CET, vous avez écrit :
> Agree we should publish with an asf account and make it part of the release
> process.
> That said I still fail to see how you can add a relevant cache. Maybe take
> the time to review plugin version in a few asf projects (let say
> maven-surefire, geronimo-openapi and spark) and check out if it works to
> cache anything.
if we choose one precise version of each plugin, we will never get the version
used by a project: IMHO, we need to choose a range for each plugin
https://maven.apache.org/plugins/
Something like dependency:resolve-plugins should be able to do the job.
With a range per plugin, I'd love to see the size of the initial local
repository
At least, using such an image would require minimal efforts from users,
particularly on CI servers
Regards,
Hervé
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau>
> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le mer. 30 oct. 2019 à 14:31, Tibor Digana <ti...@apache.org> a
>
> écrit :
> > Stephen, yeah something like you do in your scrip but it must not be a
> > personal owner. Even Carloss is person who makes this deployment to
> > DockerHub but his images are used by the entire world and we should decide
> > whether we would agree with him to have such images under his
> > responsibility or our responsibility as the Apache group. Then the script
> > would be official and we can cut the release of Maven and the Docker image
> > will be ready for the downloads together with the Maven distribution. So
> > the users will always know that it is consistent deployment and they
> > wouldn't expect that the image is missing for the new version.
> >
> > So here we should decide on who will be the deployer of these images with
> > the cache. And the technical solution is smaller problem I would say.
> >
> > On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> >
> > stephen.alan.connolly@gmail.com> wrote:
> > > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> >
> > wrote:
> > > > It's the situation when you have maven plugins in repo and it means
> >
> > that
> >
> > > > all custom plugins/deps can be still downloaded as before.
> > > > Nothing exists like this in the world and we are talking about the
> > > > approaches.
> > >
> > > Cough cough cough
> >
> > https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae41
> > 8ca3fd63c63038e896255c6fe8/Dockerfile#L50>
> > > There are issues, but it does shave a bit of time... though we end up
> > > adding our common dependencies into the seed pom so that it is an
> >
> > effective
> >
> > > speed up
> > >
> > > > I added Karl, Herve and Stephen in CC because we talked about this
> >
> > issue
> >
> > > > in ASF CON and Twitter.
> > > >
> > > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > >
> > > rmannibucau@gmail.com>
> > >
> > > > wrote:
> > > >> Hi Tibor,
> > > >>
> > > >> It has two issues:
> > > >>
> > > >> 1. It will not be the right plugin versions in 90% of the cases
> >
> > (except
> >
> > > >> demo ;))
> > > >> 2. It will miss all custom plugins
> > > >>
> > > >> Now question is: what happens if you mount your local repo when
> >
> > running
> >
> > > >> docker? It works as expected. Means we could use a custom entrypoint
> > > >> printing a warning banner if it is not done probably instead of
> > >
> > > incrasing
> > >
> > > >> the image size without being sure to reach the original goal.
> > > >>
> > > >> Wdyt?
> > > >>
> > > >> Romain
> > > >>
> > > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
> > > >>
> > > >> écrit :
> > > >> > If you use Docker images with Maven with no mapping of cache to the
> > > >> > volumes, you may notice that Maven downloads the plugins for the
> >
> > build
> >
> > > >> > lifecycle.
> > > >> >
> > > >> > This slows down the build because a lot of artifacts and plugins
> > > >> > are
> > > >> > initially downloaded.
> > > >> > This takes 50 seconds which might be even longer than the
> > > >> > productive
> > > >>
> > > >> build
> > > >>
> > > >> > itself (compiler, package, ...).
> > > >> >
> > > >> > We discussed this topic with Herve and Karl at the Apache CON 2019
> >
> > the
> >
> > > >> last
> > > >>
> > > >> > time.
> > > >> >
> > > >> > Sometime the presentations were funny because the audience had to
> > >
> > > wait a
> > >
> > > >> > minute while the console was black where the Maven was downloading
> >
> > the
> >
> > > >> > plugins in the background.
> > > >> > Nobody was sure what happened that time, whether the console hanged
> >
> > or
> >
> > > >> the
> > > >>
> > > >> > Cloud server hanged, or another issue happened with the network.
> > > >> >
> > > >> > I made a test and triggered the default lifecycle on Maven and I
> > > >>
> > > >> realized
> > > >>
> > > >> > that the cache was really very little, cca 12 MB.
> > > >> > So this little cache in the container would save 50 seconds which
> > > >> > is
> > >
> > > the
> > >
> > > >> > improvement we are discussing.
> > > >> >
> > > >> > From the use point of view, the user would use a new base image `
> > > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > > >> >
> > > >> > There are multiple technical solutions (range of plugins, extra
> > > >> > pom,
> > > >> > internal Maven plugin versions, etc).
> > > >> >
> > > >> > We understood that the best idea would be to have the image with
> > > >> > the
> > > >>
> > > >> cache
> > > >>
> > > >> > in new Docker images produced by Carloss Sanchez.
> > > >> >
> > > >> > We are discussing this topic in [1] but we do not have a consensus
> >
> > on
> >
> > > >> who
> > > >>
> > > >> > will develop the Docker scripts and how.
> > > >> >
> > > >> > We can continue here and we can propose a solution.
> > > >> >
> > > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > > >> >
> > > >> >
> > > >> > Cheers
> > > >> > Tibor17
> > > >
> > > > --
> > >
> > > Sent from my phone
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Agree we should publish with an asf account and make it part of the release
process.
That said I still fail to see how you can add a relevant cache. Maybe take
the time to review plugin version in a few asf projects (let say
maven-surefire, geronimo-openapi and spark) and check out if it works to
cache anything.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le mer. 30 oct. 2019 à 14:31, Tibor Digana <ti...@apache.org> a
écrit :
> Stephen, yeah something like you do in your scrip but it must not be a
> personal owner. Even Carloss is person who makes this deployment to
> DockerHub but his images are used by the entire world and we should decide
> whether we would agree with him to have such images under his
> responsibility or our responsibility as the Apache group. Then the script
> would be official and we can cut the release of Maven and the Docker image
> will be ready for the downloads together with the Maven distribution. So
> the users will always know that it is consistent deployment and they
> wouldn't expect that the image is missing for the new version.
>
> So here we should decide on who will be the deployer of these images with
> the cache. And the technical solution is smaller problem I would say.
>
> On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> wrote:
> >
> > > It's the situation when you have maven plugins in repo and it means
> that
> > > all custom plugins/deps can be still downloaded as before.
> > > Nothing exists like this in the world and we are talking about the
> > > approaches.
> > >
> >
> > Cough cough cough
> >
> >
> >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae418ca3fd63c63038e896255c6fe8/Dockerfile#L50
> >
> > There are issues, but it does shave a bit of time... though we end up
> > adding our common dependencies into the seed pom so that it is an
> effective
> > speed up
> >
> >
> > >
> > > I added Karl, Herve and Stephen in CC because we talked about this
> issue
> > > in ASF CON and Twitter.
> > >
> > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > rmannibucau@gmail.com>
> > > wrote:
> > >
> > >> Hi Tibor,
> > >>
> > >> It has two issues:
> > >>
> > >> 1. It will not be the right plugin versions in 90% of the cases
> (except
> > >> demo ;))
> > >> 2. It will miss all custom plugins
> > >>
> > >> Now question is: what happens if you mount your local repo when
> running
> > >> docker? It works as expected. Means we could use a custom entrypoint
> > >> printing a warning banner if it is not done probably instead of
> > incrasing
> > >> the image size without being sure to reach the original goal.
> > >>
> > >> Wdyt?
> > >>
> > >> Romain
> > >>
> > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
> > >> écrit :
> > >>
> > >> > If you use Docker images with Maven with no mapping of cache to the
> > >> > volumes, you may notice that Maven downloads the plugins for the
> build
> > >> > lifecycle.
> > >> >
> > >> > This slows down the build because a lot of artifacts and plugins are
> > >> > initially downloaded.
> > >> > This takes 50 seconds which might be even longer than the productive
> > >> build
> > >> > itself (compiler, package, ...).
> > >> >
> > >> > We discussed this topic with Herve and Karl at the Apache CON 2019
> the
> > >> last
> > >> > time.
> > >> >
> > >> > Sometime the presentations were funny because the audience had to
> > wait a
> > >> > minute while the console was black where the Maven was downloading
> the
> > >> > plugins in the background.
> > >> > Nobody was sure what happened that time, whether the console hanged
> or
> > >> the
> > >> > Cloud server hanged, or another issue happened with the network.
> > >> >
> > >> > I made a test and triggered the default lifecycle on Maven and I
> > >> realized
> > >> > that the cache was really very little, cca 12 MB.
> > >> > So this little cache in the container would save 50 seconds which is
> > the
> > >> > improvement we are discussing.
> > >> >
> > >> > From the use point of view, the user would use a new base image `
> > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > >> >
> > >> > There are multiple technical solutions (range of plugins, extra pom,
> > >> > internal Maven plugin versions, etc).
> > >> >
> > >> > We understood that the best idea would be to have the image with the
> > >> cache
> > >> > in new Docker images produced by Carloss Sanchez.
> > >> >
> > >> > We are discussing this topic in [1] but we do not have a consensus
> on
> > >> who
> > >> > will develop the Docker scripts and how.
> > >> >
> > >> > We can continue here and we can propose a solution.
> > >> >
> > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > >> >
> > >> >
> > >> > Cheers
> > >> > Tibor17
> > >> >
> > >>
> > > --
> > Sent from my phone
> >
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Tibor Digana <ti...@apache.org>.
Stephen, this is clear to me but we need to move ahead a bit.
Should I start the vote that we will have new project (
github.com/apache/maven-docker) for these purposes?
Or should we talk to Carloss and ask him if he would be fine with deploying
these images in DockerHub?
On Wed, Oct 30, 2019 at 3:21 PM Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> My script is for my cases. I'm just saying it's not rocket science, others
> are doing it already
>
> On Wed, 30 Oct 2019 at 13:31, Tibor Digana <ti...@apache.org> wrote:
>
> > Stephen, yeah something like you do in your scrip but it must not be a
> > personal owner. Even Carloss is person who makes this deployment to
> > DockerHub but his images are used by the entire world and we should
> decide
> > whether we would agree with him to have such images under his
> > responsibility or our responsibility as the Apache group. Then the script
> > would be official and we can cut the release of Maven and the Docker
> image
> > will be ready for the downloads together with the Maven distribution. So
> > the users will always know that it is consistent deployment and they
> > wouldn't expect that the image is missing for the new version.
> >
> > So here we should decide on who will be the deployer of these images with
> > the cache. And the technical solution is smaller problem I would say.
> >
> > On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> > stephen.alan.connolly@gmail.com> wrote:
> >
> > > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> > wrote:
> > >
> > > > It's the situation when you have maven plugins in repo and it means
> > that
> > > > all custom plugins/deps can be still downloaded as before.
> > > > Nothing exists like this in the world and we are talking about the
> > > > approaches.
> > > >
> > >
> > > Cough cough cough
> > >
> > >
> > >
> >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae418ca3fd63c63038e896255c6fe8/Dockerfile#L50
> > >
> > > There are issues, but it does shave a bit of time... though we end up
> > > adding our common dependencies into the seed pom so that it is an
> > effective
> > > speed up
> > >
> > >
> > > >
> > > > I added Karl, Herve and Stephen in CC because we talked about this
> > issue
> > > > in ASF CON and Twitter.
> > > >
> > > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > > rmannibucau@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi Tibor,
> > > >>
> > > >> It has two issues:
> > > >>
> > > >> 1. It will not be the right plugin versions in 90% of the cases
> > (except
> > > >> demo ;))
> > > >> 2. It will miss all custom plugins
> > > >>
> > > >> Now question is: what happens if you mount your local repo when
> > running
> > > >> docker? It works as expected. Means we could use a custom entrypoint
> > > >> printing a warning banner if it is not done probably instead of
> > > incrasing
> > > >> the image size without being sure to reach the original goal.
> > > >>
> > > >> Wdyt?
> > > >>
> > > >> Romain
> > > >>
> > > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org>
> a
> > > >> écrit :
> > > >>
> > > >> > If you use Docker images with Maven with no mapping of cache to
> the
> > > >> > volumes, you may notice that Maven downloads the plugins for the
> > build
> > > >> > lifecycle.
> > > >> >
> > > >> > This slows down the build because a lot of artifacts and plugins
> are
> > > >> > initially downloaded.
> > > >> > This takes 50 seconds which might be even longer than the
> productive
> > > >> build
> > > >> > itself (compiler, package, ...).
> > > >> >
> > > >> > We discussed this topic with Herve and Karl at the Apache CON 2019
> > the
> > > >> last
> > > >> > time.
> > > >> >
> > > >> > Sometime the presentations were funny because the audience had to
> > > wait a
> > > >> > minute while the console was black where the Maven was downloading
> > the
> > > >> > plugins in the background.
> > > >> > Nobody was sure what happened that time, whether the console
> hanged
> > or
> > > >> the
> > > >> > Cloud server hanged, or another issue happened with the network.
> > > >> >
> > > >> > I made a test and triggered the default lifecycle on Maven and I
> > > >> realized
> > > >> > that the cache was really very little, cca 12 MB.
> > > >> > So this little cache in the container would save 50 seconds which
> is
> > > the
> > > >> > improvement we are discussing.
> > > >> >
> > > >> > From the use point of view, the user would use a new base image `
> > > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > > >> >
> > > >> > There are multiple technical solutions (range of plugins, extra
> pom,
> > > >> > internal Maven plugin versions, etc).
> > > >> >
> > > >> > We understood that the best idea would be to have the image with
> the
> > > >> cache
> > > >> > in new Docker images produced by Carloss Sanchez.
> > > >> >
> > > >> > We are discussing this topic in [1] but we do not have a consensus
> > on
> > > >> who
> > > >> > will develop the Docker scripts and how.
> > > >> >
> > > >> > We can continue here and we can propose a solution.
> > > >> >
> > > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > > >> >
> > > >> >
> > > >> > Cheers
> > > >> > Tibor17
> > > >> >
> > > >>
> > > > --
> > > Sent from my phone
> > >
> >
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Stephen Connolly <st...@gmail.com>.
My script is for my cases. I'm just saying it's not rocket science, others
are doing it already
On Wed, 30 Oct 2019 at 13:31, Tibor Digana <ti...@apache.org> wrote:
> Stephen, yeah something like you do in your scrip but it must not be a
> personal owner. Even Carloss is person who makes this deployment to
> DockerHub but his images are used by the entire world and we should decide
> whether we would agree with him to have such images under his
> responsibility or our responsibility as the Apache group. Then the script
> would be official and we can cut the release of Maven and the Docker image
> will be ready for the downloads together with the Maven distribution. So
> the users will always know that it is consistent deployment and they
> wouldn't expect that the image is missing for the new version.
>
> So here we should decide on who will be the deployer of these images with
> the cache. And the technical solution is smaller problem I would say.
>
> On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
> stephen.alan.connolly@gmail.com> wrote:
>
> > On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org>
> wrote:
> >
> > > It's the situation when you have maven plugins in repo and it means
> that
> > > all custom plugins/deps can be still downloaded as before.
> > > Nothing exists like this in the world and we are talking about the
> > > approaches.
> > >
> >
> > Cough cough cough
> >
> >
> >
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae418ca3fd63c63038e896255c6fe8/Dockerfile#L50
> >
> > There are issues, but it does shave a bit of time... though we end up
> > adding our common dependencies into the seed pom so that it is an
> effective
> > speed up
> >
> >
> > >
> > > I added Karl, Herve and Stephen in CC because we talked about this
> issue
> > > in ASF CON and Twitter.
> > >
> > > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> > rmannibucau@gmail.com>
> > > wrote:
> > >
> > >> Hi Tibor,
> > >>
> > >> It has two issues:
> > >>
> > >> 1. It will not be the right plugin versions in 90% of the cases
> (except
> > >> demo ;))
> > >> 2. It will miss all custom plugins
> > >>
> > >> Now question is: what happens if you mount your local repo when
> running
> > >> docker? It works as expected. Means we could use a custom entrypoint
> > >> printing a warning banner if it is not done probably instead of
> > incrasing
> > >> the image size without being sure to reach the original goal.
> > >>
> > >> Wdyt?
> > >>
> > >> Romain
> > >>
> > >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
> > >> écrit :
> > >>
> > >> > If you use Docker images with Maven with no mapping of cache to the
> > >> > volumes, you may notice that Maven downloads the plugins for the
> build
> > >> > lifecycle.
> > >> >
> > >> > This slows down the build because a lot of artifacts and plugins are
> > >> > initially downloaded.
> > >> > This takes 50 seconds which might be even longer than the productive
> > >> build
> > >> > itself (compiler, package, ...).
> > >> >
> > >> > We discussed this topic with Herve and Karl at the Apache CON 2019
> the
> > >> last
> > >> > time.
> > >> >
> > >> > Sometime the presentations were funny because the audience had to
> > wait a
> > >> > minute while the console was black where the Maven was downloading
> the
> > >> > plugins in the background.
> > >> > Nobody was sure what happened that time, whether the console hanged
> or
> > >> the
> > >> > Cloud server hanged, or another issue happened with the network.
> > >> >
> > >> > I made a test and triggered the default lifecycle on Maven and I
> > >> realized
> > >> > that the cache was really very little, cca 12 MB.
> > >> > So this little cache in the container would save 50 seconds which is
> > the
> > >> > improvement we are discussing.
> > >> >
> > >> > From the use point of view, the user would use a new base image `
> > >> > 3.6.2-jdk-14-prefetched` which is my idea.
> > >> >
> > >> > There are multiple technical solutions (range of plugins, extra pom,
> > >> > internal Maven plugin versions, etc).
> > >> >
> > >> > We understood that the best idea would be to have the image with the
> > >> cache
> > >> > in new Docker images produced by Carloss Sanchez.
> > >> >
> > >> > We are discussing this topic in [1] but we do not have a consensus
> on
> > >> who
> > >> > will develop the Docker scripts and how.
> > >> >
> > >> > We can continue here and we can propose a solution.
> > >> >
> > >> > [1] https://github.com/carlossg/docker-maven/issues/130
> > >> >
> > >> >
> > >> > Cheers
> > >> > Tibor17
> > >> >
> > >>
> > > --
> > Sent from my phone
> >
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Tibor Digana <ti...@apache.org>.
Stephen, yeah something like you do in your scrip but it must not be a
personal owner. Even Carloss is person who makes this deployment to
DockerHub but his images are used by the entire world and we should decide
whether we would agree with him to have such images under his
responsibility or our responsibility as the Apache group. Then the script
would be official and we can cut the release of Maven and the Docker image
will be ready for the downloads together with the Maven distribution. So
the users will always know that it is consistent deployment and they
wouldn't expect that the image is missing for the new version.
So here we should decide on who will be the deployer of these images with
the cache. And the technical solution is smaller problem I would say.
On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org> wrote:
>
> > It's the situation when you have maven plugins in repo and it means that
> > all custom plugins/deps can be still downloaded as before.
> > Nothing exists like this in the world and we are talking about the
> > approaches.
> >
>
> Cough cough cough
>
>
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae418ca3fd63c63038e896255c6fe8/Dockerfile#L50
>
> There are issues, but it does shave a bit of time... though we end up
> adding our common dependencies into the seed pom so that it is an effective
> speed up
>
>
> >
> > I added Karl, Herve and Stephen in CC because we talked about this issue
> > in ASF CON and Twitter.
> >
> > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> rmannibucau@gmail.com>
> > wrote:
> >
> >> Hi Tibor,
> >>
> >> It has two issues:
> >>
> >> 1. It will not be the right plugin versions in 90% of the cases (except
> >> demo ;))
> >> 2. It will miss all custom plugins
> >>
> >> Now question is: what happens if you mount your local repo when running
> >> docker? It works as expected. Means we could use a custom entrypoint
> >> printing a warning banner if it is not done probably instead of
> incrasing
> >> the image size without being sure to reach the original goal.
> >>
> >> Wdyt?
> >>
> >> Romain
> >>
> >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
> >> écrit :
> >>
> >> > If you use Docker images with Maven with no mapping of cache to the
> >> > volumes, you may notice that Maven downloads the plugins for the build
> >> > lifecycle.
> >> >
> >> > This slows down the build because a lot of artifacts and plugins are
> >> > initially downloaded.
> >> > This takes 50 seconds which might be even longer than the productive
> >> build
> >> > itself (compiler, package, ...).
> >> >
> >> > We discussed this topic with Herve and Karl at the Apache CON 2019 the
> >> last
> >> > time.
> >> >
> >> > Sometime the presentations were funny because the audience had to
> wait a
> >> > minute while the console was black where the Maven was downloading the
> >> > plugins in the background.
> >> > Nobody was sure what happened that time, whether the console hanged or
> >> the
> >> > Cloud server hanged, or another issue happened with the network.
> >> >
> >> > I made a test and triggered the default lifecycle on Maven and I
> >> realized
> >> > that the cache was really very little, cca 12 MB.
> >> > So this little cache in the container would save 50 seconds which is
> the
> >> > improvement we are discussing.
> >> >
> >> > From the use point of view, the user would use a new base image `
> >> > 3.6.2-jdk-14-prefetched` which is my idea.
> >> >
> >> > There are multiple technical solutions (range of plugins, extra pom,
> >> > internal Maven plugin versions, etc).
> >> >
> >> > We understood that the best idea would be to have the image with the
> >> cache
> >> > in new Docker images produced by Carloss Sanchez.
> >> >
> >> > We are discussing this topic in [1] but we do not have a consensus on
> >> who
> >> > will develop the Docker scripts and how.
> >> >
> >> > We can continue here and we can propose a solution.
> >> >
> >> > [1] https://github.com/carlossg/docker-maven/issues/130
> >> >
> >> >
> >> > Cheers
> >> > Tibor17
> >> >
> >>
> > --
> Sent from my phone
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Tibor Digana <ti...@apache.org>.
I don't want your dependencies of course because it is user specific issue.
I only want to trigger "mvn clean install" or deploy and this will fetch
the plugins, not the dependencies from a dummy empty pom. (don't mean
user's pom). Finally, the user will have plugins in the image which come
from the definition of the bindings for particular Maven dist. In 3.7.0 we
will have new plugins in the bindings and so the dummy POM will not
necessarily specify any. That's my point. I think it is very simple but the
problem is if our group would deploy Docker images with default plugins in
the cache or Carloss will do it.
On Wed, Oct 30, 2019 at 10:28 AM Stephen Connolly <
stephen.alan.connolly@gmail.com> wrote:
> On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org> wrote:
>
> > It's the situation when you have maven plugins in repo and it means that
> > all custom plugins/deps can be still downloaded as before.
> > Nothing exists like this in the world and we are talking about the
> > approaches.
> >
>
> Cough cough cough
>
>
> https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae418ca3fd63c63038e896255c6fe8/Dockerfile#L50
>
> There are issues, but it does shave a bit of time... though we end up
> adding our common dependencies into the seed pom so that it is an effective
> speed up
>
>
> >
> > I added Karl, Herve and Stephen in CC because we talked about this issue
> > in ASF CON and Twitter.
> >
> > On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <
> rmannibucau@gmail.com>
> > wrote:
> >
> >> Hi Tibor,
> >>
> >> It has two issues:
> >>
> >> 1. It will not be the right plugin versions in 90% of the cases (except
> >> demo ;))
> >> 2. It will miss all custom plugins
> >>
> >> Now question is: what happens if you mount your local repo when running
> >> docker? It works as expected. Means we could use a custom entrypoint
> >> printing a warning banner if it is not done probably instead of
> incrasing
> >> the image size without being sure to reach the original goal.
> >>
> >> Wdyt?
> >>
> >> Romain
> >>
> >> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
> >> écrit :
> >>
> >> > If you use Docker images with Maven with no mapping of cache to the
> >> > volumes, you may notice that Maven downloads the plugins for the build
> >> > lifecycle.
> >> >
> >> > This slows down the build because a lot of artifacts and plugins are
> >> > initially downloaded.
> >> > This takes 50 seconds which might be even longer than the productive
> >> build
> >> > itself (compiler, package, ...).
> >> >
> >> > We discussed this topic with Herve and Karl at the Apache CON 2019 the
> >> last
> >> > time.
> >> >
> >> > Sometime the presentations were funny because the audience had to
> wait a
> >> > minute while the console was black where the Maven was downloading the
> >> > plugins in the background.
> >> > Nobody was sure what happened that time, whether the console hanged or
> >> the
> >> > Cloud server hanged, or another issue happened with the network.
> >> >
> >> > I made a test and triggered the default lifecycle on Maven and I
> >> realized
> >> > that the cache was really very little, cca 12 MB.
> >> > So this little cache in the container would save 50 seconds which is
> the
> >> > improvement we are discussing.
> >> >
> >> > From the use point of view, the user would use a new base image `
> >> > 3.6.2-jdk-14-prefetched` which is my idea.
> >> >
> >> > There are multiple technical solutions (range of plugins, extra pom,
> >> > internal Maven plugin versions, etc).
> >> >
> >> > We understood that the best idea would be to have the image with the
> >> cache
> >> > in new Docker images produced by Carloss Sanchez.
> >> >
> >> > We are discussing this topic in [1] but we do not have a consensus on
> >> who
> >> > will develop the Docker scripts and how.
> >> >
> >> > We can continue here and we can propose a solution.
> >> >
> >> > [1] https://github.com/carlossg/docker-maven/issues/130
> >> >
> >> >
> >> > Cheers
> >> > Tibor17
> >> >
> >>
> > --
> Sent from my phone
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Stephen Connolly <st...@gmail.com>.
On Wed 30 Oct 2019 at 08:21, Tibor Digana <ti...@apache.org> wrote:
> It's the situation when you have maven plugins in repo and it means that
> all custom plugins/deps can be still downloaded as before.
> Nothing exists like this in the world and we are talking about the
> approaches.
>
Cough cough cough
https://github.com/stephenc/docker-git-java8-maven-vim/blob/168b9968deae418ca3fd63c63038e896255c6fe8/Dockerfile#L50
There are issues, but it does shave a bit of time... though we end up
adding our common dependencies into the seed pom so that it is an effective
speed up
>
> I added Karl, Herve and Stephen in CC because we talked about this issue
> in ASF CON and Twitter.
>
> On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <rm...@gmail.com>
> wrote:
>
>> Hi Tibor,
>>
>> It has two issues:
>>
>> 1. It will not be the right plugin versions in 90% of the cases (except
>> demo ;))
>> 2. It will miss all custom plugins
>>
>> Now question is: what happens if you mount your local repo when running
>> docker? It works as expected. Means we could use a custom entrypoint
>> printing a warning banner if it is not done probably instead of incrasing
>> the image size without being sure to reach the original goal.
>>
>> Wdyt?
>>
>> Romain
>>
>> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
>> écrit :
>>
>> > If you use Docker images with Maven with no mapping of cache to the
>> > volumes, you may notice that Maven downloads the plugins for the build
>> > lifecycle.
>> >
>> > This slows down the build because a lot of artifacts and plugins are
>> > initially downloaded.
>> > This takes 50 seconds which might be even longer than the productive
>> build
>> > itself (compiler, package, ...).
>> >
>> > We discussed this topic with Herve and Karl at the Apache CON 2019 the
>> last
>> > time.
>> >
>> > Sometime the presentations were funny because the audience had to wait a
>> > minute while the console was black where the Maven was downloading the
>> > plugins in the background.
>> > Nobody was sure what happened that time, whether the console hanged or
>> the
>> > Cloud server hanged, or another issue happened with the network.
>> >
>> > I made a test and triggered the default lifecycle on Maven and I
>> realized
>> > that the cache was really very little, cca 12 MB.
>> > So this little cache in the container would save 50 seconds which is the
>> > improvement we are discussing.
>> >
>> > From the use point of view, the user would use a new base image `
>> > 3.6.2-jdk-14-prefetched` which is my idea.
>> >
>> > There are multiple technical solutions (range of plugins, extra pom,
>> > internal Maven plugin versions, etc).
>> >
>> > We understood that the best idea would be to have the image with the
>> cache
>> > in new Docker images produced by Carloss Sanchez.
>> >
>> > We are discussing this topic in [1] but we do not have a consensus on
>> who
>> > will develop the Docker scripts and how.
>> >
>> > We can continue here and we can propose a solution.
>> >
>> > [1] https://github.com/carlossg/docker-maven/issues/130
>> >
>> >
>> > Cheers
>> > Tibor17
>> >
>>
> --
Sent from my phone
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Tibor Digana <ti...@apache.org>.
It's the situation when you have maven plugins in repo and it means that
all custom plugins/deps can be still downloaded as before.
Nothing exists like this in the world and we are talking about the
approaches.
I added Karl, Herve and Stephen in CC because we talked about this issue in
ASF CON and Twitter.
On Wed, Oct 30, 2019 at 6:36 AM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi Tibor,
>
> It has two issues:
>
> 1. It will not be the right plugin versions in 90% of the cases (except
> demo ;))
> 2. It will miss all custom plugins
>
> Now question is: what happens if you mount your local repo when running
> docker? It works as expected. Means we could use a custom entrypoint
> printing a warning banner if it is not done probably instead of incrasing
> the image size without being sure to reach the original goal.
>
> Wdyt?
>
> Romain
>
> Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
> écrit :
>
> > If you use Docker images with Maven with no mapping of cache to the
> > volumes, you may notice that Maven downloads the plugins for the build
> > lifecycle.
> >
> > This slows down the build because a lot of artifacts and plugins are
> > initially downloaded.
> > This takes 50 seconds which might be even longer than the productive
> build
> > itself (compiler, package, ...).
> >
> > We discussed this topic with Herve and Karl at the Apache CON 2019 the
> last
> > time.
> >
> > Sometime the presentations were funny because the audience had to wait a
> > minute while the console was black where the Maven was downloading the
> > plugins in the background.
> > Nobody was sure what happened that time, whether the console hanged or
> the
> > Cloud server hanged, or another issue happened with the network.
> >
> > I made a test and triggered the default lifecycle on Maven and I realized
> > that the cache was really very little, cca 12 MB.
> > So this little cache in the container would save 50 seconds which is the
> > improvement we are discussing.
> >
> > From the use point of view, the user would use a new base image `
> > 3.6.2-jdk-14-prefetched` which is my idea.
> >
> > There are multiple technical solutions (range of plugins, extra pom,
> > internal Maven plugin versions, etc).
> >
> > We understood that the best idea would be to have the image with the
> cache
> > in new Docker images produced by Carloss Sanchez.
> >
> > We are discussing this topic in [1] but we do not have a consensus on who
> > will develop the Docker scripts and how.
> >
> > We can continue here and we can propose a solution.
> >
> > [1] https://github.com/carlossg/docker-maven/issues/130
> >
> >
> > Cheers
> > Tibor17
> >
>
Re: Docker image with initialized local cache saves 50 seconds in startup
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi Tibor,
It has two issues:
1. It will not be the right plugin versions in 90% of the cases (except
demo ;))
2. It will miss all custom plugins
Now question is: what happens if you mount your local repo when running
docker? It works as expected. Means we could use a custom entrypoint
printing a warning banner if it is not done probably instead of incrasing
the image size without being sure to reach the original goal.
Wdyt?
Romain
Le mer. 30 oct. 2019 à 02:03, Tibor Digana <ti...@apache.org> a
écrit :
> If you use Docker images with Maven with no mapping of cache to the
> volumes, you may notice that Maven downloads the plugins for the build
> lifecycle.
>
> This slows down the build because a lot of artifacts and plugins are
> initially downloaded.
> This takes 50 seconds which might be even longer than the productive build
> itself (compiler, package, ...).
>
> We discussed this topic with Herve and Karl at the Apache CON 2019 the last
> time.
>
> Sometime the presentations were funny because the audience had to wait a
> minute while the console was black where the Maven was downloading the
> plugins in the background.
> Nobody was sure what happened that time, whether the console hanged or the
> Cloud server hanged, or another issue happened with the network.
>
> I made a test and triggered the default lifecycle on Maven and I realized
> that the cache was really very little, cca 12 MB.
> So this little cache in the container would save 50 seconds which is the
> improvement we are discussing.
>
> From the use point of view, the user would use a new base image `
> 3.6.2-jdk-14-prefetched` which is my idea.
>
> There are multiple technical solutions (range of plugins, extra pom,
> internal Maven plugin versions, etc).
>
> We understood that the best idea would be to have the image with the cache
> in new Docker images produced by Carloss Sanchez.
>
> We are discussing this topic in [1] but we do not have a consensus on who
> will develop the Docker scripts and how.
>
> We can continue here and we can propose a solution.
>
> [1] https://github.com/carlossg/docker-maven/issues/130
>
>
> Cheers
> Tibor17
>