You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Kevin Burton <bu...@spinn3r.com> on 2015/02/24 19:05:34 UTC
Any work to use maven with docker or linux containers?
The ‘singleton’ problem with maven tests around port numbers, file names,
is a big problem that’s bitten me over the years.
I’d love if Maven could fork tests or with parallelism, run tests in a
container.
Right now I have two modules running tests and they are conflicting on
ports.
If they were run in containers the ports wouldn’t conflict.
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Any work to use maven with docker or linux containers?
Posted by Kevin Burton <bu...@spinn3r.com>.
And for us this might be a week project to resolve. We’re using embedded
zookeeper, activemq, cassandra, etc in a number of our unit tests and the
embedding frameworks don’t always have a way to change the port number.
On Tue, Feb 24, 2015 at 10:12 AM, Kevin Burton <bu...@spinn3r.com> wrote:
> Yes. But while this solves this problem for this one situation,
> singletons pop up everywhere. It might not be MY library but it might be
> something else.
>
> File names, database table names, port numbers. I’m sure I’m missing
> something.
>
> With containers, all of this stuff is solved and I don’t need to code
> around these issues.
>
> On Tue, Feb 24, 2015 at 10:08 AM, Aldrin Leal <al...@leal.eng.br> wrote:
>
>> If you set the port number to zero, you'd get a random one. Try looking at
>> some tests out there to find out how
>>
>> for reference:
>>
>> http://stackoverflow.com/questions/2231467/dynamically-choosing-port-number
>>
>>
>> --
>> -- Aldrin Leal, <al...@leal.eng.br>
>> Master your EC2-fu! Get the latest ekaterminal public beta
>> http://www.ingenieux.com.br/products/ekaterminal/
>>
>> On Tue, Feb 24, 2015 at 3:05 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>>
>> > The ‘singleton’ problem with maven tests around port numbers, file
>> names,
>> > is a big problem that’s bitten me over the years.
>> >
>> > I’d love if Maven could fork tests or with parallelism, run tests in a
>> > container.
>> >
>> > Right now I have two modules running tests and they are conflicting on
>> > ports.
>> >
>> > If they were run in containers the ports wouldn’t conflict.
>> >
>> > --
>> >
>> > Founder/CEO Spinn3r.com
>> > Location: *San Francisco, CA*
>> > blog: http://burtonator.wordpress.com
>> > … or check out my Google+ profile
>> > <https://plus.google.com/102718274791889610666/posts>
>> > <http://spinn3r.com>
>> >
>>
>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Any work to use maven with docker or linux containers?
Posted by Kevin Burton <bu...@spinn3r.com>.
And also, you’re right about surefire… each individual test (or chunks of
say 5-10 at a time) could be run on containers too.
On Tue, Feb 24, 2015 at 10:31 AM, Kevin Burton <bu...@spinn3r.com> wrote:
> On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal <al...@leal.eng.br> wrote:
>
>> Not quite - but I've been using Drone <http://www.drone.io/> (which is
>> wrapped around Docker) with Maven with little to no problems, although
>> building up a baseline takes a little practice.
>>
>>
> Meaning you run maven WITHIN a container?
>
> I’m using Codeship.. they have this new ParallelCI which works (in theory)
> with multiple modules but I’d need to first decompose and present the
> dependencies to codeship so it could do the work. But I think maven could
> probably provide that with about a day of work.
>
>
>> Perhaps a better question would be, say, how to make Surefire/Failsafe
>> fork
>> not across a machine, but instead across a fleet of Docker containers.
>> Viable? Yes, of course.
>>
>>
> YES. This is what I meant. But I actually think it’s Maven because what I
> want/need to start with is for the -T parallel option to , instead of
> forking, create a new container.
>
>
>> That is a question better addressed by Yarn and Mesosphere (Kubernetes
>> perhaps?)
>>
>
> Yes. Ideally each container would be on a separate piece of hardware so
> you could use 5-10 boxes to complete the tests quickly.
>
> Kevin
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Any work to use maven with docker or linux containers?
Posted by Kevin Burton <bu...@spinn3r.com>.
I agree.. my problem is I have a backlog of about 1000 weekend hacks :-P
On Tue, Feb 24, 2015 at 10:40 AM, Aldrin Leal <al...@leal.eng.br> wrote:
> that requires resource coordination. Not a huge problem at all, given the
> right tools. I'd say it is worthy a weekend hack
>
> --
> -- Aldrin Leal, <al...@leal.eng.br>
> Master your EC2-fu! Get the latest ekaterminal public beta
> http://www.ingenieux.com.br/products/ekaterminal/
>
> On Tue, Feb 24, 2015 at 3:31 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
> > On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal <al...@leal.eng.br>
> wrote:
> >
> > > Not quite - but I've been using Drone <http://www.drone.io/> (which is
> > > wrapped around Docker) with Maven with little to no problems, although
> > > building up a baseline takes a little practice.
> > >
> > >
> > Meaning you run maven WITHIN a container?
> >
> > I’m using Codeship.. they have this new ParallelCI which works (in
> theory)
> > with multiple modules but I’d need to first decompose and present the
> > dependencies to codeship so it could do the work. But I think maven
> could
> > probably provide that with about a day of work.
> >
> >
> > > Perhaps a better question would be, say, how to make Surefire/Failsafe
> > fork
> > > not across a machine, but instead across a fleet of Docker containers.
> > > Viable? Yes, of course.
> > >
> > >
> > YES. This is what I meant. But I actually think it’s Maven because what I
> > want/need to start with is for the -T parallel option to , instead of
> > forking, create a new container.
> >
> >
> > > That is a question better addressed by Yarn and Mesosphere (Kubernetes
> > > perhaps?)
> > >
> >
> > Yes. Ideally each container would be on a separate piece of hardware so
> > you could use 5-10 boxes to complete the tests quickly.
> >
> > Kevin
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
> > <http://spinn3r.com>
> >
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Any work to use maven with docker or linux containers?
Posted by Aldrin Leal <al...@leal.eng.br>.
that requires resource coordination. Not a huge problem at all, given the
right tools. I'd say it is worthy a weekend hack
--
-- Aldrin Leal, <al...@leal.eng.br>
Master your EC2-fu! Get the latest ekaterminal public beta
http://www.ingenieux.com.br/products/ekaterminal/
On Tue, Feb 24, 2015 at 3:31 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal <al...@leal.eng.br> wrote:
>
> > Not quite - but I've been using Drone <http://www.drone.io/> (which is
> > wrapped around Docker) with Maven with little to no problems, although
> > building up a baseline takes a little practice.
> >
> >
> Meaning you run maven WITHIN a container?
>
> I’m using Codeship.. they have this new ParallelCI which works (in theory)
> with multiple modules but I’d need to first decompose and present the
> dependencies to codeship so it could do the work. But I think maven could
> probably provide that with about a day of work.
>
>
> > Perhaps a better question would be, say, how to make Surefire/Failsafe
> fork
> > not across a machine, but instead across a fleet of Docker containers.
> > Viable? Yes, of course.
> >
> >
> YES. This is what I meant. But I actually think it’s Maven because what I
> want/need to start with is for the -T parallel option to , instead of
> forking, create a new container.
>
>
> > That is a question better addressed by Yarn and Mesosphere (Kubernetes
> > perhaps?)
> >
>
> Yes. Ideally each container would be on a separate piece of hardware so
> you could use 5-10 boxes to complete the tests quickly.
>
> Kevin
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
Re: Any work to use maven with docker or linux containers?
Posted by Kevin Burton <bu...@spinn3r.com>.
On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal <al...@leal.eng.br> wrote:
> Not quite - but I've been using Drone <http://www.drone.io/> (which is
> wrapped around Docker) with Maven with little to no problems, although
> building up a baseline takes a little practice.
>
>
Meaning you run maven WITHIN a container?
I’m using Codeship.. they have this new ParallelCI which works (in theory)
with multiple modules but I’d need to first decompose and present the
dependencies to codeship so it could do the work. But I think maven could
probably provide that with about a day of work.
> Perhaps a better question would be, say, how to make Surefire/Failsafe fork
> not across a machine, but instead across a fleet of Docker containers.
> Viable? Yes, of course.
>
>
YES. This is what I meant. But I actually think it’s Maven because what I
want/need to start with is for the -T parallel option to , instead of
forking, create a new container.
> That is a question better addressed by Yarn and Mesosphere (Kubernetes
> perhaps?)
>
Yes. Ideally each container would be on a separate piece of hardware so
you could use 5-10 boxes to complete the tests quickly.
Kevin
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Any work to use maven with docker or linux containers?
Posted by Aldrin Leal <al...@leal.eng.br>.
Not quite - but I've been using Drone <http://www.drone.io/> (which is
wrapped around Docker) with Maven with little to no problems, although
building up a baseline takes a little practice.
Perhaps a better question would be, say, how to make Surefire/Failsafe fork
not across a machine, but instead across a fleet of Docker containers.
Viable? Yes, of course.
That is a question better addressed by Yarn and Mesosphere (Kubernetes
perhaps?)
--
-- Aldrin Leal, <al...@leal.eng.br>
Master your EC2-fu! Get the latest ekaterminal public beta
http://www.ingenieux.com.br/products/ekaterminal/
On Tue, Feb 24, 2015 at 3:12 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> Yes. But while this solves this problem for this one situation, singletons
> pop up everywhere. It might not be MY library but it might be something
> else.
>
> File names, database table names, port numbers. I’m sure I’m missing
> something.
>
> With containers, all of this stuff is solved and I don’t need to code
> around these issues.
>
> On Tue, Feb 24, 2015 at 10:08 AM, Aldrin Leal <al...@leal.eng.br> wrote:
>
> > If you set the port number to zero, you'd get a random one. Try looking
> at
> > some tests out there to find out how
> >
> > for reference:
> >
> http://stackoverflow.com/questions/2231467/dynamically-choosing-port-number
> >
> >
> > --
> > -- Aldrin Leal, <al...@leal.eng.br>
> > Master your EC2-fu! Get the latest ekaterminal public beta
> > http://www.ingenieux.com.br/products/ekaterminal/
> >
> > On Tue, Feb 24, 2015 at 3:05 PM, Kevin Burton <bu...@spinn3r.com>
> wrote:
> >
> > > The ‘singleton’ problem with maven tests around port numbers, file
> names,
> > > is a big problem that’s bitten me over the years.
> > >
> > > I’d love if Maven could fork tests or with parallelism, run tests in a
> > > container.
> > >
> > > Right now I have two modules running tests and they are conflicting on
> > > ports.
> > >
> > > If they were run in containers the ports wouldn’t conflict.
> > >
> > > --
> > >
> > > Founder/CEO Spinn3r.com
> > > Location: *San Francisco, CA*
> > > blog: http://burtonator.wordpress.com
> > > … or check out my Google+ profile
> > > <https://plus.google.com/102718274791889610666/posts>
> > > <http://spinn3r.com>
> > >
> >
>
>
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>
Re: Any work to use maven with docker or linux containers?
Posted by Kevin Burton <bu...@spinn3r.com>.
Yes. But while this solves this problem for this one situation, singletons
pop up everywhere. It might not be MY library but it might be something
else.
File names, database table names, port numbers. I’m sure I’m missing
something.
With containers, all of this stuff is solved and I don’t need to code
around these issues.
On Tue, Feb 24, 2015 at 10:08 AM, Aldrin Leal <al...@leal.eng.br> wrote:
> If you set the port number to zero, you'd get a random one. Try looking at
> some tests out there to find out how
>
> for reference:
> http://stackoverflow.com/questions/2231467/dynamically-choosing-port-number
>
>
> --
> -- Aldrin Leal, <al...@leal.eng.br>
> Master your EC2-fu! Get the latest ekaterminal public beta
> http://www.ingenieux.com.br/products/ekaterminal/
>
> On Tue, Feb 24, 2015 at 3:05 PM, Kevin Burton <bu...@spinn3r.com> wrote:
>
> > The ‘singleton’ problem with maven tests around port numbers, file names,
> > is a big problem that’s bitten me over the years.
> >
> > I’d love if Maven could fork tests or with parallelism, run tests in a
> > container.
> >
> > Right now I have two modules running tests and they are conflicting on
> > ports.
> >
> > If they were run in containers the ports wouldn’t conflict.
> >
> > --
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
> > <http://spinn3r.com>
> >
>
--
Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>
Re: Any work to use maven with docker or linux containers?
Posted by Aldrin Leal <al...@leal.eng.br>.
If you set the port number to zero, you'd get a random one. Try looking at
some tests out there to find out how
for reference:
http://stackoverflow.com/questions/2231467/dynamically-choosing-port-number
--
-- Aldrin Leal, <al...@leal.eng.br>
Master your EC2-fu! Get the latest ekaterminal public beta
http://www.ingenieux.com.br/products/ekaterminal/
On Tue, Feb 24, 2015 at 3:05 PM, Kevin Burton <bu...@spinn3r.com> wrote:
> The ‘singleton’ problem with maven tests around port numbers, file names,
> is a big problem that’s bitten me over the years.
>
> I’d love if Maven could fork tests or with parallelism, run tests in a
> container.
>
> Right now I have two modules running tests and they are conflicting on
> ports.
>
> If they were run in containers the ports wouldn’t conflict.
>
> --
>
> Founder/CEO Spinn3r.com
> Location: *San Francisco, CA*
> blog: http://burtonator.wordpress.com
> … or check out my Google+ profile
> <https://plus.google.com/102718274791889610666/posts>
> <http://spinn3r.com>
>