You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Ali Ahmed <al...@gmail.com> on 2018/06/14 18:21:13 UTC
Test library migration proposal
I am proposing we move our current integration test harness from using the
current arquillian framework to testcontainers ```
https://www.testcontainers.org/```.
The benefits are improved stability and slightly faster execution but more
importably arquillian has some issues with developing reusable test classes
and objects so we end creating multiple test projects each with there own
arquillian xml. This is mostly due to lifecycle management of container
objects during test execution.
These problems are avoid in testcontainers, and therefore we can build a
more robust test framework and increase coverage.
-Ali
Re: Test library migration proposal
Posted by Ali Ahmed <al...@gmail.com>.
Agreed will prepare a sample test.
-Ali
On Mon, Jun 18, 2018 at 6:03 AM, Ivan Kelly <iv...@apache.org> wrote:
> > We do have access to the docker client via test containers.
> >
> > There is no binding between test containers and any test framework all
> the
> > underlying methods and classes and fully exposed so they can be mapped to
> > any test harness.
>
> In that case, the first thing to do is try to migrate the smoke test
> to see how the approaches compare.
>
> -Ivan
>
Re: Test library migration proposal
Posted by Ivan Kelly <iv...@apache.org>.
> We do have access to the docker client via test containers.
>
> There is no binding between test containers and any test framework all the
> underlying methods and classes and fully exposed so they can be mapped to
> any test harness.
In that case, the first thing to do is try to migrate the smoke test
to see how the approaches compare.
-Ivan
Re: Test library migration proposal
Posted by Ali Ahmed <al...@gmail.com>.
Hi Ivan ,
We do have access to the docker client via test containers.
There is no binding between test containers and any test framework all the
underlying methods and classes and fully exposed so they can be mapped to
any test harness.
-Ali
On Fri, Jun 15, 2018 at 11:37 AM, Jerry Peng <je...@gmail.com>
wrote:
> Sounds good!
>
> On Fri, Jun 15, 2018 at 1:53 AM Ivan Kelly <iv...@apache.org> wrote:
>
> > Thanks for starting this conversation Ali. The current tests use
> > arquillian, but there's nothing in the tests that's intimately tied to
> > it, so if we were to remove it there wouldn't be that much that needs
> > to change.
> >
> > On Thu, Jun 14, 2018 at 8:21 PM, Ali Ahmed <al...@gmail.com> wrote:
> > > I am proposing we move our current integration test harness from using
> > the
> > > current arquillian framework to testcontainers ```
> > > https://www.testcontainers.org/``` <https://www.testcontainers.org/>.
> >
> > Test containers looks interesting.
> >
> > A couple of questions around testcontainers.
> > - Does it support testng? The website only mentions junit, and all
> > pulsar testing is testng.
> > - Does it give you access to the docker-java DockerClient? Lots of the
> > current utilities use that.
> >
> > Before committing to a migration, it would be good to validate that
> > everything we currently do with arquillian can be done with test
> > containers by migrating one of the existing tests (smoke for example).
> >
> > > The benefits are improved stability
> >
> > ?? I've not seen any stability issues with arquillian which were
> > caused by the framework itself.
> >
> > > and slightly faster execution but more
> >
> > Do you have any numbers for this? The slowness i've seen in the tests
> > is mostly down container creation, and I don't think that will go away
> > if we change frameworks.
> >
> > > importably arquillian has some issues with developing reusable test
> > classes
> > > and objects so we end creating multiple test projects each with there
> own
> > > arquillian xml.
> >
> > It would be nice to get rid of this limitation.
> >
> > -Ivan
> >
>
Re: Test library migration proposal
Posted by Jerry Peng <je...@gmail.com>.
Sounds good!
On Fri, Jun 15, 2018 at 1:53 AM Ivan Kelly <iv...@apache.org> wrote:
> Thanks for starting this conversation Ali. The current tests use
> arquillian, but there's nothing in the tests that's intimately tied to
> it, so if we were to remove it there wouldn't be that much that needs
> to change.
>
> On Thu, Jun 14, 2018 at 8:21 PM, Ali Ahmed <al...@gmail.com> wrote:
> > I am proposing we move our current integration test harness from using
> the
> > current arquillian framework to testcontainers ```
> > https://www.testcontainers.org/``` <https://www.testcontainers.org/>.
>
> Test containers looks interesting.
>
> A couple of questions around testcontainers.
> - Does it support testng? The website only mentions junit, and all
> pulsar testing is testng.
> - Does it give you access to the docker-java DockerClient? Lots of the
> current utilities use that.
>
> Before committing to a migration, it would be good to validate that
> everything we currently do with arquillian can be done with test
> containers by migrating one of the existing tests (smoke for example).
>
> > The benefits are improved stability
>
> ?? I've not seen any stability issues with arquillian which were
> caused by the framework itself.
>
> > and slightly faster execution but more
>
> Do you have any numbers for this? The slowness i've seen in the tests
> is mostly down container creation, and I don't think that will go away
> if we change frameworks.
>
> > importably arquillian has some issues with developing reusable test
> classes
> > and objects so we end creating multiple test projects each with there own
> > arquillian xml.
>
> It would be nice to get rid of this limitation.
>
> -Ivan
>
Re: Test library migration proposal
Posted by Ivan Kelly <iv...@apache.org>.
Thanks for starting this conversation Ali. The current tests use
arquillian, but there's nothing in the tests that's intimately tied to
it, so if we were to remove it there wouldn't be that much that needs
to change.
On Thu, Jun 14, 2018 at 8:21 PM, Ali Ahmed <al...@gmail.com> wrote:
> I am proposing we move our current integration test harness from using the
> current arquillian framework to testcontainers ```
> https://www.testcontainers.org/```.
Test containers looks interesting.
A couple of questions around testcontainers.
- Does it support testng? The website only mentions junit, and all
pulsar testing is testng.
- Does it give you access to the docker-java DockerClient? Lots of the
current utilities use that.
Before committing to a migration, it would be good to validate that
everything we currently do with arquillian can be done with test
containers by migrating one of the existing tests (smoke for example).
> The benefits are improved stability
?? I've not seen any stability issues with arquillian which were
caused by the framework itself.
> and slightly faster execution but more
Do you have any numbers for this? The slowness i've seen in the tests
is mostly down container creation, and I don't think that will go away
if we change frameworks.
> importably arquillian has some issues with developing reusable test classes
> and objects so we end creating multiple test projects each with there own
> arquillian xml.
It would be nice to get rid of this limitation.
-Ivan