You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@polygene.apache.org by Paul Merlin <pa...@apache.org> on 2017/02/25 14:33:36 UTC
Re: Docker based integration testing
Gang,
I finally found time to take a stab at fixing Docker based integration
testing on Linux.
It should now work out of the box on Linux, given that a Docker service
is running locally and that your user has the correct permissions.
Docker based integration testing is now disabled if unable to contact a
Docker service. No need to set DOCKER_HOST.
Niclas, could you confirm it works for you?
Cheers
/Paul
Paul Merlin a �crit :
> Thanks for your feedback Niclas!
>
> Using the unix:///path/to/docker.sock URL on Linux should do.
> I'll do another round on that later this week.
>
> Cheers
>
>
> Niclas Hedhman a �crit :
>> DOCKER_* may not be the best way to determine if Docker is present. I am on
>> a Debian system, and no env variable named DOCKER* is set by default, yet
>> docker is operational. IIUIC, these variables are needed for Docker Swarm
>> and other specialized setup where one is not running the container on the
>> same system as issuing the `docker` command.
>>
>> That said, running
>> ./gradlew :ext:entity-redis:test
>>
>> without changing my defaults gives me some errors;
>>
>> :extensions:entitystore-redis:test
>>
>> org.apache.polygene.entitystore.redis.RedisMapEntityStoreWithCacheTest >
>> classMethod FAILED
>> java.lang.ExceptionInInitializerError
>> Caused by: java.lang.IllegalStateException at
>> RedisMapEntityStoreWithCacheTest.java:38
>> Caused by: com.spotify.docker.client.exceptions.DockerException
>> at RedisMapEntityStoreWithCacheTest.java:38
>> Caused by: java.util.concurrent.ExecutionException at
>> RedisMapEntityStoreWithCacheTest.java:38
>> Caused by: javax.ws.rs.ProcessingException
>> Caused by:
>> org.apache.http.conn.HttpHostConnectException
>> Caused by: java.net.ConnectException
>>
>> org.apache.polygene.entitystore.redis.RedisMapEntityStoreTest > classMethod
>> FAILED
>> java.lang.ExceptionInInitializerError
>> Caused by: java.lang.IllegalStateException at
>> RedisMapEntityStoreTest.java:38
>> Caused by: com.spotify.docker.client.exceptions.DockerException
>> at RedisMapEntityStoreTest.java:38
>> Caused by: java.util.concurrent.ExecutionException at
>> RedisMapEntityStoreTest.java:38
>> Caused by: javax.ws.rs.ProcessingException
>> Caused by:
>> org.apache.http.conn.HttpHostConnectException
>> Caused by: java.net.ConnectException
>>
>> 2 tests completed, 2 failed
>>
>>
>> And the underlying exception is;
>>
>> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to
>> localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed:
>> Connection refused (Connection refused)
>>
>>
>> Which I presume is due to the gradle-docker-plugin going on the network
>> protocol instead of the command-line interface. The default docker
>> interface on Linux seems to be Unix sockets, which most Java apps tend to
>> ignore and not support.
>>
>> 1059 ? Ssl 0:02 docker-containerd -l
>> unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
>> docker-containerd-shim --metrics-interval=0 --start-timeout 2m --state-dir
>> /var/run/docker/libcontainerd/containerd --runtime docker-runc
>>
>>
>> Hope that helps. I am sending you login separately to a clean/unused server
>> in Germany, in case you want to run experiments on it.
>>
>> Cheers
>>
>> On Sun, Jan 15, 2017 at 8:08 PM, Paul Merlin <pa...@apache.org> wrote:
>>
>>> Could anybody confirm it works for them?
>>>
>>> Paul Merlin a �crit :
>>>> Gang,
>>>>
>>>> I pushed changes to the build system and tests to automate integration
>>>> testing.
>>>>
>>>> It uses docker to build the necessary images and run docker containers
>>>> as a JUnit rule.
>>>> All this is done only if the DOCKER_HOST environment variable is set,
>>>> otherwise it's skipped.
>>>>
>>>> On linux with docker installed it should work out of the box given you
>>>> have DOCKER_HOST and DOCKER_CERT_PATH set.
>>>>
>>>> On mac you need to:
>>>> - docker-machine create someFancyNameYouChoose
>>>> - eval $(docker-machine env someFancyNameYouChoose)
>>>>
>>>> On windows, see https://docs.docker.com/machine/reference/env/
>>>>
>>>> With that done, when you now run e.g:
>>>>
>>>> ./gradlew :ext:entity-redis:test
>>>>
>>>> it will build the docker images and run Redis in a container around
>>>> the EntityStore tests.
>>>>
>>>> The same goes for these extensions:
>>>>
>>>> - Memcached Cache
>>>> - SQL EntityStore using both MySQL and PostgreSQL
>>>> - SQL Index/Query using PostgreSQL
>>>> - Riak EntityStore
>>>>
>>>> All the others extensions are tested using embedded solutions,
>>>> including MongoDB.
>>>>
>>>> Docker images are built by the :internals:testsupport-internal
>>>> project, you'll find the Dockerfiles there.
>>>>
>>>> Thanks to this I was able to fix a bug I introduced in the Riak ES in
>>>> recent changes.
>>>> The only thing actually broken is SQL Index/Query, see
>>>> https://issues.apache.org/jira/browse/POLYGENE-222
>>>>
>>>> For now these test do not run on CI.
>>>> builds.apache.org supports docker so we should be able to do that too
>>>> at some point.
>>>> In the meantime some of us should run the *whole* test suite from time
>>>> to time.
>>>> At least now it should be damn easy to do!
>>>>
>>>> Before calling this automation done I'd like to get someone to confirm
>>>> it 'works for them'.
>>>> I only tested on a mac.
>>>>
>>>> Cheers
>>>>
>>>> /Paul
>>>>
>
Re: Docker based integration testing
Posted by Paul Merlin <pa...@apache.org>.
We currently have several failures related to docker testing on CI:
https://builds.apache.org/view/P/view/Polygene/job/Polygene(JavaEdition)-develop-java8-check/272/testReport/
All seems related to a docker/system/networking issue.
Locally, on a mac, only the Indexing-SQL tests fail as expected since it
is broken.
I'll add a skipDocker property to the build and use it on CI to silent
those failures on our Java 8 & 9 checks jobs. I'll leave the docker
tests enabled on our checkDistributions job that doesn't run on each git
push.
Niclas Hedhman a �crit :
> Yeah, I did
> ./gradlew clean build assemble install
>
> and got the Riak test failures locally
>
> On Sun, Feb 26, 2017 at 5:15 PM, Paul Merlin <pa...@nosphere.org> wrote:
>
>> Le 26 f�vrier 2017 05:37:00 GMT+01:00, Niclas Hedhman <ni...@hedhman.org>
>> a �crit :
>>> So only the Docker daemon is needed?
>>>
>>> And I run what target for ./gradlew ?
>>>
>>> Cheers
>>> Niclas
>>>
>>> On Sat, Feb 25, 2017 at 10:33 PM, Paul Merlin <pa...@apache.org> wrote:
>>>
>>> Gang,
>>>> I finally found time to take a stab at fixing Docker based integration
>>>> testing on Linux.
>>>>
>>>> It should now work out of the box on Linux, given that a Docker service
>>>> is running locally and that your user has the correct permissions.
>>>>
>>>> Docker based integration testing is now disabled if unable to contact a
>>>> Docker service. No need to set DOCKER_HOST.
>>>>
>>>> Niclas, could you confirm it works for you?
>>>>
>>>> Cheers
>>>>
>>>> /Paul
>>>>
>>>>
>>>>
>>>> Paul Merlin a �crit :
>>>>
>>>>> Thanks for your feedback Niclas!
>>>>>
>>>>> Using the unix:///path/to/docker.sock URL on Linux should do.
>>>>> I'll do another round on that later this week.
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>> Niclas Hedhman a �crit :
>>>>>
>>>>>> DOCKER_* may not be the best way to determine if Docker is present. I
>>>>>>
>>>>> am on
>>>>> a Debian system, and no env variable named DOCKER* is set by default,
>>>>> yet
>>>>> docker is operational. IIUIC, these variables are needed for Docker
>>>>> Swarm
>>>>> and other specialized setup where one is not running the container on
>>>>> the
>>>>> same system as issuing the `docker` command.
>>>>>> That said, running
>>>>>> ./gradlew :ext:entity-redis:test
>>>>>>
>>>>>> without changing my defaults gives me some errors;
>>>>>>
>>>>>> :extensions:entitystore-redis:test
>>>>>>
>>>>>> org.apache.polygene.entitystore.redis.RedisMapEntityStoreWithCacheTest
>>>>>>
>>>>> classMethod FAILED
>>>>>> java.lang.ExceptionInInitializerError
>>>>>> Caused by: java.lang.IllegalStateException at
>>>>>> RedisMapEntityStoreWithCacheTest.java:38
>>>>>> Caused by: com.spotify.docker.client.
>>>>>>
>>>>> exceptions.DockerException
>>>>> at RedisMapEntityStoreWithCacheTest.java:38
>>>>>> Caused by: java.util.concurrent.ExecutionException at
>>>>>> RedisMapEntityStoreWithCacheTest.java:38
>>>>>> Caused by: javax.ws.rs.ProcessingException
>>>>>> Caused by:
>>>>>> org.apache.http.conn.HttpHostConnectException
>>>>>> Caused by: java.net.ConnectException
>>>>>>
>>>>>> org.apache.polygene.entitystore.redis.RedisMapEntityStoreTest >
>>>>>>
>>>>> classMethod
>>>>> FAILED
>>>>>> java.lang.ExceptionInInitializerError
>>>>>> Caused by: java.lang.IllegalStateException at
>>>>>> RedisMapEntityStoreTest.java:38
>>>>>> Caused by: com.spotify.docker.client.
>>>>>>
>>>>> exceptions.DockerException
>>>>> at RedisMapEntityStoreTest.java:38
>>>>>> Caused by: java.util.concurrent.ExecutionException at
>>>>>> RedisMapEntityStoreTest.java:38
>>>>>> Caused by: javax.ws.rs.ProcessingException
>>>>>> Caused by:
>>>>>> org.apache.http.conn.HttpHostConnectException
>>>>>> Caused by: java.net.ConnectException
>>>>>>
>>>>>> 2 tests completed, 2 failed
>>>>>>
>>>>>>
>>>>>> And the underlying exception is;
>>>>>>
>>>>>> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to
>>>>>> localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed:
>>>>>> Connection refused (Connection refused)
>>>>>>
>>>>>>
>>>>>> Which I presume is due to the gradle-docker-plugin going on the network
>>>>>> protocol instead of the command-line interface. The default docker
>>>>>> interface on Linux seems to be Unix sockets, which most Java apps tend
>>>>>>
>>>>> to
>>>>> ignore and not support.
>>>>>> 1059 ? Ssl 0:02 docker-containerd -l
>>>>>> unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
>>>>>> docker-containerd-shim --metrics-interval=0 --start-timeout 2m
>>>>>>
>>>>> --state-dir
>>>>> /var/run/docker/libcontainerd/containerd --runtime docker-runc
>>>>>> Hope that helps. I am sending you login separately to a clean/unused
>>>>>>
>>>>> server
>>>>> in Germany, in case you want to run experiments on it.
>>>>>> Cheers
>>>>>>
>>>>>> On Sun, Jan 15, 2017 at 8:08 PM, Paul Merlin <pa...@apache.org>
>>>>>>
>>>>> wrote:
>>>>>> Could anybody confirm it works for them?
>>>>>>> Paul Merlin a �crit :
>>>>>>>
>>>>>>>> Gang,
>>>>>>>>
>>>>>>>> I pushed changes to the build system and tests to automate integration
>>>>>>>> testing.
>>>>>>>>
>>>>>>>> It uses docker to build the necessary images and run docker containers
>>>>>>>> as a JUnit rule.
>>>>>>>> All this is done only if the DOCKER_HOST environment variable is set,
>>>>>>>> otherwise it's skipped.
>>>>>>>>
>>>>>>>> On linux with docker installed it should work out of the box given you
>>>>>>>> have DOCKER_HOST and DOCKER_CERT_PATH set.
>>>>>>>>
>>>>>>>> On mac you need to:
>>>>>>>> - docker-machine create someFancyNameYouChoose
>>>>>>>> - eval $(docker-machine env someFancyNameYouChoose)
>>>>>>>>
>>>>>>>> On windows, see https://docs.docker.com/machine/reference/env/
>>>>>>>>
>>>>>>>> With that done, when you now run e.g:
>>>>>>>>
>>>>>>>> ./gradlew :ext:entity-redis:test
>>>>>>>>
>>>>>>>> it will build the docker images and run Redis in a container around
>>>>>>>> the EntityStore tests.
>>>>>>>>
>>>>>>>> The same goes for these extensions:
>>>>>>>>
>>>>>>>> - Memcached Cache
>>>>>>>> - SQL EntityStore using both MySQL and PostgreSQL
>>>>>>>> - SQL Index/Query using PostgreSQL
>>>>>>>> - Riak EntityStore
>>>>>>>>
>>>>>>>> All the others extensions are tested using embedded solutions,
>>>>>>>> including MongoDB.
>>>>>>>>
>>>>>>>> Docker images are built by the :internals:testsupport-internal
>>>>>>>> project, you'll find the Dockerfiles there.
>>>>>>>>
>>>>>>>> Thanks to this I was able to fix a bug I introduced in the Riak ES in
>>>>>>>> recent changes.
>>>>>>>> The only thing actually broken is SQL Index/Query, see
>>>>>>>> https://issues.apache.org/jira/browse/POLYGENE-222
>>>>>>>>
>>>>>>>> For now these test do not run on CI.
>>>>>>>> builds.apache.org supports docker so we should be able to do that too
>>>>>>>> at some point.
>>>>>>>> In the meantime some of us should run the *whole* test suite from time
>>>>>>>> to time.
>>>>>>>> At least now it should be damn easy to do!
>>>>>>>>
>>>>>>>> Before calling this automation done I'd like to get someone to confirm
>>>>>>>> it 'works for them'.
>>>>>>>> I only tested on a mac.
>>>>>>>>
>>>>>>>> Cheers
>>>>>>>>
>>>>>>>> /Paul
>> ./gradlew test
>> for the whole test suite
>>
>> --
>> Envoy� de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma
>> bri�vet�.
>>
>
>
>
Re: Docker based integration testing
Posted by Niclas Hedhman <ni...@hedhman.org>.
Yeah, I did
./gradlew clean build assemble install
and got the Riak test failures locally
On Sun, Feb 26, 2017 at 5:15 PM, Paul Merlin <pa...@nosphere.org> wrote:
> Le 26 février 2017 05:37:00 GMT+01:00, Niclas Hedhman <ni...@hedhman.org>
> a écrit :
>>
>> So only the Docker daemon is needed?
>>
>> And I run what target for ./gradlew ?
>>
>> Cheers
>> Niclas
>>
>> On Sat, Feb 25, 2017 at 10:33 PM, Paul Merlin <pa...@apache.org> wrote:
>>
>> Gang,
>>>
>>> I finally found time to take a stab at fixing Docker based integration
>>> testing on Linux.
>>>
>>> It should now work out of the box on Linux, given that a Docker service
>>> is running locally and that your user has the correct permissions.
>>>
>>> Docker based integration testing is now disabled if unable to contact a
>>> Docker service. No need to set DOCKER_HOST.
>>>
>>> Niclas, could you confirm it works for you?
>>>
>>> Cheers
>>>
>>> /Paul
>>>
>>>
>>>
>>> Paul Merlin a écrit :
>>>
>>>> Thanks for your feedback Niclas!
>>>>
>>>> Using the unix:///path/to/docker.sock URL on Linux should do.
>>>> I'll do another round on that later this week.
>>>>
>>>> Cheers
>>>>
>>>>
>>>> Niclas Hedhman a écrit :
>>>>
>>>>> DOCKER_* may not be the best way to determine if Docker is present. I
>>>>>
>>>> am on
>>>
>>>> a Debian system, and no env variable named DOCKER* is set by default,
>>>>>
>>>> yet
>>>
>>>> docker is operational. IIUIC, these variables are needed for Docker
>>>>>
>>>> Swarm
>>>
>>>> and other specialized setup where one is not running the container on
>>>>>
>>>> the
>>>
>>>> same system as issuing the `docker` command.
>>>>>
>>>>> That said, running
>>>>> ./gradlew :ext:entity-redis:test
>>>>>
>>>>> without changing my defaults gives me some errors;
>>>>>
>>>>> :extensions:entitystore-redis:test
>>>>>
>>>>> org.apache.polygene.entitystore.redis.RedisMapEntityStoreWithCacheTest
>>>>>
>>>>
>>>> classMethod FAILED
>>>>> java.lang.ExceptionInInitializerError
>>>>> Caused by: java.lang.IllegalStateException at
>>>>> RedisMapEntityStoreWithCacheTest.java:38
>>>>> Caused by: com.spotify.docker.client.
>>>>>
>>>> exceptions.DockerException
>>>
>>>> at RedisMapEntityStoreWithCacheTest.java:38
>>>>> Caused by: java.util.concurrent.ExecutionException at
>>>>> RedisMapEntityStoreWithCacheTest.java:38
>>>>> Caused by: javax.ws.rs.ProcessingException
>>>>> Caused by:
>>>>> org.apache.http.conn.HttpHostConnectException
>>>>> Caused by: java.net.ConnectException
>>>>>
>>>>> org.apache.polygene.entitystore.redis.RedisMapEntityStoreTest >
>>>>>
>>>> classMethod
>>>
>>>> FAILED
>>>>> java.lang.ExceptionInInitializerError
>>>>> Caused by: java.lang.IllegalStateException at
>>>>> RedisMapEntityStoreTest.java:38
>>>>> Caused by: com.spotify.docker.client.
>>>>>
>>>> exceptions.DockerException
>>>
>>>> at RedisMapEntityStoreTest.java:38
>>>>> Caused by: java.util.concurrent.ExecutionException at
>>>>> RedisMapEntityStoreTest.java:38
>>>>> Caused by: javax.ws.rs.ProcessingException
>>>>> Caused by:
>>>>> org.apache.http.conn.HttpHostConnectException
>>>>> Caused by: java.net.ConnectException
>>>>>
>>>>> 2 tests completed, 2 failed
>>>>>
>>>>>
>>>>> And the underlying exception is;
>>>>>
>>>>> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to
>>>>> localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed:
>>>>> Connection refused (Connection refused)
>>>>>
>>>>>
>>>>> Which I presume is due to the gradle-docker-plugin going on the network
>>>>> protocol instead of the command-line interface. The default docker
>>>>> interface on Linux seems to be Unix sockets, which most Java apps tend
>>>>>
>>>> to
>>>
>>>> ignore and not support.
>>>>>
>>>>> 1059 ? Ssl 0:02 docker-containerd -l
>>>>> unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
>>>>> docker-containerd-shim --metrics-interval=0 --start-timeout 2m
>>>>>
>>>> --state-dir
>>>
>>>> /var/run/docker/libcontainerd/containerd --runtime docker-runc
>>>>>
>>>>>
>>>>> Hope that helps. I am sending you login separately to a clean/unused
>>>>>
>>>> server
>>>
>>>> in Germany, in case you want to run experiments on it.
>>>>>
>>>>> Cheers
>>>>>
>>>>> On Sun, Jan 15, 2017 at 8:08 PM, Paul Merlin <pa...@apache.org>
>>>>>
>>>> wrote:
>>>
>>>>
>>>>> Could anybody confirm it works for them?
>>>>>>
>>>>>> Paul Merlin a écrit :
>>>>>>
>>>>>>> Gang,
>>>>>>>
>>>>>>> I pushed changes to the build system and tests to automate integration
>>>>>>> testing.
>>>>>>>
>>>>>>> It uses docker to build the necessary images and run docker containers
>>>>>>> as a JUnit rule.
>>>>>>> All this is done only if the DOCKER_HOST environment variable is set,
>>>>>>> otherwise it's skipped.
>>>>>>>
>>>>>>> On linux with docker installed it should work out of the box given you
>>>>>>> have DOCKER_HOST and DOCKER_CERT_PATH set.
>>>>>>>
>>>>>>> On mac you need to:
>>>>>>> - docker-machine create someFancyNameYouChoose
>>>>>>> - eval $(docker-machine env someFancyNameYouChoose)
>>>>>>>
>>>>>>> On windows, see https://docs.docker.com/machine/reference/env/
>>>>>>>
>>>>>>> With that done, when you now run e.g:
>>>>>>>
>>>>>>> ./gradlew :ext:entity-redis:test
>>>>>>>
>>>>>>> it will build the docker images and run Redis in a container around
>>>>>>> the EntityStore tests.
>>>>>>>
>>>>>>> The same goes for these extensions:
>>>>>>>
>>>>>>> - Memcached Cache
>>>>>>> - SQL EntityStore using both MySQL and PostgreSQL
>>>>>>> - SQL Index/Query using PostgreSQL
>>>>>>> - Riak EntityStore
>>>>>>>
>>>>>>> All the others extensions are tested using embedded solutions,
>>>>>>> including MongoDB.
>>>>>>>
>>>>>>> Docker images are built by the :internals:testsupport-internal
>>>>>>> project, you'll find the Dockerfiles there.
>>>>>>>
>>>>>>> Thanks to this I was able to fix a bug I introduced in the Riak ES in
>>>>>>> recent changes.
>>>>>>> The only thing actually broken is SQL Index/Query, see
>>>>>>> https://issues.apache.org/jira/browse/POLYGENE-222
>>>>>>>
>>>>>>> For now these test do not run on CI.
>>>>>>> builds.apache.org supports docker so we should be able to do that too
>>>>>>> at some point.
>>>>>>> In the meantime some of us should run the *whole* test suite from time
>>>>>>> to time.
>>>>>>> At least now it should be damn easy to do!
>>>>>>>
>>>>>>> Before calling this automation done I'd like to get someone to confirm
>>>>>>> it 'works for them'.
>>>>>>> I only tested on a mac.
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>> /Paul
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>>
> ./gradlew test
> for the whole test suite
>
> --
> Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma
> brièveté.
>
--
Niclas Hedhman, Software Developer
http://polygene.apache.org <http://zest.apache.org> - New Energy for Java
Re: Docker based integration testing
Posted by Paul Merlin <pa...@nosphere.org>.
Le 26 février 2017 05:37:00 GMT+01:00, Niclas Hedhman <ni...@hedhman.org> a écrit :
>So only the Docker daemon is needed?
>
>And I run what target for ./gradlew ?
>
>Cheers
>Niclas
>
>On Sat, Feb 25, 2017 at 10:33 PM, Paul Merlin <pa...@apache.org>
>wrote:
>
>> Gang,
>>
>> I finally found time to take a stab at fixing Docker based
>integration
>> testing on Linux.
>>
>> It should now work out of the box on Linux, given that a Docker
>service
>> is running locally and that your user has the correct permissions.
>>
>> Docker based integration testing is now disabled if unable to contact
>a
>> Docker service. No need to set DOCKER_HOST.
>>
>> Niclas, could you confirm it works for you?
>>
>> Cheers
>>
>> /Paul
>>
>>
>>
>> Paul Merlin a écrit :
>> > Thanks for your feedback Niclas!
>> >
>> > Using the unix:///path/to/docker.sock URL on Linux should do.
>> > I'll do another round on that later this week.
>> >
>> > Cheers
>> >
>> >
>> > Niclas Hedhman a écrit :
>> >> DOCKER_* may not be the best way to determine if Docker is
>present. I
>> am on
>> >> a Debian system, and no env variable named DOCKER* is set by
>default,
>> yet
>> >> docker is operational. IIUIC, these variables are needed for
>Docker
>> Swarm
>> >> and other specialized setup where one is not running the container
>on
>> the
>> >> same system as issuing the `docker` command.
>> >>
>> >> That said, running
>> >> ./gradlew :ext:entity-redis:test
>> >>
>> >> without changing my defaults gives me some errors;
>> >>
>> >> :extensions:entitystore-redis:test
>> >>
>> >>
>org.apache.polygene.entitystore.redis.RedisMapEntityStoreWithCacheTest
>> >
>> >> classMethod FAILED
>> >> java.lang.ExceptionInInitializerError
>> >> Caused by: java.lang.IllegalStateException at
>> >> RedisMapEntityStoreWithCacheTest.java:38
>> >> Caused by: com.spotify.docker.client.
>> exceptions.DockerException
>> >> at RedisMapEntityStoreWithCacheTest.java:38
>> >> Caused by: java.util.concurrent.ExecutionException
>at
>> >> RedisMapEntityStoreWithCacheTest.java:38
>> >> Caused by: javax.ws.rs.ProcessingException
>> >> Caused by:
>> >> org.apache.http.conn.HttpHostConnectException
>> >> Caused by: java.net.ConnectException
>> >>
>> >> org.apache.polygene.entitystore.redis.RedisMapEntityStoreTest >
>> classMethod
>> >> FAILED
>> >> java.lang.ExceptionInInitializerError
>> >> Caused by: java.lang.IllegalStateException at
>> >> RedisMapEntityStoreTest.java:38
>> >> Caused by: com.spotify.docker.client.
>> exceptions.DockerException
>> >> at RedisMapEntityStoreTest.java:38
>> >> Caused by: java.util.concurrent.ExecutionException
>at
>> >> RedisMapEntityStoreTest.java:38
>> >> Caused by: javax.ws.rs.ProcessingException
>> >> Caused by:
>> >> org.apache.http.conn.HttpHostConnectException
>> >> Caused by: java.net.ConnectException
>> >>
>> >> 2 tests completed, 2 failed
>> >>
>> >>
>> >> And the underlying exception is;
>> >>
>> >> Caused by: org.apache.http.conn.HttpHostConnectException: Connect
>to
>> >> localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1]
>failed:
>> >> Connection refused (Connection refused)
>> >>
>> >>
>> >> Which I presume is due to the gradle-docker-plugin going on the
>network
>> >> protocol instead of the command-line interface. The default docker
>> >> interface on Linux seems to be Unix sockets, which most Java apps
>tend
>> to
>> >> ignore and not support.
>> >>
>> >> 1059 ? Ssl 0:02 docker-containerd -l
>> >> unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
>> >> docker-containerd-shim --metrics-interval=0 --start-timeout 2m
>> --state-dir
>> >> /var/run/docker/libcontainerd/containerd --runtime docker-runc
>> >>
>> >>
>> >> Hope that helps. I am sending you login separately to a
>clean/unused
>> server
>> >> in Germany, in case you want to run experiments on it.
>> >>
>> >> Cheers
>> >>
>> >> On Sun, Jan 15, 2017 at 8:08 PM, Paul Merlin
><pa...@apache.org>
>> wrote:
>> >>
>> >>> Could anybody confirm it works for them?
>> >>>
>> >>> Paul Merlin a écrit :
>> >>>> Gang,
>> >>>>
>> >>>> I pushed changes to the build system and tests to automate
>integration
>> >>>> testing.
>> >>>>
>> >>>> It uses docker to build the necessary images and run docker
>containers
>> >>>> as a JUnit rule.
>> >>>> All this is done only if the DOCKER_HOST environment variable is
>set,
>> >>>> otherwise it's skipped.
>> >>>>
>> >>>> On linux with docker installed it should work out of the box
>given you
>> >>>> have DOCKER_HOST and DOCKER_CERT_PATH set.
>> >>>>
>> >>>> On mac you need to:
>> >>>> - docker-machine create someFancyNameYouChoose
>> >>>> - eval $(docker-machine env someFancyNameYouChoose)
>> >>>>
>> >>>> On windows, see https://docs.docker.com/machine/reference/env/
>> >>>>
>> >>>> With that done, when you now run e.g:
>> >>>>
>> >>>> ./gradlew :ext:entity-redis:test
>> >>>>
>> >>>> it will build the docker images and run Redis in a container
>around
>> >>>> the EntityStore tests.
>> >>>>
>> >>>> The same goes for these extensions:
>> >>>>
>> >>>> - Memcached Cache
>> >>>> - SQL EntityStore using both MySQL and PostgreSQL
>> >>>> - SQL Index/Query using PostgreSQL
>> >>>> - Riak EntityStore
>> >>>>
>> >>>> All the others extensions are tested using embedded solutions,
>> >>>> including MongoDB.
>> >>>>
>> >>>> Docker images are built by the :internals:testsupport-internal
>> >>>> project, you'll find the Dockerfiles there.
>> >>>>
>> >>>> Thanks to this I was able to fix a bug I introduced in the Riak
>ES in
>> >>>> recent changes.
>> >>>> The only thing actually broken is SQL Index/Query, see
>> >>>> https://issues.apache.org/jira/browse/POLYGENE-222
>> >>>>
>> >>>> For now these test do not run on CI.
>> >>>> builds.apache.org supports docker so we should be able to do
>that too
>> >>>> at some point.
>> >>>> In the meantime some of us should run the *whole* test suite
>from time
>> >>>> to time.
>> >>>> At least now it should be damn easy to do!
>> >>>>
>> >>>> Before calling this automation done I'd like to get someone to
>confirm
>> >>>> it 'works for them'.
>> >>>> I only tested on a mac.
>> >>>>
>> >>>> Cheers
>> >>>>
>> >>>> /Paul
>> >>>>
>> >
>>
>
>
>
>--
>Niclas Hedhman, Software Developer
>http://polygene.apache.org <http://zest.apache.org> - New Energy for
>Java
./gradlew test
for the whole test suite
--
Envoyé de mon appareil Android avec Courriel K-9 Mail. Veuillez excuser ma brièveté.
Re: Docker based integration testing
Posted by Niclas Hedhman <ni...@hedhman.org>.
So only the Docker daemon is needed?
And I run what target for ./gradlew ?
Cheers
Niclas
On Sat, Feb 25, 2017 at 10:33 PM, Paul Merlin <pa...@apache.org> wrote:
> Gang,
>
> I finally found time to take a stab at fixing Docker based integration
> testing on Linux.
>
> It should now work out of the box on Linux, given that a Docker service
> is running locally and that your user has the correct permissions.
>
> Docker based integration testing is now disabled if unable to contact a
> Docker service. No need to set DOCKER_HOST.
>
> Niclas, could you confirm it works for you?
>
> Cheers
>
> /Paul
>
>
>
> Paul Merlin a écrit :
> > Thanks for your feedback Niclas!
> >
> > Using the unix:///path/to/docker.sock URL on Linux should do.
> > I'll do another round on that later this week.
> >
> > Cheers
> >
> >
> > Niclas Hedhman a écrit :
> >> DOCKER_* may not be the best way to determine if Docker is present. I
> am on
> >> a Debian system, and no env variable named DOCKER* is set by default,
> yet
> >> docker is operational. IIUIC, these variables are needed for Docker
> Swarm
> >> and other specialized setup where one is not running the container on
> the
> >> same system as issuing the `docker` command.
> >>
> >> That said, running
> >> ./gradlew :ext:entity-redis:test
> >>
> >> without changing my defaults gives me some errors;
> >>
> >> :extensions:entitystore-redis:test
> >>
> >> org.apache.polygene.entitystore.redis.RedisMapEntityStoreWithCacheTest
> >
> >> classMethod FAILED
> >> java.lang.ExceptionInInitializerError
> >> Caused by: java.lang.IllegalStateException at
> >> RedisMapEntityStoreWithCacheTest.java:38
> >> Caused by: com.spotify.docker.client.
> exceptions.DockerException
> >> at RedisMapEntityStoreWithCacheTest.java:38
> >> Caused by: java.util.concurrent.ExecutionException at
> >> RedisMapEntityStoreWithCacheTest.java:38
> >> Caused by: javax.ws.rs.ProcessingException
> >> Caused by:
> >> org.apache.http.conn.HttpHostConnectException
> >> Caused by: java.net.ConnectException
> >>
> >> org.apache.polygene.entitystore.redis.RedisMapEntityStoreTest >
> classMethod
> >> FAILED
> >> java.lang.ExceptionInInitializerError
> >> Caused by: java.lang.IllegalStateException at
> >> RedisMapEntityStoreTest.java:38
> >> Caused by: com.spotify.docker.client.
> exceptions.DockerException
> >> at RedisMapEntityStoreTest.java:38
> >> Caused by: java.util.concurrent.ExecutionException at
> >> RedisMapEntityStoreTest.java:38
> >> Caused by: javax.ws.rs.ProcessingException
> >> Caused by:
> >> org.apache.http.conn.HttpHostConnectException
> >> Caused by: java.net.ConnectException
> >>
> >> 2 tests completed, 2 failed
> >>
> >>
> >> And the underlying exception is;
> >>
> >> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to
> >> localhost:2375 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed:
> >> Connection refused (Connection refused)
> >>
> >>
> >> Which I presume is due to the gradle-docker-plugin going on the network
> >> protocol instead of the command-line interface. The default docker
> >> interface on Linux seems to be Unix sockets, which most Java apps tend
> to
> >> ignore and not support.
> >>
> >> 1059 ? Ssl 0:02 docker-containerd -l
> >> unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim
> >> docker-containerd-shim --metrics-interval=0 --start-timeout 2m
> --state-dir
> >> /var/run/docker/libcontainerd/containerd --runtime docker-runc
> >>
> >>
> >> Hope that helps. I am sending you login separately to a clean/unused
> server
> >> in Germany, in case you want to run experiments on it.
> >>
> >> Cheers
> >>
> >> On Sun, Jan 15, 2017 at 8:08 PM, Paul Merlin <pa...@apache.org>
> wrote:
> >>
> >>> Could anybody confirm it works for them?
> >>>
> >>> Paul Merlin a écrit :
> >>>> Gang,
> >>>>
> >>>> I pushed changes to the build system and tests to automate integration
> >>>> testing.
> >>>>
> >>>> It uses docker to build the necessary images and run docker containers
> >>>> as a JUnit rule.
> >>>> All this is done only if the DOCKER_HOST environment variable is set,
> >>>> otherwise it's skipped.
> >>>>
> >>>> On linux with docker installed it should work out of the box given you
> >>>> have DOCKER_HOST and DOCKER_CERT_PATH set.
> >>>>
> >>>> On mac you need to:
> >>>> - docker-machine create someFancyNameYouChoose
> >>>> - eval $(docker-machine env someFancyNameYouChoose)
> >>>>
> >>>> On windows, see https://docs.docker.com/machine/reference/env/
> >>>>
> >>>> With that done, when you now run e.g:
> >>>>
> >>>> ./gradlew :ext:entity-redis:test
> >>>>
> >>>> it will build the docker images and run Redis in a container around
> >>>> the EntityStore tests.
> >>>>
> >>>> The same goes for these extensions:
> >>>>
> >>>> - Memcached Cache
> >>>> - SQL EntityStore using both MySQL and PostgreSQL
> >>>> - SQL Index/Query using PostgreSQL
> >>>> - Riak EntityStore
> >>>>
> >>>> All the others extensions are tested using embedded solutions,
> >>>> including MongoDB.
> >>>>
> >>>> Docker images are built by the :internals:testsupport-internal
> >>>> project, you'll find the Dockerfiles there.
> >>>>
> >>>> Thanks to this I was able to fix a bug I introduced in the Riak ES in
> >>>> recent changes.
> >>>> The only thing actually broken is SQL Index/Query, see
> >>>> https://issues.apache.org/jira/browse/POLYGENE-222
> >>>>
> >>>> For now these test do not run on CI.
> >>>> builds.apache.org supports docker so we should be able to do that too
> >>>> at some point.
> >>>> In the meantime some of us should run the *whole* test suite from time
> >>>> to time.
> >>>> At least now it should be damn easy to do!
> >>>>
> >>>> Before calling this automation done I'd like to get someone to confirm
> >>>> it 'works for them'.
> >>>> I only tested on a mac.
> >>>>
> >>>> Cheers
> >>>>
> >>>> /Paul
> >>>>
> >
>
--
Niclas Hedhman, Software Developer
http://polygene.apache.org <http://zest.apache.org> - New Energy for Java