You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Anton Kedin <ke...@google.com> on 2019/07/19 17:15:35 UTC
How to run DynamoDBIOTest?
Hi dev@,
Does anyone know if there's anything extra needed to run `DynamoDBIOTest`?
If I do `./graldew :sdks:java:io:amazon-web-services:build --debug` it
passes few tests during `:test` but then seems to sit on `DynamoDBIOTest`
forever. No errors, last meaningful log is `INFO: Container
localstack/localstack:0.8.6 started`. Happens on different machines, both
on master and release-2.14.0 branches.
Any pointers?
Regards,
Anton
Re: How to run DynamoDBIOTest?
Posted by Ismaël Mejía <ie...@gmail.com>.
Created https://issues.apache.org/jira/browse/BEAM-7794 to track this
issue, any extra info is welcomed in the meantime maybe a good idea to
disable the test.
On Mon, Jul 22, 2019 at 3:44 PM Elliotte Rusty Harold
<el...@ibiblio.org> wrote:
>
> FYI, I killed the job this morning after letting it run over the
> weeknd. Over 92 hours and no end in sight:
>
> <===========--> 84% EXECUTING [92h 1m 15s]
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > :sdks:java:io:amazon-web-services2:test > 4 tests completed
> > :sdks:java:io:amazon-web-services:test > 48 tests completed
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > :sdks:java:io:amazon-web-services:test > Executing test org.apache.beam.sdk.io.aws.dynamodb.DynamoDBIOTest
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > :sdks:java:io:amazon-web-services2:test > Executing test org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIOTest
> ^Celharo@elharo:~/beam$
> elharo@elharo:~/beam$
>
> On Sat, Jul 20, 2019 at 10:56 AM Elliotte Rusty Harold
> <el...@ibiblio.org> wrote:
> >
> > Never seems to time out or finish for me:
> >
> > $ ./gradlew :sdks:java:io:amazon-web-services:test
> > Starting a Gradle Daemon, 1 busy Daemon could not be reused, use
> > --status for details
> > Configuration on demand is an incubating feature.
> > <=<============-> 98% EXECUTING [22m 21s]
> > > IDLE
> > > IDLE
> > > IDLE
> > > IDLE
> > > :sdks:java:io:amazon-web-services:test > 48 tests completed
> > > IDLE
> > > :sdks:java:io:
> >
> > --
> > Elliotte Rusty Harold
> > elharo@ibiblio.org
>
>
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
Re: How to run DynamoDBIOTest?
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
FYI, I killed the job this morning after letting it run over the
weeknd. Over 92 hours and no end in sight:
<===========--> 84% EXECUTING [92h 1m 15s]
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> :sdks:java:io:amazon-web-services2:test > 4 tests completed
> :sdks:java:io:amazon-web-services:test > 48 tests completed
> IDLE
> IDLE
> IDLE
> IDLE
> :sdks:java:io:amazon-web-services:test > Executing test org.apache.beam.sdk.io.aws.dynamodb.DynamoDBIOTest
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> IDLE
> :sdks:java:io:amazon-web-services2:test > Executing test org.apache.beam.sdk.io.aws2.dynamodb.DynamoDBIOTest
^Celharo@elharo:~/beam$
elharo@elharo:~/beam$
On Sat, Jul 20, 2019 at 10:56 AM Elliotte Rusty Harold
<el...@ibiblio.org> wrote:
>
> Never seems to time out or finish for me:
>
> $ ./gradlew :sdks:java:io:amazon-web-services:test
> Starting a Gradle Daemon, 1 busy Daemon could not be reused, use
> --status for details
> Configuration on demand is an incubating feature.
> <=<============-> 98% EXECUTING [22m 21s]
> > IDLE
> > IDLE
> > IDLE
> > IDLE
> > :sdks:java:io:amazon-web-services:test > 48 tests completed
> > IDLE
> > :sdks:java:io:
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
--
Elliotte Rusty Harold
elharo@ibiblio.org
Re: How to run DynamoDBIOTest?
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
Never seems to time out or finish for me:
$ ./gradlew :sdks:java:io:amazon-web-services:test
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use
--status for details
Configuration on demand is an incubating feature.
<=<============-> 98% EXECUTING [22m 21s]
> IDLE
> IDLE
> IDLE
> IDLE
> :sdks:java:io:amazon-web-services:test > 48 tests completed
> IDLE
> :sdks:java:io:
--
Elliotte Rusty Harold
elharo@ibiblio.org
Re: How to run DynamoDBIOTest?
Posted by Cam Mach <ca...@gmail.com>.
Hi Anton,
It should not be a machine issue, since you have the container got started.
I would say give it around 5 - 10 minutes, it would throw out connection
time out error log. This usually happens when the integration tests can't
connect to the container. As you may notice, in the tests we don't hard
code the container IP and port, and we get it dynamically from the
TestContainer framework. Which mean, for some reason, the framework
couldn't allocate IP and port for the container. At this moment I don't
have a quick fix, but if it happens to several dev, we can file a Jira to
their dashboard, or get rid of it.
P.S. try this gradlew `./gradlew :sdks:java:io:amazon-web-services:test`
Thanks,
On Sat, Jul 20, 2019 at 6:33 AM Elliotte Rusty Harold <el...@ibiblio.org>
wrote:
> I've seen this myself trying to run tests in Linux (Goobuntu).
>
> On Fri, Jul 19, 2019 at 1:15 PM Anton Kedin <ke...@google.com> wrote:
> >
> > Hi dev@,
> >
> > Does anyone know if there's anything extra needed to run
> `DynamoDBIOTest`? If I do `./graldew
> :sdks:java:io:amazon-web-services:build --debug` it passes few tests during
> `:test` but then seems to sit on `DynamoDBIOTest` forever. No errors, last
> meaningful log is `INFO: Container localstack/localstack:0.8.6 started`.
> Happens on different machines, both on master and release-2.14.0 branches.
> >
> > Any pointers?
> >
> > Regards,
> > Anton
>
>
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
>
Re: How to run DynamoDBIOTest?
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
I've seen this myself trying to run tests in Linux (Goobuntu).
On Fri, Jul 19, 2019 at 1:15 PM Anton Kedin <ke...@google.com> wrote:
>
> Hi dev@,
>
> Does anyone know if there's anything extra needed to run `DynamoDBIOTest`? If I do `./graldew :sdks:java:io:amazon-web-services:build --debug` it passes few tests during `:test` but then seems to sit on `DynamoDBIOTest` forever. No errors, last meaningful log is `INFO: Container localstack/localstack:0.8.6 started`. Happens on different machines, both on master and release-2.14.0 branches.
>
> Any pointers?
>
> Regards,
> Anton
--
Elliotte Rusty Harold
elharo@ibiblio.org
Re: How to run DynamoDBIOTest?
Posted by Anton Kedin <ke...@google.com>.
One of the machines is macOS 10.14.5, docker desktop 2.0.0.3 (engine
18.09.2), java 1.8.0_211 (I believe Oracle version). Test log:
https://gist.github.com/akedin/da6fbc8a993f758302a6f64c42bba11b#file-gistfile1-txt
It
then spins forever with only gradle logs. Another machine I tried it on is
something debian-based, open jdk 8 212, docker 18.09.3, logs are similar
(don't have access to the details at the moment).
Regards,
Anton
On Fri, Jul 19, 2019 at 2:01 PM Ismaël Mejía <ie...@gmail.com> wrote:
> This looks weird, I run the build in my machine (Ubuntu linux 18.04 +
> OpenJDK 1.8.0_201) + docker 18.09.8 in both master and the release
> 2.14.0 branch and it executes without issue.
> This test uses a docker image as a sort of embedded server to simulate
> the Amazon backend (localstack).
> All builds were green when merged. Do you get any extra logs Anton?
> what is your OS / Java version?
> Adding Cam to the discussion since he contributed this feature to see
> if he may have any extra context.
>
> On Fri, Jul 19, 2019 at 7:15 PM Anton Kedin <ke...@google.com> wrote:
> >
> > Hi dev@,
> >
> > Does anyone know if there's anything extra needed to run
> `DynamoDBIOTest`? If I do `./graldew
> :sdks:java:io:amazon-web-services:build --debug` it passes few tests during
> `:test` but then seems to sit on `DynamoDBIOTest` forever. No errors, last
> meaningful log is `INFO: Container localstack/localstack:0.8.6 started`.
> Happens on different machines, both on master and release-2.14.0 branches.
> >
> > Any pointers?
> >
> > Regards,
> > Anton
>
Re: How to run DynamoDBIOTest?
Posted by Ismaël Mejía <ie...@gmail.com>.
This looks weird, I run the build in my machine (Ubuntu linux 18.04 +
OpenJDK 1.8.0_201) + docker 18.09.8 in both master and the release
2.14.0 branch and it executes without issue.
This test uses a docker image as a sort of embedded server to simulate
the Amazon backend (localstack).
All builds were green when merged. Do you get any extra logs Anton?
what is your OS / Java version?
Adding Cam to the discussion since he contributed this feature to see
if he may have any extra context.
On Fri, Jul 19, 2019 at 7:15 PM Anton Kedin <ke...@google.com> wrote:
>
> Hi dev@,
>
> Does anyone know if there's anything extra needed to run `DynamoDBIOTest`? If I do `./graldew :sdks:java:io:amazon-web-services:build --debug` it passes few tests during `:test` but then seems to sit on `DynamoDBIOTest` forever. No errors, last meaningful log is `INFO: Container localstack/localstack:0.8.6 started`. Happens on different machines, both on master and release-2.14.0 branches.
>
> Any pointers?
>
> Regards,
> Anton