You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "github-actions[bot] (via GitHub)" <gi...@apache.org> on 2024/11/01 14:19:44 UTC
[I] The PostCommit Python job is flaky [beam]
github-actions[bot] opened a new issue, #30513:
URL: https://github.com/apache/beam/issues/30513
The PostCommit Python is failing over 50% of the time
Please visit https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python.yml?query=is%3Afailure+branch%3Amaster to see the logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3141773015
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2173566529
Green for last two days.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2468624543
sg, will see if the fix in my mind will can work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2761715666
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3661916774
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3444427392
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4778992987
> > Another issue is related to a timeout on Python 3.12 (e.g. https://github.com/apache/beam/actions/runs/26970785446/job/79585162802)
> > From the raw log, it seems that the task of ":sdks:python:test-suites:portable:py312:postCommitPy312" is stuck.
> > ```
> > 2026-06-18T09:10:12.7126060Z > Task :sdks:python:test-suites:portable:py312:postCommitPy312IT
> > ...
> > 2026-06-18T09:20:17.6125359Z
> > 2026-06-18T09:20:17.6127989Z > Task :sdks:python:test-suites:portable:py312:postCommitPy312
> > 2026-06-18T12:54:08.8113512Z ##[error]The operation was canceled.
> > ```
>
> 3.12 started to fail (timeout) from June, 4. Didn't find the cause yet
Right. There is a sudden jump on the runtime on the between the following two test runs:
- https://github.com/apache/beam/actions/runs/26951735917 (Jun 4, 8:29 AM EDT)
- https://github.com/apache/beam/actions/runs/26970785446 (Jun 4, 2:15 PM EDT)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045550204
> I'm curious if not related - [#39284](https://github.com/apache/beam/pull/39284)
@stankiewicz yeah i think #39284 is related.
after that PR bumped kafka to 3.9.2 and swapped KafkaServerStartable to KafkaServer the local test broker in LocalKafka still uses the old port / log.dir settings and those dont actually bind a listener on kafka 3.x anymore so the broker never comes up
in the postcommit logs you see stuff like:
Connection to node -1 (localhost/127.0.0.1:...) could not be established. Node may not be available
and then it surfaces as the topic metadata timeout (Topic ... not present in metadata after 60000 ms), so i think the real issue is the broker isn't reachable but the new upgrade broke theKafka test harness
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-1989410199
I think we can pretty comfortably rule out that change, it was to the yaml sdk which is unrelated to portableWordCountSparkRunnerBatch. Note that this runs on a schedule, not on commits, though none of the commits in that scheduled time look particularly harmful
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2468573494
@shunping I think Damon is on vacation, if there is a quick fix please go ahead and apply it, otherwise could you please revert and we can try again when Damon is back/after the 2.61.0 release
cc/ @Abacn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "akashorabek (via GitHub)" <gi...@apache.org>.
akashorabek closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4533733412
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3793704816
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3835591360
The PostCommit Python workflow is failing now on the Run Python PostCommit 3.10 job and these tests are failing which use the Mock API Echo service:
test_http_enrichment
test_http_enrichment_custom_join
test_request_response_io
as The Mock api echo service at http://10.138.0.32:8080/ is returning 500 Internal Server Error and the tests send requests with id echo-should-never-exceed-quota and receive 500 instead of the expected echo also these tests are part of inferencePostCommitIT which only runs on Python 3.10 PostCommit job
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4778783916
> Another issue is related to a timeout on Python 3.12 (e.g. https://github.com/apache/beam/actions/runs/26970785446/job/79585162802)
>
> From the raw log, it seems that the task of ":sdks:python:test-suites:portable:py312:postCommitPy312" is stuck.
>
> ```
> 2026-06-18T09:10:12.7126060Z > Task :sdks:python:test-suites:portable:py312:postCommitPy312IT
> ...
> 2026-06-18T09:20:17.6125359Z
> 2026-06-18T09:20:17.6127989Z > Task :sdks:python:test-suites:portable:py312:postCommitPy312
> 2026-06-18T12:54:08.8113512Z ##[error]The operation was canceled.
> ```
3.12 started to fail from June, 4. Didn't find the cause yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4792477326
The gradle task for vllm ran successfully: https://github.com/apache/beam/actions/runs/28111217064.
Now we need to wait for a new beam master built so the updated pandas can be used in dataflow tests, which should fix the remaining failure in postcommit python.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5874618782
@aIbrahiim: have you experimented on your idea in https://github.com/apache/beam/issues/30513#issuecomment-5795669787? `LeaderBoardIT.test_leader_board_it` is still flaky.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5876116815
> On the PR PostCommit runs, LeaderBoardIT passed on 3.10 / 3.11 / 3.12, but we still saw at least one empty-checksum fail (e.g. 3.14), so it looks reduced, not fully gone
there is a PR that might address this: https://github.com/apache/beam/pull/40297
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5615316922
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2082755326
Only sorta - each component job is actually not permared - e.g. there are 2 successes here, https://github.com/apache/beam/actions/runs/8873798546
The whole workflow is permared just because our flake percentage is so high
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2469074574
Ok, take another look at this.
The test started to fail at 11/06 6:32PM (https://github.com/apache/beam/actions/runs/11713650994), the last successful run was at 11/06 12:33PM (https://github.com/apache/beam/actions/runs/11708854671). There are two commits during this time internal:
- Distro Python SDK: https://github.com/apache/beam/commit/81f35ab62298a2ec9fadeded82461b363b6401db, which causes the previously mentioned error during docker image building (:sdks:python:container:py39:docker)
- Kafka: https://github.com/apache/beam/commit/eeebae1bda6b211463e53a4e4ca469bfa9763399, which seems to be the reason for failure (:sdks:python:test-suites:portable:py39:postCommitPy39IT).
The Kafka error message is shown below:
```
FAILED apache_beam/io/external/xlang_kafkaio_it_test.py::CrossLanguageKafkaIOTest::test_local_kafkaio_populated_key - RuntimeError: Pipeline BeamApp-runner-1111115329-514dd26a_03822608-80d0-4037-bc13-11d632204f46 failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 3: org.apache.beam.sdk.util.UserCodeException: java.io.IOException: KafkaWriter : failed to send 1 records (since last report)
at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39)
at org.apache.beam.sdk.io.kafka.KafkaWriter$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:810)
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348)
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275)
at org.apache.beam.fn.harness.FnApiDoFnRunner.outputTo(FnApiDoFnRunner.java:1837)
at org.apache.beam.fn.harness.FnApiDoFnRunner.access$3100(FnApiDoFnRunner.java:145)
at org.apache.beam.fn.harness.FnApiDoFnRunner$NonWindowObservingProcessBundleContext.output(FnApiDoFnRunner.java:2695)
at org.apache.beam.sdk.transforms.MapElements$2.processElement(MapElements.java:151)
at org.apache.beam.sdk.transforms.MapElements$2$DoFnInvoker.invokeProcessElement(Unknown Source)
at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForParDo(FnApiDoFnRunner.java:810)
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:348)
at org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$MetricTrackingFnDataReceiver.accept(PCollectionConsumerRegistry.java:275)
at org.apache.beam.fn.harness.BeamFnDataReadRunner.forwardElementToConsumer(BeamFnDataReadRunner.java:213)
at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.multiplexElements(BeamFnDataInboundObserver.java:172)
at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:136)
at org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle(ProcessBundleHandler.java:550)
at org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:150)
at org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:115)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:163)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.io.IOException: KafkaWriter : failed to send 1 records (since last report)
at org.apache.beam.sdk.io.kafka.KafkaWriter.checkForFailures(KafkaWriter.java:183)
at org.apache.beam.sdk.io.kafka.KafkaWriter.processElement(KafkaWriter.java:66)
Caused by: org.apache.kafka.common.errors.TimeoutException: Topic xlang_kafkaio_test_populated_key_e9df3a07-037f-45a1-afde-7cea599f9570 not present in metadata after 60000 ms.
```
@Abacn , could you check this and see if we need to roll it back?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2139155688
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2630602367
Job has been stable for 4 days. Closing as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3258490184
Workflow stability has decreased after the #35621
CC @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3626999457
Possible culprit is #36772 or #36853
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3406871807
gbek tests are very flaky:
```
1 out of 2 runs failed: test_combineValues_with_gbek_it (apache_beam.transforms.core_it_test.GbekIT)
sdks/python/pytest_postCommitIT-df-py312.xml [took 6m 52s]
with exception 404 Secret [projects/844138762903/secrets/gbek_secret_tests_534926] not found or has no versions.
1 out of 2 runs with error: test_combineValues_with_gbek_it (apache_beam.transforms.core_it_test.GbekIT)
google.api_core.exceptions.NotFound: 404 Secret [projects/844138762903/secrets/gbek_secret_tests_534926] not found.
failed on teardown with "google.api_core.exceptions.NotFound: 404 Secret [projects/844138762903/secrets/gbek_secret_tests_643039] not found."
```
Looks like the secret name remains the same across the matrix and tests (gbek_secret_tests_534926 or 643039) causing conflicts
cc: @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4800253311
The failed tests related to pandas upgrade are all gone, but there are 5 failed tests left:
- `FAILED apache_beam/io/gcp/bigquery_file_loads_test.py::BigQueryFileLoadsIT::test_batch_copy_jobs_with_no_input_schema`
- `FAILED apache_beam/io/gcp/bigquery_write_it_test.py::BigQueryWriteIntegrationTests::test_big_query_write_temp_table_append_schema_update_2`
- `FAILED apache_beam/io/gcp/bigquery_write_it_test.py::BigQueryWriteIntegrationTests::test_big_query_write_temp_table_append_schema_update_0`
- `FAILED apache_beam/io/gcp/bigquery_write_it_test.py::BigQueryWriteIntegrationTests::test_big_query_write_temp_table_append_schema_update_1`
- `FAILED apache_beam/io/gcp/bigquery_test.py::BigQueryFileLoadsIntegrationTests::test_avro_file_load`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2320226684
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2909634761
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2714448477
Job has been stable for 5 days. Closing as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-1989367856
It first failed on https://github.com/apache/beam/actions/runs/8210266873.
The failed task is `:sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch`.
Traceback:
```
INFO:apache_beam.utils.subprocess_server:Starting service with ('java' '-jar' '/runner/_work/beam/beam/runners/spark/3/job-server/build/libs/beam-runners-spark-3-job-server-2.56.0-SNAPSHOT.jar' '--spark-master-url' 'local[4]' '--artifacts-dir' '/tmp/beam-temp8q8022zi/artifactsg6e8usou' '--job-port' '56313' '--artifact-port' '0' '--expansion-port' '0')
INFO:apache_beam.utils.subprocess_server:Error: A JNI error has occurred, please check your installation and try again
INFO:apache_beam.utils.subprocess_server:Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/beam/vendor/grpc/v1p60p1/io/grpc/BindableService
INFO:apache_beam.utils.subprocess_server: at java.lang.ClassLoader.defineClass1(Native Method)
INFO:apache_beam.utils.subprocess_server: at java.lang.ClassLoader.defineClass(ClassLoader.java:757)
INFO:apache_beam.utils.subprocess_server: at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
INFO:apache_beam.utils.subprocess_server: at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
INFO:apache_beam.utils.subprocess_server: at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
INFO:apache_beam.utils.subprocess_server: at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
INFO:apache_beam.utils.subprocess_server: at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
INFO:apache_beam.utils.subprocess_server: at java.security.AccessController.doPrivileged(Native Method)
INFO:apache_beam.utils.subprocess_server: at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
INFO:apache_beam.utils.subprocess_server: at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
INFO:apache_beam.utils.subprocess_server: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
INFO:apache_beam.utils.subprocess_server: at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
INFO:apache_beam.utils.subprocess_server: at java.lang.Class.getDeclaredMethods0(Native Method)
INFO:apache_beam.utils.subprocess_server: at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
INFO:apache_beam.utils.subprocess_server: at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
INFO:apache_beam.utils.subprocess_server: at java.lang.Class.getMethod0(Class.java:3018)
INFO:apache_beam.utils.subprocess_server: at java.lang.Class.getMethod(Class.java:1784)
INFO:apache_beam.utils.subprocess_server: at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:670)
INFO:apache_beam.utils.subprocess_server: at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:652)
INFO:apache_beam.utils.subprocess_server:Caused by: java.lang.ClassNotFoundException: org.apache.beam.vendor.grpc.v1p60p1.io.grpc.BindableService
INFO:apache_beam.utils.subprocess_server: at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
INFO:apache_beam.utils.subprocess_server: at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
INFO:apache_beam.utils.subprocess_server: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
INFO:apache_beam.utils.subprocess_server: at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
INFO:apache_beam.utils.subprocess_server: ... 19 more
ERROR:apache_beam.utils.subprocess_server:Started job service with ('java', '-jar', '/runner/_work/beam/beam/runners/spark/3/job-server/build/libs/beam-runners-spark-3-job-server-2.56.0-SNAPSHOT.jar', '--spark-master-url', 'local[4]', '--artifacts-dir', '/tmp/beam-temp8q8022zi/artifactsg6e8usou', '--job-port', '56313', '--artifact-port', '0', '--expansion-port', '0')
ERROR:apache_beam.utils.subprocess_server:Error bringing up service
Traceback (most recent call last):
File "/runner/_work/beam/beam/sdks/python/apache_beam/utils/subprocess_server.py", line 175, in start
raise RuntimeError(
RuntimeError: Service failed to start up with error 1
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/runner/_work/beam/beam/sdks/python/apache_beam/examples/wordcount.py", line 111, in <module>
run()
File "/runner/_work/beam/beam/sdks/python/apache_beam/examples/wordcount.py", line 106, in run
output | 'Write' >> WriteToText(known_args.output)
File "/runner/_work/beam/beam/sdks/python/apache_beam/pipeline.py", line 612, in __exit__
self.result = self.run()
File "/runner/_work/beam/beam/sdks/python/apache_beam/pipeline.py", line 586, in run
return self.runner.run_pipeline(self, self._options)
File "/runner/_work/beam/beam/sdks/python/apache_beam/runners/runner.py", line 192, in run_pipeline
return self.run_portable_pipeline(
File "/runner/_work/beam/beam/sdks/python/apache_beam/runners/portability/portable_runner.py", line 381, in run_portable_pipeline
job_service_handle = self.create_job_service(options)
File "/runner/_work/beam/beam/sdks/python/apache_beam/runners/portability/portable_runner.py", line 296, in create_job_service
return self.create_job_service_handle(server.start(), options)
File "/runner/_work/beam/beam/sdks/python/apache_beam/runners/portability/job_server.py", line 81, in start
self._endpoint = self._job_server.start()
File "/runner/_work/beam/beam/sdks/python/apache_beam/runners/portability/job_server.py", line 110, in start
return self._server.start()
File "/runner/_work/beam/beam/sdks/python/apache_beam/utils/subprocess_server.py", line 175, in start
raise RuntimeError(
RuntimeError: Service failed to start up with error 1
> Task :sdks:python:test-suites:portable:py38:portableWordCountSparkRunnerBatch FAILED
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-1989374642
Added the owner of the commit whose post-commit job failed at the first time.
@damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2468551929
There is no `/usr/lib/python3.9` under in the image python:3.9-bookworm. I can only see python3 and python3.11 folders there, and I think we may need to copy the python3 one.
```
$ docker run -it python:3.9-bookworm bash
root@b730cccba5a8:/# ls -d /usr/lib/python*
/usr/lib/python3 /usr/lib/python3.11
root@b730cccba5a8:/# ls -d /usr/local/lib/python*
/usr/local/lib/python3.11 /usr/local/lib/python3.9
```
@damondouglas , could you confirm that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2286763118
We need bump sklearn requirements here: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/sklearn_examples_requirements.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4780831501
I am trying to revert 3 python PRs that were pushed between the two time points in https://github.com/apache/beam/issues/30513#issuecomment-4778992987: https://github.com/apache/beam/pull/39073
Will see if the error still persists.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4790960764
The latest image was actually built to verify a PR #38701.
Looks like @damccorm made the push, but it didn't include amd64 architecture.
https://github.com/apache/beam/pull/38701#issuecomment-4625447873
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045553933
cc: @sjvanrossum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4833203674
Recent runs are healthy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4219065941
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3617468039
Execution failed for task ':sdks:python:test-suites:dataflow:py310:mockAPITests'.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3627068040
Its https://github.com/apache/beam/pull/36772 (sort of).
```
Execution failed for task ':sdks:python:test-suites:dataflow:py310:mockAPITests'.
> Process 'command 'sh'' finished with non-zero exit value 5
```
This means no tests ran (https://github.com/apache/beam/blob/32345e7cbe99084ec3c072dc628c1aed70ddc9a8/sdks/python/scripts/run_pytest.sh#L144). If I try running this task, I see:
```
SKIPPED [1] apache_beam/examples/ml_transform/ml_transform_it_test.py:34: unittest.case.SkipTest: tensorflow_transform is not installed.
SKIPPED [1] apache_beam/transforms/enrichment_handlers/cloudsql_it_test.py:59: unittest.case.SkipTest: CloudSQL dependencies not installed: No module named 'redis'
SKIPPED [1] apache_beam/examples/snippets/transforms/elementwise/mltransform_test.py:36: unittest.case.SkipTest: tensorflow_transform is not installed.
SKIPPED [1] apache_beam/examples/snippets/transforms/elementwise/runinference_test.py:38: unittest.case.SkipTest: PyTorch dependencies are not installed
SKIPPED [1] apache_beam/io/requestresponse_it_test.py:45: unittest.case.SkipTest: RequestResponseIO dependencies are not installed.
SKIPPED [1] apache_beam/ml/anomaly/detectors/pyod_adapter_test.py:44: unittest.case.SkipTest: PyOD dependencies are not installed
SKIPPED [1] apache_beam/ml/inference/gemini_inference_it_test.py:35: unittest.case.SkipTest: Gemini model handler dependencies are not installed
SKIPPED [1] apache_beam/ml/inference/gemini_inference_test.py:29: unittest.case.SkipTest: Gemini dependencies are not installed
SKIPPED [1] apache_beam/ml/inference/huggingface_inference_it_test.py:34: unittest.case.SkipTest: transformers dependencies are not installed. Check if transformers, torch, and tensorflow is installed.
SKIPPED [1] apache_beam/ml/inference/tensorflow_inference_test.py:52: unittest.case.SkipTest: Tensorflow dependencies are not installed. Make sure you have both tensorflow and tensorflow_hub installed.
SKIPPED [1] apache_beam/ml/inference/onnx_inference_test.py:35: unittest.case.SkipTest: TODO: fix https://github.com/apache/beam/issues/31254
SKIPPED [1] apache_beam/ml/inference/pytorch_inference_test.py:49: unittest.case.SkipTest: PyTorch dependencies are not installed
SKIPPED [1] apache_beam/ml/inference/tensorrt_inference_test.py:41: unittest.case.SkipTest: TensorRT dependencies are not installed
SKIPPED [1] apache_beam/ml/inference/vertex_ai_inference_it_test.py:33: unittest.case.SkipTest: Vertex AI model handler dependencies are not installed
SKIPPED [1] apache_beam/ml/inference/xgboost_inference_test.py:46: unittest.case.SkipTest: XGBoost dependencies are not installed
SKIPPED [1] apache_beam/ml/transforms/handlers_test.py:50: unittest.case.SkipTest: tensorflow_transform is not installed.
SKIPPED [1] apache_beam/ml/transforms/tft_test.py:40: unittest.case.SkipTest: tensorflow_transform is not installed.
SKIPPED [1] apache_beam/runners/dask/dask_runner_test.py:36: unittest.case.SkipTest: Dask must be installed to run tests.
SKIPPED [1] apache_beam/testing/analyzers/perf_analysis_test.py:47: unittest.case.SkipTest: Missing dependencies to run perf analysis tests.
SKIPPED [1] apache_beam/testing/benchmarks/cloudml/cloudml_benchmark_test.py:31: unittest.case.SkipTest: Dependencies are not installed
SKIPPED [1] apache_beam/transforms/enrichment_handlers/bigquery_it_test.py:42: unittest.case.SkipTest: Google Cloud BigQuery dependencies are not installed.
SKIPPED [1] apache_beam/transforms/enrichment_handlers/bigtable_it_test.py:42: unittest.case.SkipTest: Bigtable test dependencies are not installed.
SKIPPED [1] apache_beam/transforms/enrichment_handlers/feast_feature_store_it_test.py:40: unittest.case.SkipTest: Feast feature store test dependencies are not installed.
SKIPPED [1] apache_beam/transforms/enrichment_handlers/feast_feature_store_test.py:25: unittest.case.SkipTest: Feast feature store test dependencies are not installed.
SKIPPED [1] apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store_it_test.py:39: unittest.case.SkipTest: VertexAI Feature Store test dependencies are not installed.
SKIPPED [1] apache_beam/typehints/pytorch_type_compatibility_test.py:38: unittest.case.SkipTest: PyTorch dependencies are not installed
SKIPPED [1] apache_beam/yaml/yaml_ml_test.py:31: unittest.case.SkipTest: tensorflow_transform is not installed.
```
So we're now skipping tests that should have run, but we were also skipping others before (e.g. https://github.com/apache/beam/blob/32345e7cbe99084ec3c072dc628c1aed70ddc9a8/sdks/python/apache_beam/io/requestresponse_it_test.py#L142)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2286747319
Currently Python3.12 Dataflow test has two test failing consistently:
```
apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_mnist_classification
apache_beam/ml/inference/sklearn_inference_it_test.py::SklearnInference::test_sklearn_mnist_classification_large_model
Error:
subprocess.CalledProcessError: Command '['/runner/_work/beam/beam/build/gradleenv/2050596100/bin/python3.12', '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', '/tmp/tmpoq1ebvgy/tmp_requirements.txt', '--exists-action', 'i', '--no-deps', '--implementation', 'cp', '--abi', 'cp312', '--platform', 'manylinux2014_x86_64']' returned non-zero exit status 1.
Error compiling Cython file:
sklearn/utils/_vector_sentinel.pyx:31:9: Previous declaration is here
```
Cannot install sklearn from source using cython
happened as early as https://github.com/apache/beam/commits/5b2bfe96f83a5631c3a8d5c3b92a0f695ffe2d7d
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2321349181
2024-08-30T07:28:39.6571287Z if setup_options.setup_file is not None:
2024-08-30T07:28:39.6571763Z if not os.path.isfile(setup_options.setup_file):
2024-08-30T07:28:39.6572227Z > raise RuntimeError(
2024-08-30T07:28:39.6572923Z 'The file %s cannot be found. It was specified in the '
2024-08-30T07:28:39.6573578Z '--setup_file command line option.' % setup_options.setup_file)
2024-08-30T07:28:39.6574970Z [1m[31mE RuntimeError: The file /runner/_work/beam/beam/sdks/python/apache_beam/examples/complete/juliaset/src/setup.py cannot be found. It was specified in the --setup_file command line option.[0m
https://productionresultssa6.blob.core.windows.net/actions-results/9f18d66f-dabf-46e8-8b29-ae50d075f3dd/workflow-job-run-912db29d-d57b-5850-6efb-b125ca814b95/logs/job/job-logs.txt?rsct=text%2Fplain&se=2024-08-30T14%3A06%3A43Z&sig=aqESnfP68oo0sF7TUtpq%2BNFgdgfCbq8Ey3q%2BFMLZtvI%3D&ske=2024-08-31T00%3A21%3A54Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2024-08-30T12%3A21%3A54Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2024-05-04&sp=r&spr=https&sr=b&st=2024-08-30T13%3A56%3A38Z&sv=2024-05-04
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2451953578
@jrmccluskey would you mind taking a look at these?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2468497963
It started to fail last week again (Friday days ago) since the distroless python sdk PR: https://github.com/apache/beam/commit/81f35ab62298a2ec9fadeded82461b363b6401db.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4150097917
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3851769744
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4034650998
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2722744163
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2724822414
Job has been stable for 1 day. Closing as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2973537633
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5302292529
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045328509
I investigated last failures and found out that the workflow is failing on KafkaIO tests with:
TimeoutException: Topic xlang_kafkaio_test_... not present in metadata after 60000 ms.
so it looks like the Kafka producer cant fetch topic metadata in time (hits the default 60s timeout), which then blows up KafkaWriter and takes down the flink job as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4777675164
Another issue is related to a timeout on Python 3.12 (e.g. https://github.com/apache/beam/actions/runs/26970785446/job/79585162802)
From the raw log, it seems that the task of ":sdks:python:test-suites:portable:py312:postCommitPy312" is stuck.
```
2026-06-18T09:10:12.7126060Z > Task :sdks:python:test-suites:portable:py312:postCommitPy312IT
...
2026-06-18T09:20:17.6125359Z
2026-06-18T09:20:17.6127989Z > Task :sdks:python:test-suites:portable:py312:postCommitPy312
2026-06-18T12:54:08.8113512Z ##[error]The operation was canceled.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4777183082
[Update] The latest postcommit failure on Python versions other than 3.12 is due to the update of pandas (https://github.com/apache/beam/pull/39056).
```
E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
E Traceback (most recent call last):
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/sdk_worker.py", line 310, in _execute
E response = task()
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/sdk_worker.py", line 384, in <lambda>
E lambda: self.create_worker().do_instruction(request), request)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/sdk_worker.py", line 663, in do_instruction
E return getattr(self, request_type)(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/sdk_worker.py", line 695, in process_bundle
E bundle_processor = self.bundle_processor_cache.get(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/sdk_worker.py", line 511, in get
E processor = bundle_processor.BundleProcessor(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1136, in __init__
E self.ops = future.result(timeout=3600)
E File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
E return self.__get_result()
E File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
E raise self._exception
E File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
E result = self.fn(*self.args, **self.kwargs)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1212, in create_execution_tree
E return collections.OrderedDict([(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1215, in <listcomp>
E get_operation(transform_id))) for transform_id in sorted(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1036, in wrapper
E result = cache[args] = func(*args)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1189, in get_operation
E transform_consumers = ***
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <dictcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <listcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1036, in wrapper
E result = cache[args] = func(*args)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1189, in get_operation
E transform_consumers = ***
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <dictcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <listcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1036, in wrapper
E result = cache[args] = func(*args)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1189, in get_operation
E transform_consumers = ***
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <dictcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <listcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1036, in wrapper
E result = cache[args] = func(*args)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1189, in get_operation
E transform_consumers = ***
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <dictcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <listcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1036, in wrapper
E result = cache[args] = func(*args)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1189, in get_operation
E transform_consumers = ***
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <dictcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1190, in <listcomp>
E tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1036, in wrapper
E result = cache[args] = func(*args)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1200, in get_operation
E return transform_factory.create_operation(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1518, in create_operation
E return creator(self, transform_id, transform_proto, payload, consumers)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1849, in create_par_do
E return _create_pardo_operation(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1893, in _create_pardo_operation
E dofn_data = pickler.loads(serialized_fn)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/internal/pickler.py", line 72, in loads
E return desired_pickle_lib.loads(
E File "/usr/local/lib/python3.10/site-packages/apache_beam/internal/cloudpickle_pickler.py", line 224, in loads
E return _loads(s)
E File "/usr/local/lib/python3.10/site-packages/apache_beam/internal/cloudpickle_pickler.py", line 229, in _loads
E unpickled = cloudpickle.loads(s)
E TypeError: StringDtype.__init__() takes from 1 to 2 positional arguments but 3 were given
```
There is change of signature on `StringDtype` in pandas 2.3.x:
- https://github.com/pandas-dev/pandas/blob/2.2.x/pandas/core/arrays/string_.py#L118
- https://github.com/pandas-dev/pandas/blob/2.3.x/pandas/core/arrays/string_.py#L146
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "akashorabek (via GitHub)" <gi...@apache.org>.
akashorabek commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2812638999
Closing the issue since the last 4 runs were green
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2293977760
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2176008981
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4188705247
This pr addresses and fixes the errors: https://github.com/apache/beam/pull/38069
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4251567603
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2486211355
green now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3414102003
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3555642146
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3725559381
Fixed by https://github.com/apache/beam/pull/37250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4739939512
Possible causes are: #38749, #38797, #38778
Currently testing reverts
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045909434
> Not sure if `port=$PORT` is still around, replacing with `listeners=PLAINTEXT://localhost:$PORT`.
>
> Yeah, that's probably it considering [apache/kafka#10872](https://github.com/apache/kafka/pull/10872).
yeah same place i landed while poking at the logs as port= looks like a no-op on 3.9 so swapping to listeners=PLAINTEXT://localhost:$PORT makes sense and this might lines up with the connection refused on the random test port
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "sjvanrossum (via GitHub)" <gi...@apache.org>.
sjvanrossum commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045875517
Not sure if `port=$PORT` is still around, replacing with `listeners=PLAINTEXT://localhost:$PORT`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4619164602
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5536836184
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5776757191
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5796126716
The query of `SELECT total_score FROM `apache-beam-testing.leader_board_it_dataset*.leader_board_users` WHERE total_score=5000 LIMIT 1` is running against the user score table in the code.
https://github.com/apache/beam/blob/d1ebefb6f6301f6b99358b63dafa1c206cce5849/sdks/python/apache_beam/examples/complete/game/leader_board.py#L354-L363
I see 4 rows should be written in that step.
<img width="1120" height="1672" alt="Image" src="https://github.com/user-attachments/assets/d0545e0f-c8f3-40f5-8c47-ddcf8d678f78" />
> I was thinking we could pass timeout_secs (maybe 5 min) on both users/teams matchers in leader_board_it_test.py keep the same query and expected checksum, and let it retry until the data shows up
Sounds good. Please try that and see if we can reduce the test flakiness.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2454732214
Looks like after sickbaying TensorRT tests, there are still failures. https://ge.apache.org/s/27igat7sfmcsu/console-log/task/:sdks:python:test-suites:portable:py310:portableWordCountSparkRunnerBatch?anchor=60&page=1 is an example, it looks like we're failing because we're missing a class in the spark runner.
@Abacn would you mind taking a look? Its unclear why this is happening now, but I'm guessing it may be related to https://github.com/apache/beam/pull/32976 (and maybe some caching kept it from showing up?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2137655511
Great. Thanks @liferoad
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2082738212
Permared right now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2156168672
New error:
```
==================================== ERRORS ==================================== |
-- | --
| [31m[1m________________ ERROR at setup of ReadTests.test_native_source ________________[0m |
| [gw5] linux -- Python 3.9.19 /runner/_work/beam/beam/build/gradleenv/1398941893/bin/python3.9 |
| |
| self = <apache_beam.io.gcp.bigquery_tools.BigQueryWrapper object at 0x7f248f59baf0> |
| project_id = 'apache-beam-testing' |
| dataset_id = 'python_read_table_17178042710ffd3b', location = None |
| labels = None |
| |
| [37m@retry[39;49;00m.with_exponential_backoff([90m[39;49;00m |
| num_retries=MAX_RETRIES,[90m[39;49;00m |
| retry_filter=retry.retry_on_server_errors_and_timeout_filter)[90m[39;49;00m |
| [94mdef[39;49;00m [92mget_or_create_dataset[39;49;00m([90m[39;49;00m |
| [96mself[39;49;00m, project_id, dataset_id, location=[94mNone[39;49;00m, labels=[94mNone[39;49;00m):[90m[39;49;00m |
| [90m# Check if dataset already exists otherwise create it[39;49;00m[90m[39;49;00m |
| [94mtry[39;49;00m:[90m[39;49;00m |
| > dataset = [96mself[39;49;00m.client.datasets.Get([90m[39;49;00m |
| bigquery.BigqueryDatasetsGetRequest([90m[39;49;00m |
| projectId=project_id, datasetId=dataset_id))[90m[39;49;00m |
| |
| [1m[31mapache_beam/io/gcp/bigquery_tools.py[0m:809:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2660973581
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2707505422
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2719946981
Job has been stable for 1 day. Closing as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3144497680
The issue appears to be related to the Redis test container (especially for Python 3.9).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3617541478
> Execution failed for task ':sdks:python:test-suites:dataflow:py310:mockAPITests'.
This test only runs on Python 3.10. It also fails on release-2.70 branch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3617420639
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3707601096
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3606893429
Culprit: #36773
:hdfsIntegrationTest
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4185883121
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045504666
hey, yeah i think #39284 is related.
after that PR bumped kafka to 3.9.2 and swapped `KafkaServerStartable` -> `KafkaServer`, the local test broker in `LocalKafka` still uses the old `port` / `log.dir` settings. those don't actually bind a listener on kafka 3.x anymore, so the broker never comes up.
in the postcommit logs you see stuff like:
```
Connection to node -1 (localhost/127.0.0.1:...) could not be established. Node may not be available.
```
and then it surfaces as the topic metadata timeout (`Topic ... not present in metadata after 60000 ms`), but that's just the follow-on — the real issue is the broker isn't reachable.
so not a missing topic / test flake in the usual sense. looks like leftover config from the kafka 3.9 upgrade.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5043067193
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2322066136
Currently failing test:
gradlew :sdks:python:test-suites:portable:py312:portableLocalRunnerJuliaSetWithSetupPy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2451983092
Failure in the 3.9 postcommit is apache_beam/examples/fastavro_it_test.py::FastavroIT::test_avro_it, will dive deeper into that shortly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2134154475
A new flaky test in py39:
https://ge.apache.org/s/hb7syztoolfhu/console-log?page=17
```
=================================== FAILURES =================================== |
-- | --
| [31m[1m_______________ BigQueryQueryToTableIT.test_big_query_legacy_sql _______________[0m |
| [gw3] linux -- Python 3.9.19 /runner/_work/beam/beam/build/gradleenv/1398941893/bin/python3.9 |
| |
| self = <apache_beam.io.gcp.big_query_query_to_table_it_test.BigQueryQueryToTableIT testMethod=test_big_query_legacy_sql> |
| |
| [37m@pytest[39;49;00m.mark.it_postcommit[90m[39;49;00m |
| [94mdef[39;49;00m [92mtest_big_query_legacy_sql[39;49;00m([96mself[39;49;00m):[90m[39;49;00m |
| verify_query = DIALECT_OUTPUT_VERIFY_QUERY % [96mself[39;49;00m.output_table[90m[39;49;00m |
| expected_checksum = test_utils.compute_hash(DIALECT_OUTPUT_EXPECTED)[90m[39;49;00m |
| pipeline_verifiers = [[90m[39;49;00m |
| PipelineStateMatcher(),[90m[39;49;00m |
| BigqueryMatcher([90m[39;49;00m |
| project=[96mself[39;49;00m.project,[90m[39;49;00m |
| query=verify_query,[90m[39;49;00m |
| checksum=expected_checksum)[90m[39;49;00m |
| ][90m[39;49;00m |
| [90m[39;49;00m |
| extra_opts = {[90m[39;49;00m |
| [33m'[39;49;00m[33mquery[39;49;00m[33m'[39;49;00m: LEGACY_QUERY,[90m[39;49;00m |
| [33m'[39;49;00m[33moutput[39;49;00m[33m'[39;49;00m: [96mself[39;49;00m.output_table,[90m[39;49;00m |
| [33m'[39;49;00m[33moutput_schema[39;49;00m[33m'[39;49;00m: DIALECT_OUTPUT_SCHEMA,[90m[39;49;00m |
| [33m'[39;49;00m[33muse_standard_sql[39;49;00m[33m'[39;49;00m: [94mFalse[39;49;00m,[90m[39;49;00m |
| [33m'[39;49;00m[33mwait_until_finish_duration[39;49;00m[33m'[39;49;00m: WAIT_UNTIL_FINISH_DURATION_MS,[90m[39;49;00m |
| [33m'[39;49;00m[33mon_success_matcher[39;49;00m[33m'[39;49;00m: all_of(*pipeline_verifiers),[90m[39;49;00m |
| }[90m[39;49;00m |
| options = [96mself[39;49;00m.test_pipeline.get_full_options_as_args(**extra_opts)[90m[39;49;00m |
| > big_query_query_to_table_pipeline.run_bq_pipeline(options)[90m[39;49;00m |
| |
| [1m[31mapache_beam/io/gcp/big_query_query_to_table_it_test.py[0m:178: |
| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
| [1m[31mapache_beam/io/gcp/big_query_query_to_table_pipeline.py[0m:103: in run_bq_pipeline |
| result = p.run()[90m[39;49;00m |
| [1m[31mapache_beam/testing/test_pipeline.py[0m:115: in run |
| result = [96msuper[39;49;00m().run([90m[39;49;00m |
| [1m[31mapache_beam/pipeline.py[0m:560: in run |
| [94mreturn[39;49;00m Pipeline.from_runner_api([90m[39;49;00m |
| [1m[31mapache_beam/pipeline.py[0m:587: in run |
| [94mreturn[39;49;00m [96mself[39;49;00m.runner.run_pipeline([96mself[39;49;00m, [96mself[39;49;00m._options)[90m[39;49;00m |
| [1m[31mapache_beam/runners/direct/test_direct_runner.py[0m:42: in run_pipeline |
| [96mself[39;49;00m.result = [96msuper[39;49;00m().run_pipeline(pipeline, options)[90m[39;49;00m |
| [1m[31mapache_beam/runners/direct/direct_runner.py[0m:117: in run_pipeline |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mportability[39;49;00m[04m[96m.[39;49;00m[04m[96mfn_api_runner[39;49;00m [94mimport[39;49;00m fn_runner[90m[39;49;00m |
| [1m[31mapache_beam/runners/portability/fn_api_runner/__init__.py[0m:18: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mportability[39;49;00m[04m[96m.[39;49;00m[04m[96mfn_api_runner[39;49;00m[04m[96m.[39;49;00m[04m[96mfn_runner[39;49;00m [94mimport[39;49;00m FnApiRunner[90m[39;49;00m |
| [1m[31mapache_beam/runners/portability/fn_api_runner/fn_runner.py[0m:68: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mportability[39;49;00m[04m[96m.[39;49;00m[04m[96mfn_api_runner[39;49;00m [94mimport[39;49;00m execution[90m[39;49;00m |
| [1m[31mapache_beam/runners/portability/fn_api_runner/execution.py[0m:62: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mportability[39;49;00m[04m[96m.[39;49;00m[04m[96mfn_api_runner[39;49;00m [94mimport[39;49;00m translations[90m[39;49;00m |
| [1m[31mapache_beam/runners/portability/fn_api_runner/translations.py[0m:55: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mworker[39;49;00m [94mimport[39;49;00m bundle_processor[90m[39;49;00m |
| [1m[31mapache_beam/runners/worker/bundle_processor.py[0m:69: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mworker[39;49;00m [94mimport[39;49;00m operations[90m[39;49;00m |
| _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ |
| |
| > [04m[91m?[39;49;00m[04m[91m?[39;49;00m[04m[91m?[39;49;00m[90m[39;49;00m |
| [1m[31mE KeyError: '__pyx_vtable__'[0m |
| |
| [1m[31mapache_beam/runners/worker/operations.py[0m:1: KeyError
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2176255793
```
[31m[1m_______ ERROR collecting apache_beam/runners/worker/log_handler_test.py ________[0m |
-- | --
| [1m[31mapache_beam/runners/worker/log_handler_test.py[0m:34: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mworker[39;49;00m [94mimport[39;49;00m bundle_processor[90m[39;49;00m |
| [1m[31mapache_beam/runners/worker/bundle_processor.py[0m:69: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mworker[39;49;00m [94mimport[39;49;00m operations[90m[39;49;00m |
| [1m[31mapache_beam/runners/worker/operations.py[0m:1: in init apache_beam.runners.worker.operations |
| [04m[91m?[39;49;00m[04m[91m?[39;49;00m[04m[91m?[39;49;00m[90m[39;49;00m |
| [1m[31mE KeyError: '__pyx_vtable__'[0m |
| [31m[1m________ ERROR collecting apache_beam/runners/worker/opcounters_test.py ________[0m |
| [1m[31mapache_beam/runners/worker/opcounters_test.py[0m:27: in <module> |
| [94mfrom[39;49;00m [04m[96mapache_beam[39;49;00m[04m[96m.[39;49;00m[04m[96mrunners[39;49;00m[04m[96m.[39;49;00m[04m[96mworker[39;49;00m [94mimport[39;49;00m opcounters[90m[39;49;00m |
| [1m[31mapache_beam/runners/worker/opcounters.py[0m:1: in init apache_beam.runners.worker.opcounters |
| [04m[91m?[39;49;00m[04m[91m?[39;49;00m[04m[91m?[39;49;00m[90m[39;49;00m |
| [1m[31mE ValueError: apache_beam.utils.counters.Counter size changed, may indicate binary incompatibility. Expected 56 from C header, got 32 from PyObject[0m
```
https://ge.apache.org/s/w6kem3hrdnwii/console-log/task/:sdks:python:test-suites:direct:py38:tensorflowInferenceTest?anchor=1334&page=2
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-1989422230
I see. It was red for the last two weeks and flaky before that too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2082920470
Could make sense to find a way to get separate top-level signal for Python versions, assuming we can use software engineering to share everything necessary so they don't get out of sync.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3406903339
Thanks, I hadn't seen this and will take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3627822926
> Its [#36772](https://github.com/apache/beam/pull/36772) (sort of).
>
> ```
> Execution failed for task ':sdks:python:test-suites:dataflow:py310:mockAPITests'.
> > Process 'command 'sh'' finished with non-zero exit value 5
> ```
>
> This means no tests ran (
>
> [beam/sdks/python/scripts/run_pytest.sh](https://github.com/apache/beam/blob/32345e7cbe99084ec3c072dc628c1aed70ddc9a8/sdks/python/scripts/run_pytest.sh#L144)
>
> Line 144 in [32345e7](/apache/beam/commit/32345e7cbe99084ec3c072dc628c1aed70ddc9a8)
>
> # Exit with error if no tests were run in either suite (status code 5).
> ). If I try running this task, I see:
> ```
> SKIPPED [1] apache_beam/examples/ml_transform/ml_transform_it_test.py:34: unittest.case.SkipTest: tensorflow_transform is not installed.
> SKIPPED [1] apache_beam/transforms/enrichment_handlers/cloudsql_it_test.py:59: unittest.case.SkipTest: CloudSQL dependencies not installed: No module named 'redis'
> SKIPPED [1] apache_beam/examples/snippets/transforms/elementwise/mltransform_test.py:36: unittest.case.SkipTest: tensorflow_transform is not installed.
> SKIPPED [1] apache_beam/examples/snippets/transforms/elementwise/runinference_test.py:38: unittest.case.SkipTest: PyTorch dependencies are not installed
> SKIPPED [1] apache_beam/io/requestresponse_it_test.py:45: unittest.case.SkipTest: RequestResponseIO dependencies are not installed.
> SKIPPED [1] apache_beam/ml/anomaly/detectors/pyod_adapter_test.py:44: unittest.case.SkipTest: PyOD dependencies are not installed
> SKIPPED [1] apache_beam/ml/inference/gemini_inference_it_test.py:35: unittest.case.SkipTest: Gemini model handler dependencies are not installed
> SKIPPED [1] apache_beam/ml/inference/gemini_inference_test.py:29: unittest.case.SkipTest: Gemini dependencies are not installed
> SKIPPED [1] apache_beam/ml/inference/huggingface_inference_it_test.py:34: unittest.case.SkipTest: transformers dependencies are not installed. Check if transformers, torch, and tensorflow is installed.
> SKIPPED [1] apache_beam/ml/inference/tensorflow_inference_test.py:52: unittest.case.SkipTest: Tensorflow dependencies are not installed. Make sure you have both tensorflow and tensorflow_hub installed.
> SKIPPED [1] apache_beam/ml/inference/onnx_inference_test.py:35: unittest.case.SkipTest: TODO: fix https://github.com/apache/beam/issues/31254
> SKIPPED [1] apache_beam/ml/inference/pytorch_inference_test.py:49: unittest.case.SkipTest: PyTorch dependencies are not installed
> SKIPPED [1] apache_beam/ml/inference/tensorrt_inference_test.py:41: unittest.case.SkipTest: TensorRT dependencies are not installed
> SKIPPED [1] apache_beam/ml/inference/vertex_ai_inference_it_test.py:33: unittest.case.SkipTest: Vertex AI model handler dependencies are not installed
> SKIPPED [1] apache_beam/ml/inference/xgboost_inference_test.py:46: unittest.case.SkipTest: XGBoost dependencies are not installed
> SKIPPED [1] apache_beam/ml/transforms/handlers_test.py:50: unittest.case.SkipTest: tensorflow_transform is not installed.
> SKIPPED [1] apache_beam/ml/transforms/tft_test.py:40: unittest.case.SkipTest: tensorflow_transform is not installed.
> SKIPPED [1] apache_beam/runners/dask/dask_runner_test.py:36: unittest.case.SkipTest: Dask must be installed to run tests.
> SKIPPED [1] apache_beam/testing/analyzers/perf_analysis_test.py:47: unittest.case.SkipTest: Missing dependencies to run perf analysis tests.
> SKIPPED [1] apache_beam/testing/benchmarks/cloudml/cloudml_benchmark_test.py:31: unittest.case.SkipTest: Dependencies are not installed
> SKIPPED [1] apache_beam/transforms/enrichment_handlers/bigquery_it_test.py:42: unittest.case.SkipTest: Google Cloud BigQuery dependencies are not installed.
> SKIPPED [1] apache_beam/transforms/enrichment_handlers/bigtable_it_test.py:42: unittest.case.SkipTest: Bigtable test dependencies are not installed.
> SKIPPED [1] apache_beam/transforms/enrichment_handlers/feast_feature_store_it_test.py:40: unittest.case.SkipTest: Feast feature store test dependencies are not installed.
> SKIPPED [1] apache_beam/transforms/enrichment_handlers/feast_feature_store_test.py:25: unittest.case.SkipTest: Feast feature store test dependencies are not installed.
> SKIPPED [1] apache_beam/transforms/enrichment_handlers/vertex_ai_feature_store_it_test.py:39: unittest.case.SkipTest: VertexAI Feature Store test dependencies are not installed.
> SKIPPED [1] apache_beam/typehints/pytorch_type_compatibility_test.py:38: unittest.case.SkipTest: PyTorch dependencies are not installed
> SKIPPED [1] apache_beam/yaml/yaml_ml_test.py:31: unittest.case.SkipTest: tensorflow_transform is not installed.
> ```
>
> So we're now skipping tests that should have run, but we were also skipping others before (e.g.
>
> [beam/sdks/python/apache_beam/io/requestresponse_it_test.py](https://github.com/apache/beam/blob/32345e7cbe99084ec3c072dc628c1aed70ddc9a8/sdks/python/apache_beam/io/requestresponse_it_test.py#L142)
>
> Line 142 in [32345e7](/apache/beam/commit/32345e7cbe99084ec3c072dc628c1aed70ddc9a8)
>
>
> )
Yes, confirmed that it is #36772
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3433751816
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3665869573
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4783782607
Trying to isolate the stuck tasks:
- With only `:sdks:python:test-suites:direct:py312:postCommitIT` and `:sdks:python:test-suites:direct:py312:hdfsIntegrationTest`, the job can finished successfully: https://github.com/apache/beam/actions/runs/28058482799/job/83066687144
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5307379253
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5603760626
@Abacn @shunping this is also the 2.44 and 2.42 DirectRow mismatch, will be fixed by #40042 and the new beam-master image
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2455005910
> Looks like after sickbaying TensorRT tests, there are still failures. https://ge.apache.org/s/27igat7sfmcsu/console-log/task/:sdks:python:test-suites:portable:py310:portableWordCountSparkRunnerBatch?anchor=60&page=1 is an example, it looks like we're failing because we're missing a class in the spark runner.
>
> @Abacn would you mind taking a look? Its unclear why this is happening now, but I'm guessing it may be related to #32976 (and maybe some caching kept it from showing up?)
It's bad gradle cache. Cannot reproduce locally on master branch. Also inspected the expansion jar.
For some reason, recently, Gradle cache for shadowJar breaks more frequently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2134151500
The Vertex AI package version issue:
```
../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
-- | --
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33 |
| /runner/_work/beam/beam/build/gradleenv/-1734967050/lib/python3.9/site-packages/vertexai/preview/developer/__init__.py:33: DeprecationWarning: |
| After May 30, 2024, importing any code below will result in an error. |
| Please verify that you are explicitly pinning to a version of `google-cloud-aiplatform` |
| (e.g., google-cloud-aiplatform==[1.32.0, 1.49.0]) if you need to continue using this |
| library. |
| |
| from vertexai.preview import ( |
| init, |
| remote, |
| VertexModel, |
| register, |
| from_pretrained, |
| developer, |
| hyperparameter_tuning, |
| tabular_models, |
| ) |
|
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2082763155
Yea, let's work out how to get top-level signal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2082896879
The lowest and highest Python version (3.8, 3.11) are running more tests than (3.9, 3.10), could be those tests or task permared
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2667831441
Possible cause: https://github.com/apache/beam/pull/33943.
I'm testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3956661256
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3827425174
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3627081994
I'll put up a pr in a moment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3627167868
Actually, I think all the skipped tests come from redis.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3424789468
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3435380914
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "stankiewicz (via GitHub)" <gi...@apache.org>.
stankiewicz commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045409680
I'm curious if not related - https://github.com/apache/beam/pull/39284
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "sjvanrossum (via GitHub)" <gi...@apache.org>.
sjvanrossum commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045605559
Checking, thanks for flagging me here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4800940578
I am trying to revert this PR (#39106) and run the post commit tests (https://github.com/apache/beam/actions/runs/28180238767/job/83467598018?pr=39106).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4791174377
I just finished running the workflow to publish vllm image(https://github.com/apache/beam/actions/runs/28106876263), and now I am running postcommit python 3.12 on the previously stuck gradle task (https://github.com/apache/beam/actions/runs/28111217064).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4784859275
@damccorm @Amar3tto: do you know how `us.gcr.io/apache-beam-testing/python-postcommit-it/vllm:latest` was built? I see the comment below:
https://github.com/apache/beam/blob/f29d9af139a3c0ccf1ce33a43485bc140bb55e0c/sdks/python/test-suites/dataflow/common.gradle#L461-L465
And I also see there are two dockerfiles in the folder:
https://github.com/apache/beam/tree/master/sdks/python/apache_beam/ml/inference/test_resources
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5781534525
The test `LeaderBoardIT.test_leader_board_it ` has been flaky recently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2993847528
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2921819324
Job has been stable for 1 day. Closing as resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2452041369
The problem in the TensorRT container is that we seem to have two different versions of CUDA installed, one at version 11.8 and the other at 12.1 (we want everything at 12.1)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2451953053
This is red again - https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python.yml?query=branch%3Amaster
It looks like there are currently 2 issues:
1. Python 3.9 job is failing, I think probably because of the mypy changes. [example failure](https://ge.apache.org/s/yjfdaekyijkum)
2. The TensorRT tests are failing. Originally, they were failing because of a mismatch between container/local python versions, but now they seem to be running into CUDA issues with the new container. [example failure](https://ge.apache.org/s/xlkjjsw6za5zq/console-log/task/:sdks:python:test-suites:dataflow:py310:tensorRTtests?anchor=38&page=1) and corresponding failing [Dataflow job](https://console.cloud.google.com/dataflow/jobs/us-central1/2024-10-31_22_53_13-11627397312971955153;bottomTab=WORKER_LOGS;expandBottomPanel=true;logsSeverity=ERROR;graphView=0?project=apache-beam-testing&pageState=(%22dfTime%22:(%22l%22:%22dfJobMaxTime%22))&e=13802955&mods=dataflow_dev)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2469139990
Thanks for taking care of it. I am +1 for rollback. The first distroless PR was expected to be a no-op for 2.61.0 release. Good to know it broke something before release cut.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2717242544
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2614195568
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2662795769
`Unable to pull image apache/beam_python3.9_sdk:2.64.0.dev, defaulting to local image if it exists`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping closed issue #30513: The PostCommit Python job is flaky
URL: https://github.com/apache/beam/issues/30513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-4777543356
Updating base image dependencies at #39070 to include pandas 2.3.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-3384234385
`[31mFAILED[0m apache_beam/io/gcp/bigquery_write_it_test.py::BigQueryWriteIntegrationTests::test_big_query_write_temp_table_append_schema_update_2 `
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2160912953
I looked at a couple flakes and could not discern if they represented anything that should be release blocking, so I am moving this to the next release milestone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2082933487
Yeah, we used to have this for Jenkins where each Python PostCommit had its own task
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-2137574553
Last three runs are green now.
<img width="1339" alt="image" src="https://github.com/apache/beam/assets/7833268/cce6d6cb-f858-457c-9835-c017e14f0953">
Close this for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5059298240
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5045308280
hey, i dug into the latest postcommit failures a bit and think i found what's going on.
looking at recent runs (e.g. https://github.com/apache/beam/actions/runs/29896319687), the job is failing on the cross-language KafkaIO tests with:
```
TimeoutException: Topic xlang_kafkaio_test_... not present in metadata after 60000 ms.
```
so it looks like the Kafka producer can't fetch topic metadata in time (hits the default 60s timeout), which then blows up `KafkaWriter` and takes down the Flink job.
feels more like a test infra / Kafka readiness timing issue than an actual Beam code bug. not sure yet what the best fix is — bumping producer timeouts, waiting for Kafka to be ready before the test, making sure topics get created, etc. — happy to help chase that down if someone has a preferred approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5795669787
@shunping I looked at a few recent PostCommit Python runs. Same failure: empty checksum da39a3ee..., and the BQ query for total_score=5000 returns 0 rows aslo the Timing out on waiting for job after around 600s part seems normal for this streaming IT and what stands out is that it fails on different SHAs, and the same SHA can fail once then pass later so I’m thinking toward a race on the BQ side rather than a recent commit breaking the pipeline
Also the LeaderBoardIT writes with streaming inserts but the matchers are created without timeout_secs, so they only query once after the wait. If the rows arent visible yet, we get exactly this empty checksum
I was thinking we could pass timeout_secs (maybe 5 min) on both users/teams matchers in leader_board_it_test.py keep the same query and expected checksum, and let it retry until the data shows up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5314609254
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "Amar3tto (via GitHub)" <gi...@apache.org>.
Amar3tto commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5393897306
Stabilized
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5382005607
Reopening since the workflow is still flaky
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] The PostCommit Python job is flaky [beam]
Posted by "aIbrahiim (via GitHub)" <gi...@apache.org>.
aIbrahiim commented on issue #30513:
URL: https://github.com/apache/beam/issues/30513#issuecomment-5874855709
> [@aIbrahiim](https://github.com/aIbrahiim): have you experimented on your idea in [#30513 (comment)](https://github.com/apache/beam/issues/30513#issuecomment-5795669787)? `LeaderBoardIT.test_leader_board_it` is still flaky.
@shunping yes I tried that approach in https://github.com/apache/beam/pull/40241
What I did there:
1. timeout_secs on both BigQuery matchers (poll instead of one-shot)
2. wait on Pub/Sub publish futures before asserting
On the PR PostCommit runs, LeaderBoardIT passed on 3.10 / 3.11 / 3.12, but we still saw at least one empty-checksum fail (e.g. 3.14), so it looks reduced, not fully gone
I wasn’t able to dig further at the time because PostCommit was also hitting IAM permission issues (403 PermissionDenied on GCP APIs like Vertex AI / Recommendations AI, and earlier some Pub/Sub / BigQuery 403s), which made the suite red for unrelated reasons.
I’ll retrigger PostCommit on that PR and check the latest results
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org