You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Chamikara Jayalath (JIRA)" <ji...@apache.org> on 2019/04/16 17:29:00 UTC
[jira] [Commented] (BEAM-7063) beam_PostCommit_Python_Verify is
consistently failing due to a sideinput materialization related issue
[ https://issues.apache.org/jira/browse/BEAM-7063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16819309#comment-16819309 ]
Chamikara Jayalath commented on BEAM-7063:
------------------------------------------
Looks like one test of postCommit is still failing.
[https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PostCommit_Python3_Verify/562/]
[https://scans.gradle.com/s/2jqz3qlygliv4/console-log?task=:beam-sdks-python-test-suites-dataflow-py3:postCommitIT#L29]
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
java.lang.IllegalArgumentException: This handler is only capable of dealing with urn:beam:sideinput:materialization:multimap:0.1 materializations but was asked to handle beam:side_input:multimap:v1 for PCollectionView with tag side0-write/Write/WriteImpl/WriteBundles.
at org.apache.beam.vendor.guava.v20_0.com.google.common.base.Preconditions.checkArgument(Preconditions.java:399)
at org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.transformSideInputForRunner(RegisterNodeFunc
tion.java:506)
at org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:327)
at org.apache.beam.runners.dataflow.worker.graph.RegisterNodeFunction.apply(RegisterNodeFunction.java:97)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:208)
at org.apache.beam.runners.dataflow.worker.graph.CreateRegisterFnOperationFunction.apply(CreateRegisterFnOperationFunction.java:75)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at java.util.function.Function.lambda$andThen$1(Function.java:88)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:347)
at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
> beam_PostCommit_Python_Verify is consistently failing due to a sideinput materialization related issue
> ------------------------------------------------------------------------------------------------------
>
> Key: BEAM-7063
> URL: https://issues.apache.org/jira/browse/BEAM-7063
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, test-failures
> Reporter: Chamikara Jayalath
> Assignee: Luke Cwik
> Priority: Critical
> Fix For: 2.13.0
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Seems like this was the first post-commit failure: [https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PostCommit_Python_Verify/7855/]
> (the one before this also failed but that looks like a flake)
>
>
> Looking at failed tests I noticed that some of the were due to failed Dataflow jobs. I noticed three issues.
> (1) Error in Dataflow jobs "This handler is only capable of dealing with urn:beam:sideinput:materialization:multimap:0.1 materializations but was asked to handle beam:side_input:multimap:v1 for PCollectionView with tag side0-FilterOutSpammers."
> [https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_17_57_01-16397932959358575026?project=apache-beam-testing]
> [https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_07_11-1342017143600540529?project=apache-beam-testing]
>
> (2) BigQuery job failures (possibly this is transient)
> [https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-04-08_18_25_22-16437560120793921616?project=apache-beam-testing]
>
> (3) Some batch Dataflow jobs timed out and were cancelled (possibly also transient).
>
> (1) is probably due to [https://github.com/apache/beam/commit/684f8130284a7c7979773300d04e5473ca0ac8f3]
>
> cc: [~altay] [~lostluck] [~alanmyrvold]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)