[jira] [Commented] (BEAM-11066) Java wordcount example throw exception on dataflow runner with streaming beam_fn_api

["Kenneth Knowles (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BEAM-11066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17214872#comment-17214872 ] 

Kenneth Knowles commented on BEAM-11066:
----------------------------------------

Commented on the PR: "streaming mode" is a specific concept for some runners, but is not a concept for a pipeline. This looks like a bug in the DataflowRunner overrides.

> Java wordcount example throw exception on dataflow runner with streaming beam_fn_api
> ------------------------------------------------------------------------------------
>
>                 Key: BEAM-11066
>                 URL: https://issues.apache.org/jira/browse/BEAM-11066
>             Project: Beam
>          Issue Type: Bug
>          Components: examples-java
>            Reporter: Yichi Zhang
>            Priority: P2
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Looks like the TextIO.write() is not properly writing windowed output in streaming mode. 
>  
> {{Exception in thread "main" java.lang.IllegalStateException: Found nodes that matched overrides. Matches: \{Node{fullName=WriteCounts/WriteFiles/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values}=[PTransformOverride\{matcher=EqualClassPTransformMatcher{class=class org.apache.beam.sdk.transforms.Create$Values}, overrideFactory=org.apache.beam.runners.dataflow.DataflowRunner$StreamingFnApiCreateOverrideFactory@62923ee6}]}
> 	at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:588)
> 	at org.apache.beam.sdk.Pipeline$1.leaveCompositeTransform(Pipeline.java:237)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:661)
> 	at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
> 	at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
> 	at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:463)
> 	at org.apache.beam.sdk.Pipeline.checkNoMoreMatches(Pipeline.java:218)
> 	at org.apache.beam.sdk.Pipeline.replaceAll(Pipeline.java:214)
> 	at org.apache.beam.runners.dataflow.DataflowRunner.replaceTransforms(DataflowRunner.java:1180)
> 	at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:871)
> 	at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:192)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
> 	at org.apache.beam.sdk.Pipeline.run(Pipeline.java:303)
> 	at org.apache.beam.examples.WordCount.runWordCount(WordCount.java:185)
> 	at org.apache.beam.examples.WordCount.main(WordCount.java:192)}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)