You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Daniel Collins (Jira)" <ji...@apache.org> on 2021/09/09 20:54:00 UTC

[jira] [Updated] (BEAM-12867) Either Create or DirectRunner fails to produce all elements to the following transform

     [ https://issues.apache.org/jira/browse/BEAM-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Collins updated BEAM-12867:
----------------------------------
    Issue Type: Bug  (was: Improvement)

> Either Create or DirectRunner fails to produce all elements to the following transform
> --------------------------------------------------------------------------------------
>
>                 Key: BEAM-12867
>                 URL: https://issues.apache.org/jira/browse/BEAM-12867
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-direct
>            Reporter: Daniel Collins
>            Priority: P1
>
> The following pipeline fails to print out all numbers 1 to 100 when run on DirectRunner in streaming mode.
> This was identified implementing org.apache.beam.sdk.io.gcp.pubsublite.ReadWriteIT, which uses a workaround for the bug in either Create or DirectRunner:
> ```
> private static final int COUNT = 100;
> private static AtomicInteger CREATED_COUNT = new AtomicInteger();
> public static void run(Pipeline pipeline) {
>   
> PCollection<Integer> indexes =
>  pipeline.apply(
>  "createIndexes",
>  Create.of(IntStream.range(0, COUNT).boxed().collect(Collectors.toList())));
> indexes.apply(
>  "createMessages",
>  MapElements.via(
>  new SimpleFunction<Integer, Integer>(
>  index -> {
>  System.err.println("Created message index " + createdCount.incrementAndGet());
> return index; }) {}));
> pipeline.run().waitUntilFinish();  // Never terminates
> }
> ```



--
This message was sent by Atlassian Jira
(v8.3.4#803005)