You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Daniel Collins (Jira)" <ji...@apache.org> on 2021/09/09 20:54:00 UTC
[jira] [Updated] (BEAM-12867) Either Create or DirectRunner fails
to produce all elements to the following transform
[ https://issues.apache.org/jira/browse/BEAM-12867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Collins updated BEAM-12867:
----------------------------------
Issue Type: Bug (was: Improvement)
> Either Create or DirectRunner fails to produce all elements to the following transform
> --------------------------------------------------------------------------------------
>
> Key: BEAM-12867
> URL: https://issues.apache.org/jira/browse/BEAM-12867
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
> Reporter: Daniel Collins
> Priority: P1
>
> The following pipeline fails to print out all numbers 1 to 100 when run on DirectRunner in streaming mode.
> This was identified implementing org.apache.beam.sdk.io.gcp.pubsublite.ReadWriteIT, which uses a workaround for the bug in either Create or DirectRunner:
> ```
> private static final int COUNT = 100;
> private static AtomicInteger CREATED_COUNT = new AtomicInteger();
> public static void run(Pipeline pipeline) {
>
> PCollection<Integer> indexes =
> pipeline.apply(
> "createIndexes",
> Create.of(IntStream.range(0, COUNT).boxed().collect(Collectors.toList())));
> indexes.apply(
> "createMessages",
> MapElements.via(
> new SimpleFunction<Integer, Integer>(
> index -> {
> System.err.println("Created message index " + createdCount.incrementAndGet());
> return index; }) {}));
> pipeline.run().waitUntilFinish(); // Never terminates
> }
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)