[GitHub] [beam] damccorm opened a new issue, #20850: GroupByKey doesn't seem to work with FixedWindows in DirectRunner

[GitBox <gi...@apache.org>]

damccorm opened a new issue, #20850:
URL: https://github.com/apache/beam/issues/20850

   After applying `FixedWindows` on a streaming source, a `GroupByKey` operation won't emit keyed elements in a window. 
   
   This example without `GroupByKey` prints all the windowed elements:
   
    
   ```
   
   pipeline
    .apply("ReadFromPubsub", PubsubIO.readStrings().fromSubscription(subscriptionPath))
    .apply(Window.into(FixedWindows.of(Duration.standardSeconds(5L))))
   
   .apply(WithKeys.of("bobcat"))
    .apply(MapElements.into(TypeDescriptors.nulls()).via(
        (KV<String,
   String> pair) -> {
            LOG.info("Key: " + pair.getKey() + "\tValue: " + pair.getValue());
     
         return null;
        }
    ));
   ```
   
    
   
   This example with `GroupByKey` doesn't emit anything:
   
    
   ```
   
   pipeline
    .apply("ReadFromPubsub", PubsubIO.readStrings().fromSubscription(subscriptionPath))
    .apply(Window.into(FixedWindows.of(Duration.standardSeconds(5L))))
   
   .apply(WithKeys.of("bobcat"))
    .apply(GroupByKey.create())
    .apply(FlatMapElements.into(TypeDescriptors.nulls()).via(
   
       (KV<String, Iterable<String>> pair) -> {
            pair.getValue().forEach(message -> LOG.info("Message:
   " + message));
            return null;
        }
    ));
   ```
   
    
   
   I'm using DirectRunner. The same logic works for Python using both the DirectRunner and DataflowRunner.
   
   Imported from Jira [BEAM-12075](https://issues.apache.org/jira/browse/BEAM-12075). Original Jira may contain additional context.
   Reported by: tianzi.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] licongchao commented on issue #20850: GroupByKey doesn't seem to work with FixedWindows in DirectRunner

["licongchao (via GitHub)" <gi...@apache.org>]

licongchao commented on issue #20850:
URL: https://github.com/apache/beam/issues/20850#issuecomment-1408124129

   Any updates about this issue? I've tried with DirectRunner and FlinkRunner, also tried adding TimestampedValue in data and tried adding  "trigger.Repeatedly" in window parameters, cannot make it working either. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org