You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/06/04 22:30:33 UTC

[GitHub] [beam] damccorm opened a new issue, #21260: Python DirectRunner does not emit data at GC time

damccorm opened a new issue, #21260:
URL: https://github.com/apache/beam/issues/21260

   The following should succeed but does not:
   ```
   
   test_options = PipelineOptions(flags=['--allow_unsafe_triggers'])
    with TestPipeline(options=test_options)
   as pipeline:
     pcoll = (
       pipeline
       | beam.Create([(1, 1), (1, 2), (1, 3), (1, 4)])
       |
   WindowInto(
         window.GlobalWindows(),
         trigger=trigger.AfterCount(5),
         accumulation_mode=trigger.AccumulationMode.ACCUMULATING)
    
     | beam.GroupByKey())
     assert_that(pcoll, equal_to([(1, [1, 2, 3, 4])]))
   ```
   
   However, it  currently fails, because  pcoll will be empty. It appears that the Direct Runner drops data if the trigger never fired.
   
   Imported from Jira [BEAM-13078](https://issues.apache.org/jira/browse/BEAM-13078). Original Jira may contain additional context.
   Reported by: zhoufek.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] kennknowles commented on issue #21260: Python DirectRunner does not emit data at GC time

Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #21260:
URL: https://github.com/apache/beam/issues/21260#issuecomment-1246018323

   @zhoufek are you aware of any follow-up to this? If not, this is certainly an issue. It would be a direct runner issue, not a Python SDK issue, I think. Dataflow has one implementation of triggers while Java-based runners have another, and neither is shared with the local Python runner.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org