You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/11/09 17:58:02 UTC
[GitHub] [beam] johnjcasey commented on issue #22809: [Bug]: Python SDK gets stuck when using Unbounded PCollection in streaming mode on GroupByKey after ReadFromKafka on DirectRunner, FlinkRunner and DataflowRunner
johnjcasey commented on issue #22809:
URL: https://github.com/apache/beam/issues/22809#issuecomment-1309137732
Ok, I've dug into this quite a bit:
1. It appears to be unrelated to underlying use of Kafka UnboundedSource v. Kafka SDF
2. Data flows normally with no window+gbk
3. with
| "Fixed window 5s" >> beam.WindowInto(window.FixedWindows(5))
| "Group by key" >> beam.GroupByKey()
data does not appear to flow normally, but will show up on a drain request to dataflow
4. with
| "Fixed window 5s" >> beam.WindowInto(window.FixedWindows(5), trigger=Repeatedly(AfterProcessingTime()), accumulation_mode=AccumulationMode.ACCUMULATING)
| "Group by key" >> beam.GroupByKey()
Data appears to flow normally. This suggests there is some issue triggering in the default window, which may or may not be related to xlang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org