[GitHub] [beam] nehsyc commented on pull request #15256: Revert "[BEAM-11934] Remove Dataflow override of streaming WriteFiles with runner determined sharding"

[GitBox <gi...@apache.org>]

nehsyc commented on pull request #15256:
URL: https://github.com/apache/beam/pull/15256#issuecomment-891195968


   If I understand it correctly the flakiness was caused by running a streaming pipeline on a bounded source in the word count pipeline. Note that the implementation for runner determined sharding in the `WriteFiles` is different for bounded and unbounded source. So the override was only used by bounded source in streaming and without the override the word count pipeline unexpectedly picked up the implementation for batch.
   
   Perhaps a better fix might be to modify the override to also check the bounded-ness of the input so unbounded datacan still adopt proper runner determined sharding implementation for streaming.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org