[GitHub] [beam] lukecwik edited a comment on pull request #12603: [WIP][BEAM-10670] Make SparkRunner opt-out for using an SDF powered Read transform.

[GitBox <gi...@apache.org>]

lukecwik edited a comment on pull request #12603:
URL: https://github.com/apache/beam/pull/12603#issuecomment-703777328


   > > @iemejia Since streaming is effectively broken due to lack of support for watermark holds. What do you think about enabling SDF for Spark and it only working in batch?
   > 
   > Can you be more explicit on what you mean on 'it only working in batch'? Isn't it the current case for Bounded SDF?
   > Or what you mean is to have now Unbounded Reads translated using SDF still with the same level of functionality than the old translation (no holds)?
   
   BoundedSource converted to an SDF will work just as well as the current BoundedSource implementation since they don't rely on watermark holds. The current implementation of UnboundedSource and the new implementation using UnboundedSource as an SDF both set the watermark but triggers don't honor it since there is lack of support for watermark holds.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org