You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Georg Heiler <ge...@gmail.com> on 2019/09/19 09:14:31 UTC

Hi,

How can I create an initial state by hands so that structured streaming
files source only reads data which is semantically (i.e. using a file path
lexicographically) greater than the minimum committed initial state?

Details here:
https://stackoverflow.com/questions/58004832/spark-structured-streaming-file-source-read-from-a-certain-partition-onwards

Best,
Georg