You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by kant kodali <ka...@gmail.com> on 2017/06/04 04:23:32 UTC

Is there a way to do partial sort in update mode in Spark Structured Streaming?

Hi All,

1. Is there a way to do partial sort of say (timestamp column) in update
mode? I am currently using Spark 2.1.1 and its looks like it is not
possible however I am wondering if this possible in 2.2?
2. can we do full sort in update mode with a specified watermark? since
after a specified watermark I assume one should be able to materialize and
sort the full batch.

Thanks!