You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tathagata Das (JIRA)" <ji...@apache.org> on 2016/12/15 21:18:58 UTC
[jira] [Resolved] (SPARK-18826) Make FileStream be able to start
with most recent files
[ https://issues.apache.org/jira/browse/SPARK-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tathagata Das resolved SPARK-18826.
-----------------------------------
Resolution: Fixed
Fix Version/s: 2.1.0
Issue resolved by pull request 16251
[https://github.com/apache/spark/pull/16251]
> Make FileStream be able to start with most recent files
> -------------------------------------------------------
>
> Key: SPARK-18826
> URL: https://issues.apache.org/jira/browse/SPARK-18826
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Reporter: Shixiong Zhu
> Assignee: Shixiong Zhu
> Fix For: 2.1.0
>
>
> When starting a stream with a lot of backfill and maxFilesPerTrigger, the user could often want to start with most recent files first. This would let you keep low latency for recent data and slowly backfill historical data.
> It's better to add an option to control this behavior.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org