You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "LvDongrong (JIRA)" <ji...@apache.org> on 2017/05/03 03:07:04 UTC
[jira] [Created] (SPARK-20572) Spark Streaming fail to read file on
Hdfs
LvDongrong created SPARK-20572:
----------------------------------
Summary: Spark Streaming fail to read file on Hdfs
Key: SPARK-20572
URL: https://issues.apache.org/jira/browse/SPARK-20572
Project: Spark
Issue Type: Bug
Components: DStreams
Affects Versions: 2.1.0
Reporter: LvDongrong
When I move a file on hdfs to the target directory where Spark Streaming read from, Spark Streaming fail to read the file. Then I found the Spark Streaming Hdfs interface only read the file whose modtime is in the interval of the current batch,but when we move the file to the target directory,the modtime of the file is never changed. May be the method to find new file is not appropriate.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org