jssc.textFileStream(directory) how to ensure it read entire all incoming files

[unk1102 <um...@gmail.com>]

Hi my actual use case is streaming text files in HDFS directory and send it
to Kafka please let me know if is there any existing solution for this.
Anyways I have the following code 

//lets assume directory contains one file a.txt and it has 100 lines 
JavaDStream<String> logData = jssc.textFileStream(directory);

//how do I make sure jssc.textFileStream() read all 100 lines so I can
delete it later

Also please let me know how do I send above logData to Kafka using Spark
streaming



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/jssc-textFileStream-directory-how-to-ensure-it-read-entire-all-incoming-files-tp26181.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org