You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Ashish Soni <as...@gmail.com> on 2015/08/10 21:40:37 UTC
Java Streaming Context - File Stream use
Please help as not sure what is incorrect with below code as it gives me
complilaton error in eclipse
SparkConf sparkConf = new
SparkConf().setMaster("local[4]").setAppName("JavaDirectKafkaWordCount");
JavaStreamingContext jssc = new JavaStreamingContext(sparkConf,
Durations.seconds(2));
* jssc.fileStream("/home/", String.class, String.class,
TextInputFormat.class);*
Re: Java Streaming Context - File Stream use
Posted by Akhil Das <ak...@sigmoidanalytics.com>.
Like this: (Including the filter function)
JavaPairInputDStream<LongWritable, Text> inputStream = ssc.fileStream(
testDir.toString(),
LongWritable.class,
Text.class,
TextInputFormat.class,
new Function<Path, Boolean>() {
@Override
public Boolean call(Path v1) throws Exception {
return Boolean.TRUE;
}
},
true);
Thanks
Best Regards
On Tue, Aug 11, 2015 at 1:10 AM, Ashish Soni <as...@gmail.com> wrote:
> Please help as not sure what is incorrect with below code as it gives me
> complilaton error in eclipse
>
> SparkConf sparkConf = new
> SparkConf().setMaster("local[4]").setAppName("JavaDirectKafkaWordCount");
> JavaStreamingContext jssc = new
> JavaStreamingContext(sparkConf, Durations.seconds(2));
>
> * jssc.fileStream("/home/", String.class, String.class,
> TextInputFormat.class);*
>
>