Reading from multiple sockets

[Varad Joshi <vj...@pivotal.io>]

Still fairly new to Spark so please bear with me. I am trying to write a
streaming app that has multiple workers that read from sockets and process
the data. Here is a very simplified version of what I am trying to do:

val carStreamSeq = (1 to 2).map( _ => ssc.socketTextStream(host, port)
).toArray
val unionCarStream = ssc.union(carStreamSeq)
val connectedCars = unionCarStream.count()
connectedCars.foreachRDD(r => println("count: " + r.collect().mkString))

I can see the workers are running and the data is coming through but
anything I put inside the 'foreachRDD' does not seem to get executed. I
don't see the output of println in either stdout or in the master output
file. What am I doing wrong?

Varad



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Reading-from-multiple-sockets-tp14000.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org