You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by prabeesh k <pr...@gmail.com> on 2013/11/14 06:46:06 UTC
Problem with Spark Streaming After update to spark-0.8.0 and mesos-0.14.1
After update from spark-0.7.3 and mesos-0.9.0 to spark-0.8.0 and
mesos-0.14.1.
For long running streaming project, around 5 to 6 hours I have no output
for twitter spark streaming examples. I am using some queue to take the
output as json format using spray json.
Is there any problem using spray json in spark-0.8.0.
My system setting for multi-user support
-------------------------------------------------------------
System.setProperty("spark.cores.max", "1")
System.setProperty("spark.mesos.coarse", "true")
System.setProperty("spark.executor.memory", "2g")
System.setProperty("spark.scheduler.mode", "FAIR")
System.setProperty("spark.executor.uri", "
http://ip/spark/spark-0.8.0-incubating.tar.gz")
//twitter auth
System.setProperty("twitter4j.oauth.consumerKey", args(0))
System.setProperty("twitter4j.oauth.consumerSecret", args(1))
System.setProperty("twitter4j.oauth.accessToken", args(2))
System.setProperty("twitter4j.oauth.accessTokenSecret", args(3))
Thanks,
prabeesh