You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by prabeesh k <pr...@gmail.com> on 2013/11/14 06:46:06 UTC

Problem with Spark Streaming After update to spark-0.8.0 and mesos-0.14.1

After update from spark-0.7.3 and mesos-0.9.0 to spark-0.8.0 and
mesos-0.14.1.
For  long running streaming project, around 5 to 6 hours I have  no output
for twitter spark streaming examples.  I am using some queue to take the
output as json format using spray json.
Is there any problem using spray json in spark-0.8.0.


My system setting for multi-user support
-------------------------------------------------------------
    System.setProperty("spark.cores.max", "1")
    System.setProperty("spark.mesos.coarse", "true")
    System.setProperty("spark.executor.memory", "2g")
    System.setProperty("spark.scheduler.mode", "FAIR")

    System.setProperty("spark.executor.uri", "
http://ip/spark/spark-0.8.0-incubating.tar.gz")

//twitter auth
    System.setProperty("twitter4j.oauth.consumerKey", args(0))
    System.setProperty("twitter4j.oauth.consumerSecret", args(1))
    System.setProperty("twitter4j.oauth.accessToken", args(2))
    System.setProperty("twitter4j.oauth.accessTokenSecret", args(3))

 Thanks,
        prabeesh