Python:Streaming Question

[Samarth Mailinglist <ma...@gmail.com>]

I’m trying to run the stateful network word count at
https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/stateful_network_wordcount.py
using the command:

./bin/spark-submit
examples/src/main/python/streaming/stateful_network_wordcount.py
localhost 9999

I am also running netcat at the same time (prior to running the above
command):

nc -lk 9999

However, no wordcount is printed (even though pprint() is being called).

   1. How do I print the results?
   2. How do I otherwise access the data at real time? Suppose I want to
   have a dashboard showing the data in running_counts?

Note that
https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/network_wordcount.py
works perfectly fine.

Running Spark 1.2.0, hadoop 2.4.x prebuilt

Thanks,
Samarth
​

Re: Python:Streaming Question

[Davies Liu <da...@databricks.com>]

There is a known bug with local scheduler, will be fixed by
https://github.com/apache/spark/pull/3779

On Sun, Dec 21, 2014 at 10:57 PM, Samarth Mailinglist
<ma...@gmail.com> wrote:
> I’m trying to run the stateful network word count at
> https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/stateful_network_wordcount.py
> using the command:
>
> ./bin/spark-submit
> examples/src/main/python/streaming/stateful_network_wordcount.py localhost
> 9999
>
> I am also running netcat at the same time (prior to running the above
> command):
>
> nc -lk 9999
>
> However, no wordcount is printed (even though pprint() is being called).
>
> How do I print the results?
> How do I otherwise access the data at real time? Suppose I want to have a
> dashboard showing the data in running_counts?
>
> Note that
> https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/network_wordcount.py
> works perfectly fine.
>
> Running Spark 1.2.0, hadoop 2.4.x prebuilt
>
> Thanks,
> Samarth

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org