You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by charles li <ch...@gmail.com> on 2016/02/02 03:23:42 UTC

questions about progress bar status [stuck]?

code:

-----------
total = int(1e8)
local_collection = range(1, total)
rdd = sc.parallelize(local_collection)
res = rdd.collect()
-----------

web ui status
-----------

​
problems:
-------

1. from the status bar, it seems that the there should be about half tasks
done, but it just say there is no tasks done in the total 16 tasks.

2. the task just stuck, I have to kill it manually, but I don't know why it
stuck? any idea about this problem?

3. I tried to set total as `1e6`,it  works fine.


-- 
*--------------------------------------*
a spark lover, a quant, a developer and a good man.

http://github.com/litaotao