[Advice works] Re: Can not get the expected output when running the BroadcastTest example program.

[杨强 <ya...@ict.ac.cn>]

Thanks, Aaron.
Your advice really works.

Does this mean that the collect() method pulls all related data from slave nodes to master node?




    Sincerely

Yang, Qiang

发件人： Aaron Davidson
发送时间： 2013年11月19日(星期二) 下午12:38
收件人： user; yangqiang
主题： Re: Can not get the expected output when running the BroadcastTestexample program.
Assuming your cluster is actually working (e.g., other examples like SparkPi work), then the problem is probably that println() doesn't actually write output back to the driver; instead, it may just be outputting locally to each slave. You can test this by replacing lines 43 through 45 with:


  sc.parallelize(1 to 10, slices).map {
    i => barr1.value.size
  }.collect().foreach(i => println(i))


which should gather the exact same data but ensure that the printlns actually occur on the driver.