Problem when trying to modify data generated with collect() method from RDD

[杨强 <ya...@ict.ac.cn>]

Hi, everyone.

I'm using scala to implement a connected component algorithm in Spark. And the question codes are as follows:

1    type Graph = ListBuffer[Array[String]] 
2    type CCS = ListBuffer[Graph]
3    val ccs_array:Array[CCS] = graphs_rdd.map{ graph => find_cc(graph)}.collect() 
4    var ccs_1 = ccs_array(0)
5    print_ccs(ccs_1)
6    ccs_1(0) += Array("1", "100")
7    print_ccs(ccs_1)

It seems that line 6 did not change ccs_1's data. line 5 and line 7 shows the same result.

And I tried this: if initialize ccs_array with ordinary data, ccs_1 can be successfully changed.


Is that mean the data returned by collect() can not be changed with the code like line 6?
How should I change my codes?





    Thanks.

Qiang Yang