You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by marylucy <qa...@hotmail.com> on 2014/09/05 15:40:51 UTC

spark 1.1.0 requested array size exceed vm limits

I am building spark-1.0-rc4 with maven,following http://spark.apache.org/docs/latest/building-with-maven.html
But running graphx edgeFileList ,some tasks failed
error:requested array size exceed vm limits 
error:executor lost

Can anyone know how to fit it

Re: spark 1.1.0 requested array size exceed vm limits

Posted by marylucy <qa...@hotmail.com>.

I set 200，it remain failed in second step,（map and mapPartition in webui）
In spark1.0.2 stable version ,it works well in first step,configuration same as 1.1.0
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: spark 1.1.0 requested array size exceed vm limits

Posted by Ankur Dave <an...@gmail.com>.

At 2014-09-05 21:40:51 +0800, marylucy <qa...@hotmail.com> wrote:
> But running graphx edgeFileList ,some tasks failed
> error:requested array size exceed vm limits 

Try passing a higher value for minEdgePartitions when calling GraphLoader.edgeListFile.

Ankur

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org