You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by ssingal05 <ss...@gmail.com> on 2015/07/31 16:18:42 UTC

Re: Big Integer number in Spark

Do you notice how you are making a List of Int's?

input: org.apache.spark.rdd.RDD[*Int*] = ParallelCollectionRDD[0] at
parallelize at <console>:21 

And these are also being mapped to more Int's

result: org.apache.spark.rdd.RDD[*Int*] = MapPartitionsRDD[1] at map at
<console>:23 

Generally, (signed) Integers can range from (-2^31) to (2^31-1), but that
mapping makes your new integer bigger than 2^31. So the number will wrap
around into the negatives.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Big-Integer-number-in-Spark-tp24095p24096.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org