You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Phillip Henry <lo...@gmail.com> on 2016/05/26 16:06:05 UTC

Distributed matrices with column counts represented by Int (rather than Long)

Hi,

I notice that some DistributedMatrix represent the number of columns with
an Int rather than a Long (RowMatrix etc). This limits the number of
columns to about 2 billion.

We're approaching that limit. What do people recommend we do to mitigate
the problem? Are there plans to use a larger data type as the trait
suggests it should be?

Regards,

Phillip