You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Samuel Shepard (Jira)" <ji...@apache.org> on 2020/01/16 20:01:02 UTC

[jira] [Commented] (SPARK-6235) Address various 2G limits

    [ https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017438#comment-17017438 ] 

Samuel Shepard commented on SPARK-6235:
---------------------------------------

[~irashid] I followed your suggestion of looking in the user archive and [found an old PR |https://github.com/apache/spark/pull/17907] that tried to fix the PCA call itself.  It was closed, but I linked it back here. [~srowen] is also on the thread. I leave this comment to help direct users to a workaround as much to encourage a future fix.

Thanks for all you guys do.

> Address various 2G limits
> -------------------------
>
>                 Key: SPARK-6235
>                 URL: https://issues.apache.org/jira/browse/SPARK-6235
>             Project: Spark
>          Issue Type: Umbrella
>          Components: Shuffle, Spark Core
>            Reporter: Reynold Xin
>            Priority: Major
>             Fix For: 2.4.0
>
>         Attachments: SPARK-6235_Design_V0.02.pdf
>
>
> An umbrella ticket to track the various 2G limit we have in Spark, due to the use of byte arrays and ByteBuffers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org