You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Feng Liu (JIRA)" <ji...@apache.org> on 2017/10/09 05:29:00 UTC

[jira] [Created] (SPARK-22222) Fix the ARRAY_MAX in BufferHolder and add a test

Feng Liu created SPARK-22222:
--------------------------------

             Summary: Fix the ARRAY_MAX in BufferHolder and add a test
                 Key: SPARK-22222
                 URL: https://issues.apache.org/jira/browse/SPARK-22222
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 2.2.1
            Reporter: Feng Liu


This is actually a followup for SPARK-22033, which set the `ARRAY_MAX` to `Int.MaxValue - 8`. It is not a valid number because it will cause the following line fail when such a large byte array is allocated: https://github.com/apache/spark/blob/master/sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/BufferHolder.java#L86 We need to make sure the new length is a multiple of 8.

We need to add one test for the fix. Note that the test should work independently with the heap size of the test JVM. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org