[jira] [Created] (IGNITE-6394) Can't create off-heap matrix with size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory)

["Oleg Ignatenko (JIRA)" <ji...@apache.org>]

Oleg Ignatenko created IGNITE-6394:
--------------------------------------

             Summary: Can't create off-heap matrix with size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory)
                 Key: IGNITE-6394
                 URL: https://issues.apache.org/jira/browse/IGNITE-6394
             Project: Ignite
          Issue Type: Bug
          Components: ml
            Reporter: Oleg Ignatenko
            Assignee: Yury Babak


In order to reproduce, try to create off-heap matrix with size larger than 2Gb, eg like this:

{code}new DenseLocalOffHeapMatrix(17_000, 17_000){code}

Above fails:
{noformat}
java.lang.IllegalArgumentException: null
        at sun.misc.Unsafe.allocateMemory(Native Method)
        at org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1068)
        at org.apache.ignite.ml.math.impls.storage.matrix.DenseOffHeapMatrixStorage.allocateMemory(DenseOffHeapMatrixStorage.java:219)
        ...
{noformat}

(you may need to allow Java get more than 2Gb off-heap memory for that with VM argument like {{-XX:MaxDirectMemorySize=1000g}})

The reason for failure is that int arguments aren't converted to long prior to multiplying in {{DenseOffHeapMatrixStorage.allocateMemory}}:

{code}ptr = GridUnsafe.allocateMemory((long)rows * cols * Double.BYTES);{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)