You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Oleg Ignatenko (JIRA)" <ji...@apache.org> on 2017/09/15 11:26:00 UTC
[jira] [Updated] (IGNITE-6394) Can't create off-heap matrix with
size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory)
[ https://issues.apache.org/jira/browse/IGNITE-6394?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Oleg Ignatenko updated IGNITE-6394:
-----------------------------------
Description:
In order to reproduce, try to create off-heap matrix with size larger than 2Gb, eg like this:
{code}new DenseLocalOffHeapMatrix(17_000, 17_000){code}
Above fails:
{noformat}
java.lang.IllegalArgumentException: null
at sun.misc.Unsafe.allocateMemory(Native Method)
at org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1068)
at org.apache.ignite.ml.math.impls.storage.matrix.DenseOffHeapMatrixStorage.allocateMemory(DenseOffHeapMatrixStorage.java:219)
...
{noformat}
(you may need to allow Java get more than 2Gb off-heap memory for that with VM argument like {{-XX:MaxDirectMemorySize=1000g}})
The reason for failure is that int arguments aren't converted to long prior to multiplying in {{DenseOffHeapMatrixStorage.allocateMemory}}:
{code}ptr = GridUnsafe.allocateMemory(rows * cols * Double.BYTES);{code}
was:
In order to reproduce, try to create off-heap matrix with size larger than 2Gb, eg like this:
{code}new DenseLocalOffHeapMatrix(17_000, 17_000){code}
Above fails:
{noformat}
java.lang.IllegalArgumentException: null
at sun.misc.Unsafe.allocateMemory(Native Method)
at org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1068)
at org.apache.ignite.ml.math.impls.storage.matrix.DenseOffHeapMatrixStorage.allocateMemory(DenseOffHeapMatrixStorage.java:219)
...
{noformat}
(you may need to allow Java get more than 2Gb off-heap memory for that with VM argument like {{-XX:MaxDirectMemorySize=1000g}})
The reason for failure is that int arguments aren't converted to long prior to multiplying in {{DenseOffHeapMatrixStorage.allocateMemory}}:
{code}ptr = GridUnsafe.allocateMemory((long)rows * cols * Double.BYTES);{code}
> Can't create off-heap matrix with size over 2 Gb (int overflow in DenseOffHeapMatrixStorage.allocateMemory)
> -----------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-6394
> URL: https://issues.apache.org/jira/browse/IGNITE-6394
> Project: Ignite
> Issue Type: Bug
> Components: ml
> Reporter: Oleg Ignatenko
> Assignee: Yury Babak
>
> In order to reproduce, try to create off-heap matrix with size larger than 2Gb, eg like this:
> {code}new DenseLocalOffHeapMatrix(17_000, 17_000){code}
> Above fails:
> {noformat}
> java.lang.IllegalArgumentException: null
> at sun.misc.Unsafe.allocateMemory(Native Method)
> at org.apache.ignite.internal.util.GridUnsafe.allocateMemory(GridUnsafe.java:1068)
> at org.apache.ignite.ml.math.impls.storage.matrix.DenseOffHeapMatrixStorage.allocateMemory(DenseOffHeapMatrixStorage.java:219)
> ...
> {noformat}
> (you may need to allow Java get more than 2Gb off-heap memory for that with VM argument like {{-XX:MaxDirectMemorySize=1000g}})
> The reason for failure is that int arguments aren't converted to long prior to multiplying in {{DenseOffHeapMatrixStorage.allocateMemory}}:
> {code}ptr = GridUnsafe.allocateMemory(rows * cols * Double.BYTES);{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)