You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2017/12/27 07:39:58 UTC
[GitHub] fcbruce opened a new issue #9207: It seems nd.array or NDArrayIter not support too large array
fcbruce opened a new issue #9207: It seems nd.array or NDArrayIter not support too large array
URL: https://github.com/apache/incubator-mxnet/issues/9207
## Description
I have a large array, cannot be transformed into `nd.array`
## Environment info (Required)
CentOS and MacOS
Package used (Python/R/Scala/Julia):
Python
## Build info (Required if built from source)
install by pip, mxnet=1.0.0
## Error Message:
```
15:31:56] /Users/travis/build/dmlc/mxnet-distro/mxnet-build/dmlc-core/include/dmlc/logging.h:308: [15:31:56] include/mxnet/./tensor_blob.h:275: Check failed: this->shape_.Size() == shape.Size() (6553600000 vs. 2258632704) TBlob.get_with_shape: new and old shape do not match total elements
Stack trace returned 7 entries:
[bt] (0) 0 libmxnet.so 0x0000000d378eaad8 _ZN4dmlc15LogMessageFatalD2Ev + 40
[bt] (1) 1 libmxnet.so 0x0000000d3790cae9 _ZNK5mxnet5TBlob14get_with_shapeIN7mshadow3cpuELi1EfEENS2_6TensorIT_XT0_ET1_EERKNS2_5ShapeIXT0_EEEPNS2_6StreamIS5_EE + 777
[bt] (2) 2 libmxnet.so 0x0000000d380fa0be _ZN5mxnet7ndarray4CopyIN7mshadow3cpuES3_EEvRKNS_5TBlobEPS4_NS_7ContextES8_NS_10RunContextE + 14382
[bt] (3) 3 libmxnet.so 0x0000000d380d9673 _ZNK5mxnet7NDArray15SyncCopyFromCPUEPKvm + 1139
[bt] (4) 4 libmxnet.so 0x0000000d37fcc1fd MXNDArraySyncCopyFromCPU + 13
[bt] (5) 5 _ctypes.cpython-36m-darwin.so 0x0000000101eb742f ffi_call_unix64 + 79
[bt] (6) 6 ??? 0x00007fff5e83a820 0x0 + 140734779074592
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/mxnet/ndarray.py", line 1295, in array
arr[:] = source_array
File "/usr/local/lib/python3.6/site-packages/mxnet/ndarray.py", line 386, in __setitem__
self._sync_copyfrom(value)
File "/usr/local/lib/python3.6/site-packages/mxnet/ndarray.py", line 560, in _sync_copyfrom
ctypes.c_size_t(source_array.size)))
File "/usr/local/lib/python3.6/site-packages/mxnet/base.py", line 129, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [15:31:56] include/mxnet/./tensor_blob.h:275: Check failed: this->shape_.Size() == shape.Size() (6553600000 vs. 2258632704) TBlob.get_with_shape: new and old shape do not match total elements
Stack trace returned 7 entries:
[bt] (0) 0 libmxnet.so 0x0000000d378eaad8 _ZN4dmlc15LogMessageFatalD2Ev + 40
[bt] (1) 1 libmxnet.so 0x0000000d3790cae9 _ZNK5mxnet5TBlob14get_with_shapeIN7mshadow3cpuELi1EfEENS2_6TensorIT_XT0_ET1_EERKNS2_5ShapeIXT0_EEEPNS2_6StreamIS5_EE + 777
[bt] (2) 2 libmxnet.so 0x0000000d380fa0be _ZN5mxnet7ndarray4CopyIN7mshadow3cpuES3_EEvRKNS_5TBlobEPS4_NS_7ContextES8_NS_10RunContextE + 14382
[bt] (3) 3 libmxnet.so 0x0000000d380d9673 _ZNK5mxnet7NDArray15SyncCopyFromCPUEPKvm + 1139
[bt] (4) 4 libmxnet.so 0x0000000d37fcc1fd MXNDArraySyncCopyFromCPU + 13
[bt] (5) 5 _ctypes.cpython-36m-darwin.so 0x0000000101eb742f ffi_call_unix64 + 79
[bt] (6) 6 ??? 0x00007fff5e83a820 0x0 + 140734779074592
```
## Minimum reproducible example
```
import numpy as np
import mxnet as mx
X = np.zeros((20000, 32768), dtypes=np.float32)
mx.nd.array(x)
```
## Steps to reproduce
just run the code above
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services