You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/09/24 18:41:13 UTC
[GitHub] Ishitori opened a new issue #12650: Need to pass state to LSTM when
doing float16 quantization
Ishitori opened a new issue #12650: Need to pass state to LSTM when doing float16 quantization
URL: https://github.com/apache/incubator-mxnet/issues/12650
There is an exception happens if I try to do float16 data type with LSTM and don't provide state as an argument.
## Environment info (Required)
```
----------Python Info----------
Version : 3.6.2
Compiler : GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)
Build : ('default', 'Jul 17 2017 16:44:45')
Arch : ('64bit', '')
------------Pip Info-----------
Version : 9.0.1
Directory : /home/.virtualenvs/check_bug/lib/python3.6/site-packages/pip-9.0.1-py3.6.egg/pip
----------MXNet Info-----------
Version : 1.3.1
Directory : /home/.virtualenvs/check_bug/lib/python3.6/site-packages/mxnet
Commit Hash : d8c51e5fbe1e50a855bc84dacd6061533d6d6a3c
----------System Info----------
Platform : Darwin-16.7.0-x86_64-i386-64bit
system : Darwin
node : dca9048716cf.ant.amazon.com
release : 16.7.0
version : Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64
----------Hardware Info----------
machine : x86_64
processor : i386
b'machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI'
b'machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 HLE AVX2 BMI2 INVPCID RTM SMAP RDSEED ADX IPT SGX FPU_CSDS MPX CLFSOPT'
b'machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX SMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C'
b'machdep.cpu.brand_string: Intel(R) Core(TM) i7-7660U CPU @ 2.50GHz'
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0058 sec, LOAD: 0.5560 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.1204 sec, LOAD: 0.5274 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.3785 sec, LOAD: 0.6223 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0628 sec, LOAD: 0.2368 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0208 sec, LOAD: 0.3941 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0233 sec, LOAD: 0.1508 sec.
```
Python
MXNet commit hash:
1.3.1b20180918
## Error Message:
```
mxnet.base.MXNetError: [11:35:44] src/operator/./rnn-inl.h:695: Check failed: (*in_type)[i] == dtype (0 vs. 2) This layer requires uniform type. Expected 'float16' v.s. given 'float32' at 'state'
Stack trace returned 9 entries:
[bt] (0) 0 libmxnet.so 0x0000000105c81890 libmxnet.so + 18576
[bt] (1) 1 libmxnet.so 0x0000000105c8163f libmxnet.so + 17983
[bt] (2) 2 libmxnet.so 0x00000001076bd10f MXTVMBridge + 3781967
[bt] (3) 3 libmxnet.so 0x0000000107315f33 MXNDListFree + 1749763
[bt] (4) 4 libmxnet.so 0x00000001071e8f2b MXNDListFree + 516859
[bt] (5) 5 libmxnet.so 0x00000001071e7624 MXNDListFree + 510452
[bt] (6) 6 libmxnet.so 0x000000010714052e MXCustomFunctionRecord + 20926
[bt] (7) 7 libmxnet.so 0x0000000107141250 MXImperativeInvokeEx + 176
[bt] (8) 8 _ctypes.cpython-36m-darwin.so 0x0000000104dbb42f ffi_call_unix64 + 79
```
## Minimum reproducible example
```
from mxnet import nd
from mxnet.gluon.rnn import LSTM
# seq_length x batch_size x channels
fake_data = nd.random.uniform(shape=(400, 5, 800), dtype="float16")
lstm_layer = LSTM(hidden_size=100, num_layers=2, dropout=0.2, bidirectional=True)
lstm_layer.cast("float16")
lstm_layer.initialize()
lstm_layer(fake_data)
```
## Steps to reproduce
1. Run the example code
## What have you tried to solve it?
1. Passing state that you get from `begin_state()` fix the issue, but it is inconvenient
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services