You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/11/08 21:02:56 UTC
[GitHub] kalyc commented on issue #8190: why cudnn_lstm_bucketing.py got so
low ppl
kalyc commented on issue #8190: why cudnn_lstm_bucketing.py got so low ppl
URL: https://github.com/apache/incubator-mxnet/issues/8190#issuecomment-437154827
Hi @xfwu
Thanks for submitting the issue.
I ran it locally on my machine - using this setup:
```
----------Python Info----------
('Version :', '2.7.12')
('Compiler :', 'GCC 5.4.0 20160609')
('Build :', ('default', 'Dec 4 2017 14:50:18'))
('Arch :', ('64bit', 'ELF'))
------------Pip Info-----------
('Version :', '18.0')
('Directory :', '/usr/local/lib/python2.7/dist-packages/pip')
----------MXNet Info-----------
('Version :', '1.3.0')
('Directory :', '/home/ubuntu/.local/lib/python2.7/site-packages/mxnet')
('Commit Hash :', 'b3be92f4a48bce62a5a8424271871c2f81c8f7f1')
----------System Info----------
('Platform :', 'Linux-4.4.0-1070-aws-x86_64-with-Ubuntu-16.04-xenial')
('system :', 'Linux')
('node :', 'ip-172-31-28-130')
('release :', '4.4.0-1070-aws')
('version :', '#80-Ubuntu SMP Thu Oct 4 13:56:07 UTC 2018')
----------Hardware Info----------
('machine :', 'x86_64')
('processor :', 'x86_64')
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
Stepping: 1
CPU MHz: 2699.984
CPU max MHz: 3000.0000
CPU min MHz: 1200.0000
BogoMIPS: 4600.12
Hypervisor vendor: Xen
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 46080K
NUMA node0 CPU(s): 0-31
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single kaiser fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx xsaveopt
----------Network Test----------
Setting timeout: 10
Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0017 sec, LOAD: 0.4840 sec.
Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0018 sec, LOAD: 0.3139 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0096 sec, LOAD: 0.0965 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0095 sec, LOAD: 0.0710 sec.
Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.1085 sec, LOAD: 0.0982 sec.
Timing for Gluon Tutorial(cn): https://zh.gluon.ai, DNS: 0.3048 sec, LOAD: 0.1140 sec.
```
These are the results I got:
```
2018-11-08 20:49:46,367 Epoch[19] Batch [50] Speed: 1201.79 samples/sec perplexity=8024.081351
2018-11-08 20:49:56,271 Epoch[19] Batch [100] Speed: 1242.01 samples/sec perplexity=8447.256914
2018-11-08 20:50:06,437 Epoch[19] Batch [150] Speed: 1209.93 samples/sec perplexity=6992.365981
2018-11-08 20:50:09,927 Epoch[19] Train-perplexity=6135.436872
2018-11-08 20:50:09,927 Epoch[19] Time cost=33.991
2018-11-08 20:50:11,832 Saved checkpoint to "train.unidir.nostrip-0020.params"
2018-11-08 20:50:12,538 Epoch[19] Validation-perplexity=3017.244922
2018-11-08 20:50:22,931 Epoch[20] Batch [50] Speed: 1211.68 samples/sec perplexity=5034.274162
2018-11-08 20:50:32,891 Epoch[20] Batch [100] Speed: 1234.94 samples/sec perplexity=6207.409898
2018-11-08 20:50:42,735 Epoch[20] Batch [150] Speed: 1249.52 samples/sec perplexity=6842.742822
2018-11-08 20:50:46,286 Epoch[20] Train-perplexity=6349.078575
2018-11-08 20:50:46,286 Epoch[20] Time cost=33.712
2018-11-08 20:50:48,180 Saved checkpoint to "train.unidir.nostrip-0021.params"
2018-11-08 20:50:48,891 Epoch[20] Validation-perplexity=6220.802213
2018-11-08 20:50:58,136 Epoch[21] Batch [50] Speed: 1357.07 samples/sec perplexity=3308.880624
2018-11-08 20:51:08,265 Epoch[21] Batch [100] Speed: 1214.29 samples/sec perplexity=5873.390678
2018-11-08 20:51:17,889 Epoch[21] Batch [150] Speed: 1278.07 samples/sec perplexity=6322.189451
2018-11-08 20:51:22,280 Epoch[21] Train-perplexity=7172.830300
2018-11-08 20:51:22,280 Epoch[21] Time cost=33.354
2018-11-08 20:51:24,172 Saved checkpoint to "train.unidir.nostrip-0022.params"
2018-11-08 20:51:24,887 Epoch[21] Validation-perplexity=4239.260860
2018-11-08 20:51:35,053 Epoch[22] Batch [50] Speed: 1230.70 samples/sec perplexity=3933.410419
2018-11-08 20:51:45,743 Epoch[22] Batch [100] Speed: 1150.60 samples/sec perplexity=6443.745215
2018-11-08 20:51:55,227 Epoch[22] Batch [150] Speed: 1296.91 samples/sec perplexity=5743.029777
2018-11-08 20:51:58,971 Epoch[22] Train-perplexity=5992.468836
2018-11-08 20:51:58,971 Epoch[22] Time cost=34.048
2018-11-08 20:52:00,859 Saved checkpoint to "train.unidir.nostrip-0023.params"
2018-11-08 20:52:01,575 Epoch[22] Validation-perplexity=1847.542092
2018-11-08 20:52:12,234 Epoch[23] Batch [50] Speed: 1189.78 samples/sec perplexity=4754.425568
2018-11-08 20:52:22,415 Epoch[23] Batch [100] Speed: 1208.19 samples/sec perplexity=5518.603893
2018-11-08 20:52:32,484 Epoch[23] Batch [150] Speed: 1221.55 samples/sec perplexity=4668.357498
2018-11-08 20:52:35,615 Epoch[23] Train-perplexity=4470.175523
2018-11-08 20:52:35,615 Epoch[23] Time cost=34.003
2018-11-08 20:52:37,478 Saved checkpoint to "train.unidir.nostrip-0024.params"
```
Could you share your system configuration where you got the above results?
I would recommend re-building using `make clean` first and then building as `make -j $(nproc) USE_OPENCV=1 USE_BLAS=openblas USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services