[GitHub] asadoughi opened a new issue #9634: [Feature request] Warning message or exception on mxnet.metric.Loss.update interface for performance

[GitBox <gi...@apache.org>]

asadoughi opened a new issue #9634: [Feature request] Warning message or exception on mxnet.metric.Loss.update interface for performance
URL: https://github.com/apache/incubator-mxnet/issues/9634
 
 
   ## Description
   As shown in the example below, `mxnet.metric.Loss.update` can be over 100 times slower if the interface is used incorrectly. I think a warning message or type checking exception (similar to `check_label_shapes`) would prevent future users from hitting the same performance pitfall in the future.
   
   ## Environment info (Required)
   
   ```
   ----------Python Info----------
   ('Version      :', '2.7.12')
   ('Compiler     :', 'GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)')
   ('Build        :', ('default', 'Oct 11 2016 05:24:00'))
   ('Arch         :', ('64bit', ''))
   ------------Pip Info-----------
   ('Version      :', '9.0.1')
   ('Directory    :', '/usr/local/lib/python2.7/site-packages/pip')
   ----------MXNet Info-----------
   ('Version      :', '1.0.0')
   ('Directory    :', '/usr/local/lib/python2.7/site-packages/mxnet')
   ('Commit Hash   :', '9ef196909ec7bf9cdda66d5b97c92793109798e1')
   ----------System Info----------
   ('Platform     :', 'Darwin-16.7.0-x86_64-i386-64bit')
   ('system       :', 'Darwin')
   ('node         :', 'localhost')
   ('release      :', '16.7.0')
   ('version      :', 'Darwin Kernel Version 16.7.0: Wed Oct  4 00:17:00 PDT 2017; root:xnu-3789.71.6~1/RELEASE_X86_64')
   ----------Hardware Info----------
   ('machine      :', 'x86_64')
   ('processor    :', 'i386')
   machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT PREFETCHW RDTSCP TSCI
   machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID SMAP RDSEED ADX IPT FPU_CSDS
   machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
   machdep.cpu.brand_string: Intel(R) Core(TM) i7-5557U CPU @ 3.10GHz
   ----------Network Test----------
   Setting timeout: 10
   Timing for MXNet: https://github.com/apache/incubator-mxnet, DNS: 0.0126 sec, LOAD: 0.6401 sec.
   Timing for PYPI: https://pypi.python.org/pypi/pip, DNS: 0.0113 sec, LOAD: 0.5746 sec.
   Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0204 sec, LOAD: 0.4498 sec.
   Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0129 sec, LOAD: 0.1008 sec.
   Timing for Gluon Tutorial(en): http://gluon.mxnet.io, DNS: 0.0152 sec, LOAD: 0.1733 sec.
   Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>, DNS finished in 0.202080965042 sec.
   ```
   
   ## Build info
   Installed from pip: mxnet==1.0.0.post4
   
   ## Minimum reproducible example
   ```
   >>> import timeit
   >>> import mxnet
   >>> mxnet.__version__
   '1.0.0'
   >>> timeit.timeit('l.update(None, preds=a)', setup='import mxnet; l = mxnet.metric.Loss(); a = mxnet.nd.array([5] * 100);', number=1000)
   5.851593017578125
   >>> timeit.timeit('l.update(None, preds=[a])', setup='import mxnet; l = mxnet.metric.Loss(); a = mxnet.nd.array([5] * 100);', number=1000)
   0.05304884910583496
   ```
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services