[GitHub] pengxin99 commented on issue #13221: Flaky tests: test_gluon_model_zoo_gpu.test_training

[GitBox <gi...@apache.org>]

pengxin99 commented on issue #13221: Flaky tests: test_gluon_model_zoo_gpu.test_training
URL: https://github.com/apache/incubator-mxnet/issues/13221#issuecomment-438551532
 
 
   @pengzhao-intel @TaoLv dose this error means that the output of CPU model parameters does not the match with the GPU?
   
   we run this testcase at our machine and can`t reproduce this error, our envs :
   
   - CPU: Intel(R) Xeon(R) CPU E5-2699
   - GPU:Tesla P100
   - mxnet: build from source, `make -j USE_MKLDNN=1 USE_BLAS=mkl USE_OPENCV=1 USE_CUDA=1 USE_CUDNN=1 USE_CUDA_PATH=/usr/local/cuda-9.0
   `
   - MXNET_TEST_SEED=241519495
   
   we got ok result as follows:
   
   > [INFO] Setting module np/mx/python random seeds, use MXNET_MODULE_SEED=1237749972 to reproduce.
   [WARNING] *** test-level seed set: all "@with_seed()" tests run deterministically ***
   [INFO] Setting test np/mx/python random seeds, use MXNET_TEST_SEED=241519495 to reproduce.
   [13:58:44] src/io/iter_image_recordio_2.cc:172: ImageRecordIOParser2: data/val-5k-256.rec, use 1 threads for decoding..
   testing resnet18_v1
   [13:58:46] src/operator/nn/mkldnn/mkldnn_base.cc:74: Allocate 147456 bytes with malloc directly
   [13:58:46] src/operator/nn/mkldnn/mkldnn_base.cc:74: Allocate 8028160 bytes with malloc directly
   [13:58:47] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   resnet18_v1: CPU 3.3956544, GPU 3.395655
   testing densenet121
   [13:58:51] src/operator/nn/./cudnn/./cudnn_algoreg-inl.h:97: Running performance tests to find the best convolution algorithm, this can take a while... (setting env variable MXNET_CUDNN_AUTOTUNE_DEFAULT to 0 to disable)
   densenet121: CPU 2.911241, GPU 2.9112394
   .
   Ran 1 test in 8.308s
   OK
   
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services