You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2018/06/22 00:39:47 UTC

[GitHub] vdantu commented on issue #8299: test_cifar10 fails in CI master build

vdantu commented on issue #8299: test_cifar10 fails in CI master build
URL: https://github.com/apache/incubator-mxnet/issues/8299#issuecomment-399285524
 
 
   I couldn't reproduce this issue, any suggestions on what I should try? Also, how could I look for last time this test failed?
   
   
   Tested it on CPU.. 
   
   
   Python environment:
   ---------------------
   **Python 2.7.12** (default, Dec  4 2017, 14:50:18)
   [GCC 5.4.0 20160609] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
   
   Test runs:
   ----------
   (env) :/incubator-mxnet# **MXNET_TEST_COUNT=30000 nosetests --logging-level=DEBUG --verbose -s  tests/python/train/test_dtype.py:test_cifar10**
   
   test_dtype.test_cifar10 ... [00:15:56] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/train.rec, use 1 threads for decoding..
   [00:15:56] src/io/iter_image_recordio_2.cc:228: Load mean image from data/cifar/mean.bin
   [00:15:56] src/io/iter_image_recordio_2.cc:246: Load mean image from data/cifar/mean.bin completed
   [00:15:57] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/test.rec, use 1 threads for decoding..
   [00:15:57] src/io/iter_image_recordio_2.cc:228: Load mean image from data/cifar/mean.bin
   [00:15:57] src/io/iter_image_recordio_2.cc:246: Load mean image from data/cifar/mean.bin completed
   /incubator-mxnet/incubator-mxnet/python/mxnet/model.py:1004: DeprecationWarning: mxnet.model.FeedForward has been deprecated. Please use mxnet.mod.Module instead.
     optimizer=optimizer, initializer=initializer, **kwargs)
   /incubator-mxnet/incubator-mxnet/python/mxnet/model.py:591: DeprecationWarning: Calling initializer with init(str, NDArray) has been deprecated.please use init(mx.init.InitDesc(...), NDArray) instead.
     self.initializer(k, v)
   Start training with [cpu(0)]
   Epoch[0] Batch [50]	Speed: 2353.29 samples/sec	accuracy=0.169531
   Epoch[0] Batch [100]	Speed: 3413.73 samples/sec	accuracy=0.270000
   Epoch[0] Batch [150]	Speed: 3929.30 samples/sec	accuracy=0.323906
   Epoch[0] Batch [200]	Speed: 5827.04 samples/sec	accuracy=0.347656
   Epoch[0] Batch [250]	Speed: 8605.79 samples/sec	accuracy=0.368750
   Epoch[0] Batch [300]	Speed: 7193.14 samples/sec	accuracy=0.380000
   Epoch[0] Batch [350]	Speed: 9871.67 samples/sec	accuracy=0.400156
   Epoch[0] Resetting Data Iterator
   Epoch[0] Time cost=10.354
   Epoch[0] Validation-accuracy=0.409019
   final accuracy = 0.409019
   Epoch[0] Batch [50]	Speed: 6709.95 samples/sec	accuracy=0.183517
   Epoch[0] Batch [100]	Speed: 2762.11 samples/sec	accuracy=0.256094
   Epoch[0] Batch [150]	Speed: 2599.24 samples/sec	accuracy=0.316719
   Epoch[0] Batch [200]	Speed: 4019.71 samples/sec	accuracy=0.352344
   Epoch[0] Batch [250]	Speed: 4482.44 samples/sec	accuracy=0.362969
   Epoch[0] Batch [300]	Speed: 2311.78 samples/sec	accuracy=0.378125
   ^[Epoch[0] Batch [350]	Speed: 6287.10 samples/sec	accuracy=0.394375
   Epoch[0] Train-accuracy=0.390039
   Epoch[0] Time cost=13.991
   Epoch[0] Validation-accuracy=0.415843
   final accuracy = 0.415565
   Start training with [cpu(0)]
   Epoch[0] Batch [50]	Speed: 4131.70 samples/sec	accuracy=0.158438
   Epoch[0] Batch [100]	Speed: 3603.14 samples/sec	accuracy=0.255469
   Epoch[0] Batch [150]	Speed: 3880.94 samples/sec	accuracy=0.309688
   Epoch[0] Batch [200]	Speed: 3466.17 samples/sec	accuracy=0.346719
   Epoch[0] Batch [250]	Speed: 1907.85 samples/sec	accuracy=0.365625
   Epoch[0] Batch [300]	Speed: 13460.59 samples/sec	accuracy=0.377344
   Epoch[0] Batch [350]	Speed: 3278.69 samples/sec	accuracy=0.395781
   Epoch[0] Resetting Data Iterator
   Epoch[0] Time cost=13.526
   Epoch[0] Validation-accuracy=0.420985
   final accuracy = 0.420985
   Epoch[0] Batch [50]	Speed: 2937.12 samples/sec	accuracy=0.171722
   Epoch[0] Batch [100]	Speed: 3230.27 samples/sec	accuracy=0.271875
   Epoch[0] Batch [150]	Speed: 7009.35 samples/sec	accuracy=0.319844
   Epoch[0] Batch [200]	Speed: 7694.27 samples/sec	accuracy=0.345938
   Epoch[0] Batch [250]	Speed: 5635.28 samples/sec	accuracy=0.365000
   Epoch[0] Batch [300]	Speed: 3673.31 samples/sec	accuracy=0.387344
   Epoch[0] Batch [350]	Speed: 1934.83 samples/sec	accuracy=0.399219
   Epoch[0] Train-accuracy=0.394531
   Epoch[0] Time cost=13.540
   Epoch[0] Validation-accuracy=0.419600
   final accuracy = 0.418570
   [00:17:02] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/train.rec, use 1 threads for decoding..
   [00:17:03] src/io/iter_image_recordio_2.cc:170: ImageRecordIOParser2: data/cifar/test.rec, use 1 threads for decoding..
   Start training with [cpu(0)]
   Epoch[0] Batch [50]	Speed: 6886.05 samples/sec	accuracy=0.157344
   Epoch[0] Batch [100]	Speed: 2879.89 samples/sec	accuracy=0.242969
   Epoch[0] Batch [150]	Speed: 2601.20 samples/sec	accuracy=0.255469
   Epoch[0] Batch [200]	Speed: 3848.15 samples/sec	accuracy=0.284531
   Epoch[0] Batch [250]	Speed: 5992.76 samples/sec	accuracy=0.293438
   Epoch[0] Batch [300]	Speed: 3103.23 samples/sec	accuracy=0.314219
   Epoch[0] Batch [350]	Speed: 6909.95 samples/sec	accuracy=0.332344
   Epoch[0] Resetting Data Iterator
   Epoch[0] Time cost=13.161
   Epoch[0] Validation-accuracy=0.352848
   final accuracy = 0.352848
   Epoch[0] Batch [50]	Speed: 7453.44 samples/sec	accuracy=0.161612
   Epoch[0] Batch [100]	Speed: 4445.21 samples/sec	accuracy=0.235469
   Epoch[0] Batch [150]	Speed: 3297.28 samples/sec	accuracy=0.266719
   Epoch[0] Batch [200]	Speed: 6435.73 samples/sec	accuracy=0.292500
   Epoch[0] Batch [250]	Speed: 9299.93 samples/sec	accuracy=0.310625
   Epoch[0] Batch [300]	Speed: 2478.55 samples/sec	accuracy=0.323594
   Epoch[0] Batch [350]	Speed: 4273.77 samples/sec	accuracy=0.324844
   Epoch[0] Train-accuracy=0.309180
   Epoch[0] Time cost=10.967
   Epoch[0] Validation-accuracy=0.358683
   final accuracy = 0.358474
   ok
   
   ----------------------------------------------------------------------
   Ran 1 test in 96.476s
   
   OK

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services