You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@singa.apache.org by GitBox <gi...@apache.org> on 2020/09/07 16:15:51 UTC
[GitHub] [singa] dcslin commented on pull request #792: half float update
dcslin commented on pull request #792:
URL: https://github.com/apache/singa/pull/792#issuecomment-688418118
current result:
- [x] training with fp16 ok, with graph, comparable accuracy, faster computation, less memory
- [x] tensor cuda backend generic support on fp16 with broadcast
- [ ] review operations resuing float32
```
root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py mlp mnist -m5
Starting Epoch 0:
Training loss = 446.399231, training accuracy = 0.870331
Evaluation accuracy = 0.922676, Elapsed Time = 4.054065s
Starting Epoch 1:
Training loss = 246.745819, training accuracy = 0.926194
Evaluation accuracy = 0.938301, Elapsed Time = 3.921566s
Starting Epoch 2:
Training loss = 201.893021, training accuracy = 0.939384
Evaluation accuracy = 0.944611, Elapsed Time = 3.735095s
Starting Epoch 3:
Training loss = 171.419769, training accuracy = 0.948289
Evaluation accuracy = 0.952524, Elapsed Time = 3.625971s
Starting Epoch 4:
Training loss = 149.009338, training accuracy = 0.955326
Evaluation accuracy = 0.956530, Elapsed Time = 3.582685s
root@1c6aaef3db53:~/singa-hp2# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py mlp mnist -m5 -pfloat16
Starting Epoch 0:
Training loss = 447.799744, training accuracy = 0.869547
Evaluation accuracy = 0.922075, Elapsed Time = 3.899604s
Starting Epoch 1:
Training loss = 249.704956, training accuracy = 0.925110
Evaluation accuracy = 0.937300, Elapsed Time = 2.524199s
Starting Epoch 2:
Training loss = 206.520721, training accuracy = 0.938334
Evaluation accuracy = 0.942809, Elapsed Time = 2.410751s
Starting Epoch 3:
Training loss = 177.916901, training accuracy = 0.946538
Evaluation accuracy = 0.950120, Elapsed Time = 2.390487s
Starting Epoch 4:
Training loss = 157.046936, training accuracy = 0.952958
Evaluation accuracy = 0.954828, Elapsed Time = 2.396067s
root@1c6aaef3db53:~/singa-hp2#
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org