You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@singa.apache.org by GitBox <gi...@apache.org> on 2020/09/16 10:20:30 UTC

[GitHub] [singa] dcslin opened a new pull request #797: Half float checkpoint

dcslin opened a new pull request #797:
URL: https://github.com/apache/singa/pull/797


   added half cpp backend, half tensor conversion, half tensor math operations, added half to linear, softmax layer, cnn layers, opt module. tested examples naive, module, train_cnn on mlp and cnn, refactored broadcast, added benchmark for fp16 vs fp32


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] codecov[bot] edited a comment on pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #797:
URL: https://github.com/apache/singa/pull/797#issuecomment-693316765


   # [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=h1) Report
   > Merging [#797](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/singa/commit/95f7b3914ce9c7d57d4f87aedb2d2c4d21898189?el=desc) will **decrease** coverage by `6.28%`.
   > The diff coverage is `20.93%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/singa/pull/797/graphs/tree.svg?width=650&height=150&src=pr&token=raMbqTl5Tl)](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##              dev     #797      +/-   ##
   ==========================================
   - Coverage   70.05%   63.76%   -6.29%     
   ==========================================
     Files         100       87      -13     
     Lines       11573     4902    -6671     
   ==========================================
   - Hits         8107     3126    -4981     
   + Misses       3466     1776    -1690     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | #singa-cpp | `63.76% <20.93%> (-1.33%)` | :arrow_down: |
   | #singa-python | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [include/half.hpp](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9oYWxmLmhwcA==) | `0.00% <ø> (ø)` | |
   | [include/singa/core/common.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL2NvbW1vbi5o) | `100.00% <ø> (ø)` | |
   | [include/singa/core/tensor.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL3RlbnNvci5o) | `73.21% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoLmg=) | `5.10% <0.00%> (ø)` | |
   | [src/model/operation/convolution.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL21vZGVsL29wZXJhdGlvbi9jb252b2x1dGlvbi5jYw==) | `90.00% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math\_cpp.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoX2NwcC5o) | `63.04% <20.68%> (-3.02%)` | :arrow_down: |
   | [src/core/tensor/tensor.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvci5jYw==) | `75.75% <25.00%> (-0.80%)` | :arrow_down: |
   | [data.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-ZGF0YS5weQ==) | | |
   | [singa\_wrap.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c2luZ2Ffd3JhcC5weQ==) | | |
   | [snapshot.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c25hcHNob3QucHk=) | | |
   | ... and [12 more](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=footer). Last update [95f7b39...f20345c](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] codecov[bot] edited a comment on pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #797:
URL: https://github.com/apache/singa/pull/797#issuecomment-693316765


   # [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=h1) Report
   > Merging [#797](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/singa/commit/95f7b3914ce9c7d57d4f87aedb2d2c4d21898189?el=desc) will **decrease** coverage by `6.30%`.
   > The diff coverage is `20.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/singa/pull/797/graphs/tree.svg?width=650&height=150&src=pr&token=raMbqTl5Tl)](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##              dev     #797      +/-   ##
   ==========================================
   - Coverage   70.05%   63.74%   -6.31%     
   ==========================================
     Files         100       87      -13     
     Lines       11573     4904    -6669     
   ==========================================
   - Hits         8107     3126    -4981     
   + Misses       3466     1778    -1688     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | #singa-cpp | `63.74% <20.00%> (-1.35%)` | :arrow_down: |
   | #singa-python | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [include/half.hpp](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9oYWxmLmhwcA==) | `0.00% <ø> (ø)` | |
   | [include/singa/core/common.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL2NvbW1vbi5o) | `100.00% <ø> (ø)` | |
   | [include/singa/core/tensor.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL3RlbnNvci5o) | `73.21% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoLmg=) | `5.03% <0.00%> (-0.08%)` | :arrow_down: |
   | [src/model/operation/convolution.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL21vZGVsL29wZXJhdGlvbi9jb252b2x1dGlvbi5jYw==) | `90.00% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math\_cpp.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoX2NwcC5o) | `63.04% <20.68%> (-3.02%)` | :arrow_down: |
   | [src/core/tensor/tensor.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvci5jYw==) | `75.75% <25.00%> (-0.80%)` | :arrow_down: |
   | [data.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-ZGF0YS5weQ==) | | |
   | [sonnx.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c29ubngucHk=) | | |
   | [autograd.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-YXV0b2dyYWQucHk=) | | |
   | ... and [12 more](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=footer). Last update [95f7b39...f20345c](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] nudles merged pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
nudles merged pull request #797:
URL: https://github.com/apache/singa/pull/797


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] codecov[bot] commented on pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #797:
URL: https://github.com/apache/singa/pull/797#issuecomment-693316765


   # [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=h1) Report
   > Merging [#797](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/singa/commit/95f7b3914ce9c7d57d4f87aedb2d2c4d21898189?el=desc) will **decrease** coverage by `6.28%`.
   > The diff coverage is `20.93%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/singa/pull/797/graphs/tree.svg?width=650&height=150&src=pr&token=raMbqTl5Tl)](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##              dev     #797      +/-   ##
   ==========================================
   - Coverage   70.05%   63.76%   -6.29%     
   ==========================================
     Files         100       87      -13     
     Lines       11573     4902    -6671     
   ==========================================
   - Hits         8107     3126    -4981     
   + Misses       3466     1776    -1690     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | #singa-cpp | `63.76% <20.93%> (-1.33%)` | :arrow_down: |
   | #singa-python | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [include/half.hpp](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9oYWxmLmhwcA==) | `0.00% <ø> (ø)` | |
   | [include/singa/core/common.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL2NvbW1vbi5o) | `100.00% <ø> (ø)` | |
   | [include/singa/core/tensor.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL3RlbnNvci5o) | `73.21% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoLmg=) | `5.10% <0.00%> (ø)` | |
   | [src/model/operation/convolution.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL21vZGVsL29wZXJhdGlvbi9jb252b2x1dGlvbi5jYw==) | `90.00% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math\_cpp.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoX2NwcC5o) | `63.04% <20.68%> (-3.02%)` | :arrow_down: |
   | [src/core/tensor/tensor.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvci5jYw==) | `75.75% <25.00%> (-0.80%)` | :arrow_down: |
   | [utils.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-dXRpbHMucHk=) | | |
   | [sonnx.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c29ubngucHk=) | | |
   | [tensor.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-dGVuc29yLnB5) | | |
   | ... and [12 more](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=footer). Last update [95f7b39...85fc6a1](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] nudles commented on a change in pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #797:
URL: https://github.com/apache/singa/pull/797#discussion_r495539835



##########
File path: include/half.hpp
##########
@@ -0,0 +1,4575 @@
+// half - IEEE 754-based half-precision floating-point library.
+//

Review comment:
       put it under singa/core/ ?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] codecov[bot] edited a comment on pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #797:
URL: https://github.com/apache/singa/pull/797#issuecomment-693316765


   # [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=h1) Report
   > Merging [#797](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=desc) into [dev](https://codecov.io/gh/apache/singa/commit/95f7b3914ce9c7d57d4f87aedb2d2c4d21898189?el=desc) will **decrease** coverage by `6.30%`.
   > The diff coverage is `20.00%`.
   
   [![Impacted file tree graph](https://codecov.io/gh/apache/singa/pull/797/graphs/tree.svg?width=650&height=150&src=pr&token=raMbqTl5Tl)](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree)
   
   ```diff
   @@            Coverage Diff             @@
   ##              dev     #797      +/-   ##
   ==========================================
   - Coverage   70.05%   63.74%   -6.31%     
   ==========================================
     Files         100       87      -13     
     Lines       11573     4904    -6669     
   ==========================================
   - Hits         8107     3126    -4981     
   + Misses       3466     1778    -1688     
   ```
   
   | Flag | Coverage Δ | |
   |---|---|---|
   | #singa-cpp | `63.74% <20.00%> (-1.35%)` | :arrow_down: |
   | #singa-python | `?` | |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Impacted Files](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=tree) | Coverage Δ | |
   |---|---|---|
   | [include/half.hpp](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9oYWxmLmhwcA==) | `0.00% <ø> (ø)` | |
   | [include/singa/core/common.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL2NvbW1vbi5o) | `100.00% <ø> (ø)` | |
   | [include/singa/core/tensor.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-aW5jbHVkZS9zaW5nYS9jb3JlL3RlbnNvci5o) | `73.21% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoLmg=) | `5.03% <0.00%> (-0.08%)` | :arrow_down: |
   | [src/model/operation/convolution.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL21vZGVsL29wZXJhdGlvbi9jb252b2x1dGlvbi5jYw==) | `90.00% <ø> (ø)` | |
   | [src/core/tensor/tensor\_math\_cpp.h](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvcl9tYXRoX2NwcC5o) | `63.04% <20.68%> (-3.02%)` | :arrow_down: |
   | [src/core/tensor/tensor.cc](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c3JjL2NvcmUvdGVuc29yL3RlbnNvci5jYw==) | `75.75% <25.00%> (-0.80%)` | :arrow_down: |
   | [sonnx.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-c29ubngucHk=) | | |
   | [opt.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-b3B0LnB5) | | |
   | [autograd.py](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree#diff-YXV0b2dyYWQucHk=) | | |
   | ... and [12 more](https://codecov.io/gh/apache/singa/pull/797/diff?src=pr&el=tree-more) | |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=continue).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=footer). Last update [95f7b39...b439aac](https://codecov.io/gh/apache/singa/pull/797?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [singa] dcslin commented on pull request #797: Half float checkpoint

Posted by GitBox <gi...@apache.org>.
dcslin commented on pull request #797:
URL: https://github.com/apache/singa/pull/797#issuecomment-693315411


   Usage example:
   ```
   root@1c6aaef3db53:~/singa-hpmerge# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py cnn mnist -m3 -pfloat16 -i 7
   Starting Epoch 0:
   Training loss = 0.061211, training accuracy = 0.752235
   Evaluation accuracy = 0.944912, Elapsed Time = 6.867703s
   Starting Epoch 1:
   Training loss = 237.194122, training accuracy = 0.920124
   Evaluation accuracy = 0.957131, Elapsed Time = 6.740274s
   Starting Epoch 2:
   Training loss = 167.028595, training accuracy = 0.944337
   Evaluation accuracy = 0.970353, Elapsed Time = 6.833484s
   root@1c6aaef3db53:~/singa-hpmerge# PYTHONPATH=build/python/ python3 examples/cnn/train_cnn.py cnn mnist -m3 -pfloat32 -i 7
   Starting Epoch 0:
   Training loss = 596.964600, training accuracy = 0.789421
   Evaluation accuracy = 0.943209, Elapsed Time = 5.502895s
   Starting Epoch 1:
   Training loss = 234.664322, training accuracy = 0.920758
   Evaluation accuracy = 0.960036, Elapsed Time = 5.506354s
   Starting Epoch 2:
   Training loss = 165.501694, training accuracy = 0.944454
   Evaluation accuracy = 0.971254, Elapsed Time = 5.482749s
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org