You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by wa...@apache.org on 2020/09/28 09:34:05 UTC
[singa-doc] branch master updated (7f5aeab -> 208c813)
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git.
from 7f5aeab Merge pull request #31 from chrishkchris/optimizer
new 089110a add the page for installation using pip and wheel
new ca74475 update the installation page for pip
new 13b6890 update docs for v3.1.0.rc1
new 96bb9ef update the docs for v3.1.0.rc1
new 9109460 merge updates from upstream
new adc4138 update graph.md doc
new 208c813 merge updates from chris
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs-site/docs/assets/singav3.1-sw.png | Bin 0 -> 20083 bytes
docs-site/docs/autograd.md | 14 +++----
docs-site/docs/dist-train.md | 4 +-
docs-site/docs/examples.md | 12 +++---
docs-site/docs/graph.md | 22 ++++++----
docs-site/docs/installation.md | 45 +++++++++++++++++++--
docs-site/docs/optimizer.md | 29 +++++++------
docs-site/docs/software-stack.md | 22 +++++++---
docs-site/docs/team-list.md | 22 +++++-----
docs-site/docs/tensor.md | 16 ++++----
docs-site/docs/time-profiling.md | 28 +++++++++----
.../docs/{issue-tracking.md => wheel-cpu-dev.md} | 11 ++---
.../docs/{source-repository.md => wheel-cpu.md} | 25 +++++-------
.../docs/{issue-tracking.md => wheel-gpu-dev.md} | 11 ++---
docs-site/docs/wheel-gpu.md | 21 ++++++++++
docs-site/website/pages/en/index.js | 2 +-
docs-site/website/static/img/singav3.1-sw.png | Bin 0 -> 20083 bytes
17 files changed, 189 insertions(+), 95 deletions(-)
create mode 100644 docs-site/docs/assets/singav3.1-sw.png
copy docs-site/docs/{issue-tracking.md => wheel-cpu-dev.md} (62%)
copy docs-site/docs/{source-repository.md => wheel-cpu.md} (50%)
copy docs-site/docs/{issue-tracking.md => wheel-gpu-dev.md} (57%)
create mode 100644 docs-site/docs/wheel-gpu.md
create mode 100644 docs-site/website/static/img/singav3.1-sw.png
[singa-doc] 06/07: update graph.md doc
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit adc4138202b80885505c68f7af7593e1ae01573a
Author: wang wei <wa...@gmail.com>
AuthorDate: Mon Sep 28 17:29:47 2020 +0800
update graph.md doc
---
docs-site/docs/graph.md | 3 ---
1 file changed, 3 deletions(-)
diff --git a/docs-site/docs/graph.md b/docs-site/docs/graph.md
index 93347c4..e08e69f 100644
--- a/docs-site/docs/graph.md
+++ b/docs-site/docs/graph.md
@@ -124,14 +124,11 @@ class
```python
class MLP(model.Model):
-<<<<<<< HEAD
-=======
def __init__(self, data_size=10, perceptron_size=100, num_classes=10):
super(MLP, self).__init__()
self.linear1 = layer.Linear(perceptron_size)
...
->>>>>>> upstream/master
def forward(self, inputs):
y = self.linear1(inputs)
[singa-doc] 02/07: update the installation page for pip
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit ca744759f15cd6730c627c461443b6c5ceeb1df3
Author: wang wei <wa...@gmail.com>
AuthorDate: Mon Jul 20 17:59:34 2020 +0800
update the installation page for pip
---
docs-site/docs/installation.md | 39 +++++++++++++++++++++++++++++++-
docs-site/docs/wheel.md | 50 ------------------------------------------
2 files changed, 38 insertions(+), 51 deletions(-)
diff --git a/docs-site/docs/installation.md b/docs-site/docs/installation.md
index b7a21cd..48e44d1 100644
--- a/docs-site/docs/installation.md
+++ b/docs-site/docs/installation.md
@@ -5,7 +5,7 @@ title: Installation
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
-## From Conda
+## Using Conda
Conda is a package manager for Python, CPP and other packages.
@@ -56,6 +56,43 @@ $ python -c "from singa import tensor"
then SINGA is installed successfully.
+## Using Pip
+
+1. CPU only
+
+```bash
+pip install singa -f http://singa.apache.org/docs/next/wheel-cpu.html --trusted-host singa.apache.org
+```
+
+You can install a specific version of SINGA via `singa==<version>`, where the
+`<version>` field should be replaced, e.g., `3.0.0`. The available SINGA
+versions are listed at the link.
+
+To install the latest develop version, replace the link with
+http://singa.apache.org/docs/next/wheel-cpu-dev.html
+
+2. GPU With CUDA and cuDNN
+
+```bash
+pip install singa -f http://singa.apache.org/docs/next/wheel-cuda.html --trusted-host singa.apache.org
+```
+
+You can also configure SINGA version and the CUDA version, like
+`singa==3.0.0+cuda10.2`. The available combinations of SINGA version and CUDA
+version are listed at the link.
+
+To install the latest develop version, replace the link with
+http://singa.apache.org/docs/next/wheel-cuda-dev.html
+
+Note: the Python version of your local Python environment will be used to find
+the corresponding wheel package. For example, if your local Python is 3.6, then
+the wheel package compiled on Python 3.6 will be selected by pip and installed.
+In fact, the wheel file's name include SINGA version, CUDA version and Python
+version. Therefore, `pip` knows which wheel file to download and install.
+
+Refer to the comments at the top of the `setup.py` file for how to build the
+wheel packages.
+
## Using Docker
Install Docker on your local host machine following the
diff --git a/docs-site/docs/wheel.md b/docs-site/docs/wheel.md
deleted file mode 100644
index 47259e8..0000000
--- a/docs-site/docs/wheel.md
+++ /dev/null
@@ -1,50 +0,0 @@
----
-id: wheel
-title: Wheel
----
-
-<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
-
-## Installation Instructions
-
-The wheel package of SINGA is installed via `pip`. Depending on the hardware
-environment, here are two ways to install the wheel package.
-
-- CPU only
-
-```bash
-pip install singa-<version> -f http://singa.apache.org/docs/next/wheel.html
-```
-
-The `<version>` field should be replaced, e.g., `3.0.0.dev1`. The available
-SINGA versions are listed below.
-
-- With CUDA and cuDNN
-
-```bash
-pip install singa-<version>+cuda<cu_version> -f http://singa.apache.org/docs/next/wheel.html
-```
-
-The `<version>` field should be replaced with specific SINGA version; The
-`<cu_version>` field should be replaced with specific CUDA version; For example,
-`singa-3.0.0.dev1+cuda10.2` is the full version specification for one package.
-The available combinations of SINGA version and CUDA version are listed below.
-
-Note: the Python version of your local Python environment will be used to find
-the corresponding wheel package. For example, if your local Python is 3.6, then
-the wheel package compiled on Python 3.6 will be selected by pip and installed.
-In fact, the wheel file's name include SINGA version, CUDA version and Python
-version. Therefore, `pip` knows which wheel file to download and install.
-
-## Building Instructions
-
-Refer to the comments at the top of the `setup.py` file.
-
-## SINGA-3.0.0.dev1
-
-- [CPU only, Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp36-cp36m-manylinux2014_x86_64.whl)
-- [CPU only, Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp37-cp37m-manylinux2014_x86_64.whl)
-- [CPU only, Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp38-cp38-manylinux2014_x86_64.whl)
-- [CUDA10.2, cuDNN 7.6.5, Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1%2Bcuda10.2-cp36-cp36m-manylinux2014_x86_64.whl)
-- [CUDA10.2, cuDNN 7.6.5, Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp37-cp37m-manylinux2014_x86_64.whl)
-- [CUDA10.2, cuDNN 7.6.5, Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp38-cp38-manylinux2014_x86_64.whl)
[singa-doc] 04/07: update the docs for v3.1.0.rc1
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit 96bb9ef89722842a2882af52cfd1d8d173a8cdb0
Author: wang wei <wa...@gmail.com>
AuthorDate: Thu Sep 24 10:39:27 2020 +0800
update the docs for v3.1.0.rc1
---
docs-site/docs/assets/singav3.1-sw.png | Bin 0 -> 20083 bytes
docs-site/docs/graph.md | 9 +++++++++
docs-site/docs/installation.md | 6 ++++--
docs-site/docs/software-stack.md | 20 +++++++++-----------
docs-site/docs/wheel-cpu-dev.md | 12 ++++++++++++
docs-site/docs/wheel-cpu.md | 18 ++++++++++++++++++
docs-site/docs/wheel-gpu-dev.md | 12 ++++++++++++
docs-site/docs/wheel-gpu.md | 21 +++++++++++++++++++++
docs-site/website/pages/en/index.js | 2 +-
docs-site/website/static/img/singav3.1-sw.png | Bin 0 -> 20083 bytes
10 files changed, 86 insertions(+), 14 deletions(-)
diff --git a/docs-site/docs/assets/singav3.1-sw.png b/docs-site/docs/assets/singav3.1-sw.png
new file mode 100644
index 0000000..2bfd2c0
Binary files /dev/null and b/docs-site/docs/assets/singav3.1-sw.png differ
diff --git a/docs-site/docs/graph.md b/docs-site/docs/graph.md
index e1e0334..a37d0e8 100644
--- a/docs-site/docs/graph.md
+++ b/docs-site/docs/graph.md
@@ -16,6 +16,15 @@ need to define the neural network model using the
[Model](https://github.com/apache/singa/blob/master/python/singa/model.py) API.
The graph is constructed and optimized at the C++ backend automatically.
+In this way, on the one hand, users implement a network using the
+[Model](./graph) API following the imperative programming style like PyTorch.
+Different to PyTorch which recreates the operations in every iteration, SINGA
+buffers the operations to create a computational graph implicitly (when this
+feature is enabled) after the first iteration. Therefore, on the other hand,
+SINGA has a similar computational graph as the one created by libraries using
+declarative programming, e.g., TensorFlow. Consequently, it can enjoy the
+optimizations done over the graph.
+
## Example
The following code illustrates the usage of the `Model` API.
diff --git a/docs-site/docs/installation.md b/docs-site/docs/installation.md
index 64af47e..72b7d3f 100644
--- a/docs-site/docs/installation.md
+++ b/docs-site/docs/installation.md
@@ -15,14 +15,14 @@ After installing miniconda, execute the one of the following commands to install
SINGA.
1. CPU only
- [](https://colab.research.google.com/drive/1Ntkhi-Z6XTR8WYPXiLwujHd2dOm0772V)
+ [](https://colab.research.google.com/drive/1Ntkhi-Z6XTR8WYPXiLwujHd2dOm0772V?usp=sharing)
```shell
$ conda install -c nusdbsystem -c conda-forge singa-cpu
```
2. GPU with CUDA and cuDNN (CUDA driver >=384.81 is required)
- [](https://colab.research.google.com/drive/1do_TLJe18IthLOnBOsHCEe-FFPGk1sPJ)
+ [](https://colab.research.google.com/drive/1do_TLJe18IthLOnBOsHCEe-FFPGk1sPJ?usp=sharing)
```shell
$ conda install -c nusdbsystem -c conda-forge singa-gpu
@@ -59,6 +59,7 @@ then SINGA is installed successfully.
## Using Pip
1. CPU only
+ [](https://colab.research.google.com/drive/17RA056Brwk0vBQTFaZ-l9EbqwADO0NA9?usp=sharing)
```bash
pip install singa -f http://singa.apache.org/docs/next/wheel-cpu.html --trusted-host singa.apache.org
@@ -72,6 +73,7 @@ To install the latest develop version, replace the link with
http://singa.apache.org/docs/next/wheel-cpu-dev.html
2. GPU With CUDA and cuDNN
+ [](https://colab.research.google.com/drive/1W30IPCqj5fG8ADAQsFqclaCLyIclVcJL?usp=sharing)
```bash
pip install singa -f http://singa.apache.org/docs/next/wheel-cuda.html --trusted-host singa.apache.org
diff --git a/docs-site/docs/software-stack.md b/docs-site/docs/software-stack.md
index 620ed61..05d8208 100644
--- a/docs-site/docs/software-stack.md
+++ b/docs-site/docs/software-stack.md
@@ -14,19 +14,9 @@ wraps some CPP data structures and provides additional high-level classes for
neural network training, which makes it convenient to implement complex neural
network models.
-SINGA's programming model enjoys the advantages of imperative programming and
-declarative programming. Users define the network structure and the training
-procedure (data flow) via imperative programming like PyTorch.
-Different to PyTorch which recreates the operations in every iteration, SINGA
-buffers the operations to create a computational graph implicitly (when this
-feature is enabled) after the first iteration. The graph is similar to that
-created by libraries using declarative programming, e.g., TensorFlow. Therefore,
-SINGA can apply the memory and speed optimization techniques over the
-computational graph.
-
Next, we introduce the software stack in a bottom-up manner.
- <br/> **Figure 1 - SINGA V3
+ <br/> **Figure 1 - SINGA V3
software stack.**
## Low-level Backend
@@ -144,6 +134,14 @@ efficiency and memory optimization.
just need to inherit `Model` and define the forward propagation of the model by
creating and calling the layers or operators. `Model` will do autograd and
update the parameters via `Opt` automatically when training data is fed into it.
+With the `Model` API, SINGA enjoys the advantages of imperative programming and
+declarative programming. Users implement a network using the [Model](./graph)
+API following the imperative programming style like PyTorch. Different to
+PyTorch which recreates the operations in every iteration, SINGA buffers the
+operations to create a computational graph implicitly (when this feature is
+enabled) after the first iteration. The graph is similar to that created by
+libraries using declarative programming, e.g., TensorFlow. Therefore, SINGA can
+apply the memory and speed optimization techniques over the computational graph.
### ONNX
diff --git a/docs-site/docs/wheel-cpu-dev.md b/docs-site/docs/wheel-cpu-dev.md
new file mode 100644
index 0000000..9f8f7af
--- /dev/null
+++ b/docs-site/docs/wheel-cpu-dev.md
@@ -0,0 +1,12 @@
+---
+id: wheel-cpu-dev
+title: CPU only Wheel Packages (develop version)
+---
+
+<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
+
+## 3.0.0.dev200720
+
+- [Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev200720-cp36-cp36m-manylinux2014_x86_64.whl)
+- [Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev200720-cp37-cp37m-manylinux2014_x86_64.whl)
+- [Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev200720-cp38-cp38-manylinux2014_x86_64.whl)
diff --git a/docs-site/docs/wheel-cpu.md b/docs-site/docs/wheel-cpu.md
new file mode 100644
index 0000000..4f0b921
--- /dev/null
+++ b/docs-site/docs/wheel-cpu.md
@@ -0,0 +1,18 @@
+---
+id: wheel-cpu
+title: CPU only Wheel Packages
+---
+
+<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
+
+## 3.1.0.RC1
+
+- [Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.1.0rc1-cp36-cp36m-manylinux2014_x86_64.whl)
+- [Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.1.0rc1-cp37-cp37m-manylinux2014_x86_64.whl)
+- [Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.1.0rc1-cp38-cp38-manylinux2014_x86_64.whl)
+
+## 3.0.0
+
+- [Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0-cp36-cp36m-manylinux2014_x86_64.whl)
+- [Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0-cp37-cp37m-manylinux2014_x86_64.whl)
+- [Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0-cp38-cp38-manylinux2014_x86_64.whl)
diff --git a/docs-site/docs/wheel-gpu-dev.md b/docs-site/docs/wheel-gpu-dev.md
new file mode 100644
index 0000000..3c061e9
--- /dev/null
+++ b/docs-site/docs/wheel-gpu-dev.md
@@ -0,0 +1,12 @@
+---
+id: wheel-gpu-dev
+title: Wheel Packages with CUDA enabled (develop version)
+---
+
+<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
+
+## 3.0.0.dev200720
+
+- [CUDA10.2, cuDNN 7.6.5, Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev200720%2Bcuda10.2-cp36-cp36m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev200720%2Bcuda10.2-cp37-cp37m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev200720%2Bcuda10.2-cp38-cp38-manylinux2014_x86_64.whl)
diff --git a/docs-site/docs/wheel-gpu.md b/docs-site/docs/wheel-gpu.md
new file mode 100644
index 0000000..c64e4f8
--- /dev/null
+++ b/docs-site/docs/wheel-gpu.md
@@ -0,0 +1,21 @@
+---
+id: wheel-gpu
+title: Wheel Packages with CUDA Enabled
+---
+
+<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
+
+## 3.1.0.RC1
+
+- [CUDA10.2, cuDNN 7.6.5, Python
+ 3.6]https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.1.0rc1%2Bcuda10.2-cp36-cp36m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python
+ 3.7]https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.1.0rc1%2Bcuda10.2-cp37-cp37m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python
+ 3.8]https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.1.0rc1%2Bcuda10.2-cp38-cp38-manylinux2014_x86_64.whl)
+
+## 3.0.0
+
+- [CUDA10.2, cuDNN 7.6.5, Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0%2Bcuda10.2-cp36-cp36m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0%2Bcuda10.2-cp37-cp37m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0%2Bcuda10.2-cp38-cp38-manylinux2014_x86_64.whl)
diff --git a/docs-site/website/pages/en/index.js b/docs-site/website/pages/en/index.js
index b26dfef..7aad38e 100644
--- a/docs-site/website/pages/en/index.js
+++ b/docs-site/website/pages/en/index.js
@@ -143,7 +143,7 @@ class Index extends React.Component {
{
content: `SINGA has a simple [software stack and Python interface](./docs/software-stack) to improve usability.`,
imageAlign: "left",
- image: `${siteConfig.baseUrl}img/singav3-sw.png`,
+ image: `${siteConfig.baseUrl}img/singav3.1-sw.png`,
imageAlt: "Usability",
title: "Usability",
},
diff --git a/docs-site/website/static/img/singav3.1-sw.png b/docs-site/website/static/img/singav3.1-sw.png
new file mode 100644
index 0000000..2bfd2c0
Binary files /dev/null and b/docs-site/website/static/img/singav3.1-sw.png differ
[singa-doc] 07/07: merge updates from chris
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit 208c813762651bc64660e1fe1f1c18417b6bbf3c
Merge: adc4138 7f5aeab
Author: wang wei <wa...@gmail.com>
AuthorDate: Mon Sep 28 17:33:20 2020 +0800
merge updates from chris
docs-site/docs/autograd.md | 103 ++++++++++++++----------
docs-site/docs/graph.md | 1 +
docs-site/docs/optimizer.md | 127 ++++++++++++++++++++++++++++++
docs-site/docs/time-profiling.md | 165 +++++++++++++++++++++++++++++++++++++++
docs-site/website/sidebars.json | 4 +-
5 files changed, 358 insertions(+), 42 deletions(-)
diff --cc docs-site/docs/autograd.md
index ece3b53,9bde49d..20cfaa5
--- a/docs-site/docs/autograd.md
+++ b/docs-site/docs/autograd.md
@@@ -192,9 -192,9 +192,9 @@@ for epoch in range(epochs)
### Using the Model API
--The following
-[example](https://github.com/apache/singa/blob/master/examples/autograd/cnn_module.py)
-implements a CNN model using the Model provided by the model.
++The following <<<<<<< HEAD
+[example](https://github.com/apache/singa/blob/master/examples/cnn/model/cnn.py)
+implements a CNN model using the [Model API](./graph).
#### Define the subclass of Model
diff --cc docs-site/docs/optimizer.md
index 0000000,1347e84..4949471
mode 000000,100644..100644
--- a/docs-site/docs/optimizer.md
+++ b/docs-site/docs/optimizer.md
@@@ -1,0 -1,122 +1,127 @@@
+ ---
+ id: optimizer
+ title: Optimizer
+ ---
+
+ <!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agre [...]
+
-SINGA supports various popular optimizers including stochastic gradient descent with momentum, Adam, RMSProp, and AdaGrad, etc. For each of the optimizer, it supports to use a decay schedular to schedule the learning rate to be applied in different epochs. The optimizers and the decay schedulers are included in `singa/opt.py`.
++SINGA supports various popular optimizers including stochastic gradient descent
++with momentum, Adam, RMSProp, and AdaGrad, etc. For each of the optimizer, it
++supports to use a decay schedular to schedule the learning rate to be applied in
++different epochs. The optimizers and the decay schedulers are included in
++`singa/opt.py`.
+
+ ## Create an optimizer
+
+ 1. SGD with momentum
+
+ ```python
+ # define hyperparameter learning rate
+ lr = 0.001
+ # define hyperparameter momentum
+ momentum = 0.9
-# define hyperparameter weight decay
++# define hyperparameter weight decay
+ weight_decay = 0.0001
+
+ from singa import opt
+ sgd = opt.SGD(lr=lr, momentum=momentum, weight_decay=weight_decay)
+ ```
+
+ 2. RMSProp
+
+ ```python
+ # define hyperparameter learning rate
+ lr = 0.001
+ # define hyperparameter rho
+ rho = 0.9
-# define hyperparameter epsilon
++# define hyperparameter epsilon
+ epsilon = 1e-8
-# define hyperparameter weight decay
++# define hyperparameter weight decay
+ weight_decay = 0.0001
+
+ from singa import opt
+ sgd = opt.RMSProp(lr=lr, rho=rho, epsilon=epsilon, weight_decay=weight_decay)
+ ```
+
+ 3. AdaGrad
+
+ ```python
+ # define hyperparameter learning rate
+ lr = 0.001
-# define hyperparameter epsilon
++# define hyperparameter epsilon
+ epsilon = 1e-8
-# define hyperparameter weight decay
++# define hyperparameter weight decay
+ weight_decay = 0.0001
+
+ from singa import opt
+ sgd = opt.AdaGrad(lr=lr, epsilon=epsilon, weight_decay=weight_decay)
+ ```
+
+ 4. Adam
+
+ ```python
+ # define hyperparameter learning rate
+ lr = 0.001
-# define hyperparameter beta 1
++# define hyperparameter beta 1
+ beta_1= 0.9
-# define hyperparameter beta 2
++# define hyperparameter beta 2
+ beta_1= 0.999
-# define hyperparameter epsilon
++# define hyperparameter epsilon
+ epsilon = 1e-8
-# define hyperparameter weight decay
++# define hyperparameter weight decay
+ weight_decay = 0.0001
+
+ from singa import opt
+ sgd = opt.Adam(lr=lr, beta_1=beta_1, beta_2=beta_2, epsilon=epsilon, weight_decay=weight_decay)
+ ```
+
+ ## Create a Decay Scheduler
+
+ ```python
+ from singa import opt
+
+ # define initial learning rate
+ lr_init = 0.001
+ # define the rate of decay in the decay scheduler
+ decay_rate = 0.95
+ # define whether the learning rate schedule is a staircase shape
+ staircase=True
+ # define the decay step of the decay scheduler (in this example the lr is decreased at every 2 steps)
+ decay_steps = 2
+
+ # create the decay scheduler, the schedule of lr becomes lr_init * (decay_rate ^ (step // decay_steps) )
+ lr = opt.ExponentialDecay(0.1, 2, 0.5, True)
+ # Use the lr to create an optimizer
+ sgd = opt.SGD(lr=lr, momentum=0.9, weight_decay=0.0001)
+ ```
+
+ ## Use the optimizer in Model API
+
+ When we create the model, we need to attach the optimizer to the model.
+
+ ```python
+ # create a CNN using the Model API
+ model = CNN()
+
+ # initialize optimizer and attach it to the model
+ sgd = opt.SGD(lr=0.005, momentum=0.9, weight_decay=1e-5)
+ model.set_optimizer(sgd)
+ ```
+
-Then, when we call the model, it runs the `train_one_batch` method that utilizes the optimizer.
++Then, when we call the model, it runs the `train_one_batch` method that utilizes
++the optimizer.
+
+ Hence, an example of an iterative loop to optimize the model is:
+
+ ```python
+ for b in range(num_train_batch):
+ # generate the next mini-batch
+ x, y = ...
+
+ # Copy the data into input tensors
+ tx.copy_from_numpy(x)
+ ty.copy_from_numpy(y)
+
+ # Training with one batch
+ out, loss = model(tx, ty)
-```
++```
diff --cc docs-site/docs/time-profiling.md
index 0000000,8f38da0..8f9fda1
mode 000000,100644..100644
--- a/docs-site/docs/time-profiling.md
+++ b/docs-site/docs/time-profiling.md
@@@ -1,0 -1,153 +1,165 @@@
+ ---
+ id: time-profiling
+ title: Time Profiling
+ ---
+
+ <!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agre [...]
+
-SINGA supports the time profiling of each of the operators buffered in the graph. To utilize the time profiling function, we first call the ```device.SetVerbosity``` method to set the verbosity of the time profilier, and then call the ```device.PrintTimeProfiling``` to print out the results of time profiling.
++SINGA supports the time profiling of each of the operators buffered in the
++graph. To utilize the time profiling function, we first call the
++`device.SetVerbosity` method to set the verbosity of the time profilier, and
++then call the `device.PrintTimeProfiling` to print out the results of time
++profiling.
+
+ ## Setup the Time Profiling Verbosity
+
-To use the time profiling function, we need to set the verbosity. There are three levels of verbosity. With the default value ```verbosity == 0```, it will not do any time profiling. When we set ```verbosity == 1```, it will profile the forward and backward propagation time. When ```verbosity == 2 ```, it will profile the time spent on every buffered operation in the graph.
++To use the time profiling function, we need to set the verbosity. There are
++three levels of verbosity. With the default value `verbosity == 0`, it will not
++do any time profiling. When we set `verbosity == 1`, it will profile the forward
++and backward propagation time. When `verbosity == 2`, it will profile the time
++spent on every buffered operation in the graph.
+
+ The following is the example code to setup the time profiling function:
+
+ ```python
+ # create a device
+ from singa import device
+ dev = device.create_cuda_gpu()
+ # set the verbosity
-verbosity = 2
++verbosity = 2
+ dev.SetVerbosity(verbosity)
+ # optional: skip the first 5 iterations when profiling the time
+ dev.SetSkipIteration(5)
+ ```
+
-Then, after we have completed the training at the end of the program, we can print the time profiling result by calling the ```device.PrintTimeProfiling``` method:
++Then, after we have completed the training at the end of the program, we can
++print the time profiling result by calling the `device.PrintTimeProfiling`
++method:
+
+ ```python
+ dev.PrintTimeProfiling()
+ ```
+
+ ## Example Outputs for Different Verbosity
+
-We can run the ResNet [example](https://github.com/apache/singa/blob/master/examples/cnn/benchmark.py) to see the output with different setting of verbosity:
++We can run the ResNet
++[example](https://github.com/apache/singa/blob/master/examples/cnn/benchmark.py)
++to see the output with different setting of verbosity:
+
-1. ```verbosity == 1```
++1. `verbosity == 1`
+
+ ```
+ Time Profiling:
+ Forward Propagation Time : 0.0409127 sec
+ Backward Propagation Time : 0.114813 sec
+ ```
+
-2. ```verbosity == 2```
++2. `verbosity == 2`
+
+ ```
+ Time Profiling:
+ OP_ID0. SetValue : 1.73722e-05 sec
+ OP_ID1. cudnnConvForward : 0.000612724 sec
+ OP_ID2. GpuBatchNormForwardTraining : 0.000559449 sec
+ OP_ID3. ReLU : 0.000375004 sec
+ OP_ID4. GpuPoolingForward : 0.000240041 sec
+ OP_ID5. SetValue : 3.4176e-06 sec
+ OP_ID6. cudnnConvForward : 0.000115619 sec
+ OP_ID7. GpuBatchNormForwardTraining : 0.000150415 sec
+ OP_ID8. ReLU : 9.95494e-05 sec
+ OP_ID9. SetValue : 3.22432e-06 sec
+ OP_ID10. cudnnConvForward : 0.000648668 sec
+ OP_ID11. GpuBatchNormForwardTraining : 0.000149793 sec
+ OP_ID12. ReLU : 9.92118e-05 sec
+ OP_ID13. SetValue : 3.37728e-06 sec
+ OP_ID14. cudnnConvForward : 0.000400953 sec
+ OP_ID15. GpuBatchNormForwardTraining : 0.000572181 sec
+ OP_ID16. SetValue : 3.21312e-06 sec
+ OP_ID17. cudnnConvForward : 0.000398698 sec
+ OP_ID18. GpuBatchNormForwardTraining : 0.00056836 sec
+ OP_ID19. Add : 0.000542246 sec
+ OP_ID20. ReLU : 0.000372783 sec
+ OP_ID21. SetValue : 3.25312e-06 sec
+ OP_ID22. cudnnConvForward : 0.000260731 sec
+ OP_ID23. GpuBatchNormForwardTraining : 0.000149041 sec
+ OP_ID24. ReLU : 9.9072e-05 sec
+ OP_ID25. SetValue : 3.10592e-06 sec
+ OP_ID26. cudnnConvForward : 0.000637481 sec
+ OP_ID27. GpuBatchNormForwardTraining : 0.000152577 sec
+ OP_ID28. ReLU : 9.90518e-05 sec
+ OP_ID29. SetValue : 3.28224e-06 sec
+ OP_ID30. cudnnConvForward : 0.000404586 sec
+ OP_ID31. GpuBatchNormForwardTraining : 0.000569679 sec
+ OP_ID32. Add : 0.000542291 sec
+ OP_ID33. ReLU : 0.00037211 sec
+ OP_ID34. SetValue : 3.13696e-06 sec
+ OP_ID35. cudnnConvForward : 0.000261219 sec
+ OP_ID36. GpuBatchNormForwardTraining : 0.000148281 sec
+ OP_ID37. ReLU : 9.89299e-05 sec
+ OP_ID38. SetValue : 3.25216e-06 sec
+ OP_ID39. cudnnConvForward : 0.000633644 sec
+ OP_ID40. GpuBatchNormForwardTraining : 0.000150711 sec
+ OP_ID41. ReLU : 9.84902e-05 sec
+ OP_ID42. SetValue : 3.18176e-06 sec
+ OP_ID43. cudnnConvForward : 0.000402752 sec
+ OP_ID44. GpuBatchNormForwardTraining : 0.000571523 sec
+ OP_ID45. Add : 0.000542435 sec
+ OP_ID46. ReLU : 0.000372539 sec
+ OP_ID47. SetValue : 3.24672e-06 sec
+ OP_ID48. cudnnConvForward : 0.000493054 sec
+ OP_ID49. GpuBatchNormForwardTraining : 0.000293142 sec
+ OP_ID50. ReLU : 0.000190047 sec
+ OP_ID51. SetValue : 3.14784e-06 sec
+ OP_ID52. cudnnConvForward : 0.00148837 sec
+ OP_ID53. GpuBatchNormForwardTraining : 8.34794e-05 sec
+ OP_ID54. ReLU : 5.23254e-05 sec
+ OP_ID55. SetValue : 3.40096e-06 sec
+ OP_ID56. cudnnConvForward : 0.000292971 sec
+ OP_ID57. GpuBatchNormForwardTraining : 0.00029174 sec
+ OP_ID58. SetValue : 3.3248e-06 sec
+ OP_ID59. cudnnConvForward : 0.000590154 sec
+ OP_ID60. GpuBatchNormForwardTraining : 0.000294149 sec
+ OP_ID61. Add : 0.000275119 sec
+ OP_ID62. ReLU : 0.000189268 sec
+ OP_ID63. SetValue : 3.2704e-06 sec
+ OP_ID64. cudnnConvForward : 0.000341232 sec
+ OP_ID65. GpuBatchNormForwardTraining : 8.3304e-05 sec
+ OP_ID66. ReLU : 5.23667e-05 sec
+ OP_ID67. SetValue : 3.19936e-06 sec
+ OP_ID68. cudnnConvForward : 0.000542484 sec
+ OP_ID69. GpuBatchNormForwardTraining : 8.60537e-05 sec
+ OP_ID70. ReLU : 5.2479e-05 sec
+ OP_ID71. SetValue : 3.41824e-06 sec
+ OP_ID72. cudnnConvForward : 0.000291295 sec
+ OP_ID73. GpuBatchNormForwardTraining : 0.000292795 sec
+ OP_ID74. Add : 0.000274438 sec
+ OP_ID75. ReLU : 0.000189689 sec
+ OP_ID76. SetValue : 3.21984e-06 sec
+ OP_ID77. cudnnConvForward : 0.000338776 sec
+ OP_ID78. GpuBatchNormForwardTraining : 8.484e-05 sec
+ OP_ID79. ReLU : 5.29408e-05 sec
+ OP_ID80. SetValue : 3.18208e-06 sec
+ OP_ID81. cudnnConvForward : 0.000545542 sec
+ OP_ID82. GpuBatchNormForwardTraining : 8.40976e-05 sec
+ OP_ID83. ReLU : 5.2256e-05 sec
+ OP_ID84. SetValue : 3.36256e-06 sec
+ OP_ID85. cudnnConvForward : 0.000293003 sec
+ OP_ID86. GpuBatchNormForwardTraining : 0.0002989 sec
+ OP_ID87. Add : 0.000275041 sec
+ OP_ID88. ReLU : 0.000189867 sec
+ OP_ID89. SetValue : 3.1184e-06 sec
+ OP_ID90. cudnnConvForward : 0.000340417 sec
+ OP_ID91. GpuBatchNormForwardTraining : 8.39395e-05 sec
+ OP_ID92. ReLU : 5.26544e-05 sec
+ OP_ID93. SetValue : 3.2336e-06 sec
+ OP_ID94. cudnnConvForward : 0.000539787 sec
+ OP_ID95. GpuBatchNormForwardTraining : 8.2753e-05 sec
+ OP_ID96. ReLU : 4.86758e-05 sec
+ OP_ID97. SetValue : 3.24384e-06 sec
+ OP_ID98. cudnnConvForward : 0.000287108 sec
+ OP_ID99. GpuBatchNormForwardTraining : 0.000293127 sec
+ OP_ID100. Add : 0.000269478 sec
+ .
+ .
+ .
-```
++```
[singa-doc] 05/07: merge updates from upstream
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit 910946083c7f379261537fdeeeec447ec80110bc
Merge: 96bb9ef 0914fee
Author: wang wei <wa...@gmail.com>
AuthorDate: Thu Sep 24 10:41:42 2020 +0800
merge updates from upstream
docs-site/docs/contribute-code.md | 2 +-
docs-site/docs/dist-train.md | 99 +++++++++++++++++-----------
docs-site/docs/examples.md | 12 ++++
docs-site/docs/graph.md | 84 +++++++++++++-----------
docs-site/docs/onnx.md | 133 ++++++++++++++++++++------------------
docs-site/docs/team-list.md | 40 ++++++------
docs-site/docs/tensor.md | 128 ++++++++++++++++++++++++------------
7 files changed, 294 insertions(+), 204 deletions(-)
diff --cc docs-site/docs/dist-train.md
index c456440,b6cc0da..f242ffb
--- a/docs-site/docs/dist-train.md
+++ b/docs-site/docs/dist-train.md
@@@ -321,6 -342,14 +342,14 @@@ SINGA provides multiple optimization st
reduce the communication cost. Refer to the API for `DistOpt` for the
configuration of each strategy.
+ When we use `model.Model` to build a model, we need to put the options for
+ distributed training in the `train_one_batch` method. Please refer to the
+ example code on top of this page. We could just copy the code for the options
+ and use it in other models.
+
-With the defined options, we can put the arguments `dist_option` and `spars` when we
-start the training with `model(tx, ty, dist_option, spars)`
++With the defined options, we can put the arguments `dist_option` and `spars`
++when we start the training with `model(tx, ty, dist_option, spars)`
+
### No Optimizations
```python
diff --cc docs-site/docs/examples.md
index 421a9bb,c143563..53255f6
--- a/docs-site/docs/examples.md
+++ b/docs-site/docs/examples.md
@@@ -50,6 -50,18 +50,18 @@@ example is given below
| ---------- | ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
| Bert-Squad | [SQuAD v1.1](https://rajpurkar.github.io/SQuAD-explorer/explore/1.1/dev/) | [Colab](https://colab.research.google.com/drive/1kud-lUPjS_u-TkDAzihBTw0Vqr0FjCE-) |
+ ## Text Classification
+
-| Model | Dataset | Links |
-| ----------- | ----------- | ---------- |
-| Simple LSTM | IMDB | [python]() |
++| Model | Dataset | Links |
++| ----------- | ------- | ---------- |
++| Simple LSTM | IMDB | [python]() |
+
+ ## Text Ranking
-| Model | Dataset | Links |
-| ----------- | ----------- | ---------- |
-| BiLSTM | InsuranceQA | [python]() |
+
++| Model | Dataset | Links |
++| ------ | ----------- | ---------- |
++| BiLSTM | InsuranceQA | [python]() |
+
## Misc.
- Restricted Boltzmann Machine over the MNIST dataset, [source](),
diff --cc docs-site/docs/graph.md
index a37d0e8,abce638..93347c4
--- a/docs-site/docs/graph.md
+++ b/docs-site/docs/graph.md
@@@ -126,14 -115,20 +124,22 @@@ clas
```python
class MLP(model.Model):
++<<<<<<< HEAD
++=======
+
+ def __init__(self, data_size=10, perceptron_size=100, num_classes=10):
+ super(MLP, self).__init__()
+ self.linear1 = layer.Linear(perceptron_size)
+ ...
++>>>>>>> upstream/master
def forward(self, inputs):
- x = autograd.matmul(inputs, self.w0)
+ y = self.linear1(inputs)
...
```
-The `Linear` layer is composed of the `mutmul` operator.
--`autograd` implements the `matmul` operator by calling the function `Mult`
--exposed from CPP via SWIG.
++The `Linear` layer is composed of the `mutmul` operator. `autograd` implements
++the `matmul` operator by calling the function `Mult` exposed from CPP via SWIG.
```python
# implementation of matmul()
diff --cc docs-site/docs/team-list.md
index 0746c88,680cb5b..41bd786
--- a/docs-site/docs/team-list.md
+++ b/docs-site/docs/team-list.md
@@@ -43,17 -46,14 +46,14 @@@ Zhejiang University, NetEase, Osaka Uni
## Contributors
--| Name | Email | Organization |
--| ------------------ | ---------------------------- | --------------------------------------------- |
--| Haibo Chen | hzchenhaibo@corp.netease.com | NetEase |
-| Shicheng Chen | chengsc@comp.nus.edu.sg | National University of Singapore |
--| Xin Ji | vincent.j.xin@gmail.com | Visenze, Singapore |
--| Anthony K. H. Tung | atung@comp.nus.edu.sg | National University of Singapore |
--| Ji Wang | wangji@mzhtechnologies.com | Hangzhou MZH Technologies |
--| Yuan Wang | wangyuan@corp.netease.com | NetEase |
--| Wenfeng Wu | dcswuw@gmail.com | Freelancer, China |
- | Chang Yao | yaochang2009@gmail.com | Hangzhou MZH Technologies |
- | Shicheng Chen | chengsc@comp.nus.edu.sg | National University of Singapore |
- | Joddiy Zhang | joddiyzhang@gmail.com | National University of Singapore |
- | Shicong Lin | dcslin@nus.edu.sg | National University of Singapore |
--| Kaiyuan Yang | yangky@comp.nus.edu.sg | National University of Singapore |
- | Rulin Xing | xjdkcsq3@gmail.com | Huazhong University of Science and Technology |
-| Chang Yao | yaochang2009@gmail.com | Hangzhou MZH Technologies |
++| Name | Email | Organization |
++| ------------------ | ---------------------------- | -------------------------------- |
++| Haibo Chen | hzchenhaibo@corp.netease.com | NetEase |
++| Shicheng Chen | chengsc@comp.nus.edu.sg | National University of Singapore |
++| Xin Ji | vincent.j.xin@gmail.com | Visenze, Singapore |
++| Anthony K. H. Tung | atung@comp.nus.edu.sg | National University of Singapore |
++| Ji Wang | wangji@mzhtechnologies.com | Hangzhou MZH Technologies |
++| Yuan Wang | wangyuan@corp.netease.com | NetEase |
++| Wenfeng Wu | dcswuw@gmail.com | Freelancer, China |
++| Kaiyuan Yang | yangky@comp.nus.edu.sg | National University of Singapore |
++| Chang Yao | yaochang2009@gmail.com | Hangzhou MZH Technologies |
diff --cc docs-site/docs/tensor.md
index 1889a0e,6e983b5..aad5366
--- a/docs-site/docs/tensor.md
+++ b/docs-site/docs/tensor.md
@@@ -45,6 -45,15 +45,16 @@@ array([[1., 0., 0.]
[0. 0.]]
```
+ `tensor` transformation up to 6 dims
-``` python
++
++```python
+ >>> a = tensor.random((2,3,4,5,6,7))
+ >>> a.shape
+ (2, 3, 4, 5, 6, 7)
+ >>> a.reshape((2,3,4,5,7,6)).transpose((3,2,1,0,4,5)).shape
+ (5, 4, 3, 2, 7, 6)
+ ```
+
### Tensor Arithmetic Methods
`tensor` is evaluated in real time.
@@@ -58,6 -67,37 +68,39 @@@
[0. 0.2 0. ]]
```
+ `tensor` broadcasting arithmetic:
-``` python
++
++```python
+ >>> a
+ [[1. 2. 3.]
+ [4. 5. 6.]]
+ >>> b
+ [[1. 2. 3.]]
+ >>> a + b
+ [[2. 4. 6.]
+ [5. 7. 9.]]
+ >>> a * b
+ [[ 1. 4. 9.]
+ [ 4. 10. 18.]]
+ >>> a / b
+ [[1. 1. 1. ]
+ [4. 2.5 2. ]]
+ >>> a/=b # inplace operation
+ >>> a
+ [[1. 1. 1. ]
+ [4. 2.5 2. ]]
+ ```
+
+ `tensor` broadcasting on matrix multiplication (GEMM)
-``` python
++
++```python
+ >>> from singa import tensor
+ >>> a = tensor.random((2,2,2,3))
+ >>> b = tensor.random((2,3,4))
+ >>> tensor.mult(a,b).shape
+ (2, 2, 2, 4)
+ ```
+
### Tensor Functions
Functions in module `singa.tensor` return new `tensor` object after applying the
@@@ -86,55 -126,55 +129,54 @@@ between `device`s via `to_device()` fun
>>> x.to_device(device.get_default_device())
```
- ### Simple Neural Network Example
-
+ ### use Tensor to train MLP
```python
- from singa import device
- from singa import tensor
- from singa import opt
- from singa import autograd
- class MLP:
- def __init__(self):
- self.linear1 = autograd.Linear(3, 4)
- self.linear2 = autograd.Linear(4, 5)
- def forward(self, x):
- y=self.linear1(x)
- return self.linear2(y)
- def train(model, x, t, dev, epochs=10):
- for i in range(epochs):
- y = model.forward(x)
- loss = autograd.mse_loss(y, t)
- print("loss: ", loss)
- sgd = opt.SGD()
- for p, gp in autograd.backward(loss):
- sgd.update(p, gp)
- sgd.step()
- print("training completed")
- if __name__ == "__main__":
- autograd.training = True
- model = MLP()
- dev = device.get_default_device()
- x = tensor.Tensor((2, 3), dev)
- t = tensor.Tensor((2, 5), dev)
- x.gaussian(1,1)
- t.gaussian(1,1)
- train(model, x, t, dev)
+
+ """
- code snipet from examples/mlp/module.py
++ code snipet from examples/mlp/module.py
+ """
+
+ label = get_label()
+ data = get_data()
+
+ dev = device.create_cuda_gpu_on(0)
+ sgd = opt.SGD(0.05)
+
+ # define tensor for input data and label
+ tx = tensor.Tensor((400, 2), dev, tensor.float32)
+ ty = tensor.Tensor((400,), dev, tensor.int32)
+ model = MLP(data_size=2, perceptron_size=3, num_classes=2)
+
+ # attached model to graph
+ model.set_optimizer(sgd)
+ model.compile([tx], is_train=True, use_graph=True, sequential=False)
+ model.train()
+
+ for i in range(1001):
+ tx.copy_from_numpy(data)
+ ty.copy_from_numpy(label)
+ out, loss = model(tx, ty, 'fp32', spars=None)
+
+ if i % 100 == 0:
+ print("training loss = ", tensor.to_numpy(loss)[0])
```
Output:
- ```
- loss: [4.917431]
- loss: [2.5147934]
- loss: [2.0670078]
- loss: [1.9179827]
- loss: [1.8192691]
- loss: [1.7269677]
- loss: [1.6308627]
- loss: [1.52674]
- loss: [1.4122975]
- loss: [1.2866782]
- training completed
-``` bash
-$ python3 examples/mlp/module.py
++```bash
++$ python3 examples/mlp/module.py
+ training loss = 0.6158037
+ training loss = 0.52852553
+ training loss = 0.4571422
+ training loss = 0.37274635
+ training loss = 0.30146334
+ training loss = 0.24906921
+ training loss = 0.21128304
+ training loss = 0.18390492
+ training loss = 0.16362564
+ training loss = 0.148164
+ training loss = 0.13589878
```
## Tensor Implementation
[singa-doc] 03/07: update docs for v3.1.0.rc1
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit 13b689086d82b004b06187f01b937c68fc5bff5b
Author: wang wei <wa...@gmail.com>
AuthorDate: Wed Sep 23 14:25:53 2020 +0800
update docs for v3.1.0.rc1
---
docs-site/docs/autograd.md | 20 ++++++++--------
docs-site/docs/graph.md | 50 ++++++++++++++++++++--------------------
docs-site/docs/installation.md | 4 ++--
docs-site/docs/software-stack.md | 22 ++++++++++++++----
4 files changed, 54 insertions(+), 42 deletions(-)
diff --git a/docs-site/docs/autograd.md b/docs-site/docs/autograd.md
index 4d42070..ece3b53 100644
--- a/docs-site/docs/autograd.md
+++ b/docs-site/docs/autograd.md
@@ -190,21 +190,21 @@ for epoch in range(epochs):
sgd.update(p, gp)
```
-### Using the Module API
+### Using the Model API
The following
-[example](https://github.com/apache/singa/blob/master/examples/autograd/cnn_module.py)
-implements a CNN model using the Module provided by the module.
+[example](https://github.com/apache/singa/blob/master/examples/cnn/model/cnn.py)
+implements a CNN model using the [Model API](./graph).
-#### Define the subclass of Module
+#### Define the subclass of Model
-Define the model class, it should be the subclass of the Module. In this way,
-all operations used during traing phase will form a calculation graph and will
-be analyzed. The operations in the graph will be scheduled and executed
-efficiently. Layers can also be included in the module class.
+Define the model class, it should be the subclass of Model. In this way, all
+operations used during the training phase will form a computational graph and
+will be analyzed. The operations in the graph will be scheduled and executed
+efficiently. Layers can also be included in the model class.
```python
-class MLP(module.Module): # the model is a subclass of Module
+class MLP(model.Model): # the model is a subclass of Model
def __init__(self, optimizer):
super(MLP, self).__init__()
@@ -262,5 +262,5 @@ for i in range(niters):
### Python API
Refer
-[here](https://singa.readthedocs.io/en/latest/docs/autograd.html#module-singa.autograd)
+[here](https://singa.readthedocs.io/en/latest/autograd.html#module-singa.autograd)
for more details of Python API.
diff --git a/docs-site/docs/graph.md b/docs-site/docs/graph.md
index fb4ba69..e1e0334 100644
--- a/docs-site/docs/graph.md
+++ b/docs-site/docs/graph.md
@@ -1,6 +1,6 @@
---
id: graph
-title: Computational Graph
+title: Model
---
<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed [...]
@@ -13,17 +13,17 @@ edge, all operations form a computational graph. With the computational graph,
speed and memory optimization can be conducted by scheduling the execution of
the operations and memory allocation/release intelligently. In SINGA, users only
need to define the neural network model using the
-[Module](https://github.com/apache/singa/blob/master/python/singa/module.py)
-API. The graph is constructed and optimized at the C++ backend automatically.
+[Model](https://github.com/apache/singa/blob/master/python/singa/model.py) API.
+The graph is constructed and optimized at the C++ backend automatically.
## Example
-The following code illustrates the usage of the `Module` API.
+The following code illustrates the usage of the `Model` API.
-1. Implement the new model as a subclass the Module class.
+1. Implement the new model as a subclass the Model class.
```Python
-class CNN(module.Module):
+class CNN(model.Model):
def __init__(self, optimizer):
super(CNN, self).__init__()
@@ -96,7 +96,7 @@ A Google Colab notebook of this example is available
More examples:
-- [MLP](https://github.com/apache/singa/blob/master/examples/mlp/module.py)
+- [MLP](https://github.com/apache/singa/blob/master/examples/mlp/model.py)
- [CNN](https://github.com/apache/singa/blob/master/examples/cnn/model/cnn.py)
- [ResNet](https://github.com/apache/singa/blob/master/examples/cnn/model/resnet.py)
@@ -111,12 +111,12 @@ SINGA constructs the computational graph in three steps:
3. create the nodes and edges based on the dependencies
Take the matrix multiplication operation from the dense layer of a
-[MLP model](https://github.com/apache/singa/blob/master/examples/mlp/module.py)
+[MLP model](https://github.com/apache/singa/blob/master/examples/mlp/model.py)
as an example. The operation is called in the `forward` function of the MLP
class
```python
-class MLP(module.Module):
+class MLP(model.Model):
def forward(self, inputs):
x = autograd.matmul(inputs, self.w0)
@@ -148,7 +148,7 @@ The `Exec` function of `Device` buffers the function and its arguments. In
addition, it also has the information about the blocks (a block is a chunk of
memory for a tensor) to be read and written by this function.
-Once `Module.forward()` has been executed once, all operations are buffered by
+Once `Model.forward()` has been executed once, all operations are buffered by
`Device`. Next, the read/write information of all operations are analyzed to
create the computational graph. For example, if a block `b` is written by one
operation O1 and is later read by another operation O2, we would know O2 depends
@@ -311,7 +311,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
- Model
- Using layer: ResNet50 in
[resnet.py](https://github.com/apache/singa/blob/master/examples/cnn/autograd/resnet_cifar10.py)
- - Using module: ResNet50 in
+ - Using model: ResNet50 in
[resnet.py](https://github.com/apache/singa/blob/master/examples/cnn/model/resnet.py)
- GPU: NVIDIA RTX 2080Ti
- Notations
@@ -346,7 +346,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0000</td>
</tr>
<tr>
- <td nowrap>module:disable graph</td>
+ <td nowrap>model:disable graph</td>
<td>4995</td>
<td>14.1264</td>
<td>14.1579</td>
@@ -355,7 +355,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0049</td>
</tr>
<tr>
- <td nowrap>module:enable graph, bfs</td>
+ <td nowrap>model:enable graph, bfs</td>
<td>3283</td>
<td>13.7438</td>
<td>14.5520</td>
@@ -364,7 +364,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0328</td>
</tr>
<tr>
- <td nowrap>module:enable graph, serial</td>
+ <td nowrap>model:enable graph, serial</td>
<td>3265</td>
<td>13.7420</td>
<td>14.5540</td>
@@ -383,7 +383,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0000</td>
</tr>
<tr>
- <td nowrap>module:disable graph</td>
+ <td nowrap>model:disable graph</td>
<td>10109</td>
<td>13.2952</td>
<td>7.5315</td>
@@ -392,7 +392,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0123</td>
</tr>
<tr>
- <td nowrap>module:enable graph, bfs</td>
+ <td nowrap>model:enable graph, bfs</td>
<td>6839</td>
<td>13.1059</td>
<td>7.6302</td>
@@ -401,7 +401,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0269</td>
</tr>
<tr>
- <td nowrap>module:enable graph, serial</td>
+ <td nowrap>model:enable graph, serial</td>
<td>6845</td>
<td>13.0489</td>
<td>7.6635</td>
@@ -414,10 +414,10 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
### Multi processes
- Experiment settings
- - Model
+ - API
- using Layer: ResNet50 in
[resnet_dist.py](https://github.com/apache/singa/blob/master/examples/cnn/autograd/resnet_dist.py)
- - using Module: ResNet50 in
+ - using Model: ResNet50 in
[resnet.py](https://github.com/apache/singa/blob/master/examples/cnn/model/resnet.py)
- GPU: NVIDIA RTX 2080Ti \* 2
- MPI: two MPI processes on one node
@@ -445,7 +445,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0000</td>
</tr>
<tr>
- <td nowrap>module:disable graph</td>
+ <td nowrap>model:disable graph</td>
<td>5427</td>
<td>17.8232</td>
<td>11.2213</td>
@@ -454,7 +454,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>0.9725</td>
</tr>
<tr>
- <td nowrap>module:enable graph, bfs</td>
+ <td nowrap>model:enable graph, bfs</td>
<td>3389</td>
<td>18.2310</td>
<td>10.9703</td>
@@ -463,7 +463,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>0.9507</td>
</tr>
<tr>
- <td nowrap>module:enable graph, serial</td>
+ <td nowrap>model:enable graph, serial</td>
<td>3437</td>
<td>17.0389</td>
<td>11.7378</td>
@@ -482,7 +482,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0000</td>
</tr>
<tr>
- <td nowrap>module:disable graph</td>
+ <td nowrap>model:disable graph</td>
<td>10503</td>
<td>14.7746</td>
<td>6.7684</td>
@@ -491,7 +491,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0060</td>
</tr>
<tr>
- <td nowrap>module:enable graph, bfs</td>
+ <td nowrap>model:enable graph, bfs</td>
<td>6935</td>
<td>14.8553</td>
<td>6.7316</td>
@@ -500,7 +500,7 @@ Tensor GpuConvBackwardx(const Tensor &dy, const Tensor &W, const Tensor &x,
<td>1.0006</td>
</tr>
<tr>
- <td nowrap>module:enable graph, serial</td>
+ <td nowrap>model:enable graph, serial</td>
<td>7027</td>
<td>14.3271</td>
<td>6.9798</td>
diff --git a/docs-site/docs/installation.md b/docs-site/docs/installation.md
index 48e44d1..64af47e 100644
--- a/docs-site/docs/installation.md
+++ b/docs-site/docs/installation.md
@@ -65,7 +65,7 @@ pip install singa -f http://singa.apache.org/docs/next/wheel-cpu.html --trusted-
```
You can install a specific version of SINGA via `singa==<version>`, where the
-`<version>` field should be replaced, e.g., `3.0.0`. The available SINGA
+`<version>` field should be replaced, e.g., `3.1.0`. The available SINGA
versions are listed at the link.
To install the latest develop version, replace the link with
@@ -78,7 +78,7 @@ pip install singa -f http://singa.apache.org/docs/next/wheel-cuda.html --trusted
```
You can also configure SINGA version and the CUDA version, like
-`singa==3.0.0+cuda10.2`. The available combinations of SINGA version and CUDA
+`singa==3.1.0+cuda10.2`. The available combinations of SINGA version and CUDA
version are listed at the link.
To install the latest develop version, replace the link with
diff --git a/docs-site/docs/software-stack.md b/docs-site/docs/software-stack.md
index c4244f0..620ed61 100644
--- a/docs-site/docs/software-stack.md
+++ b/docs-site/docs/software-stack.md
@@ -12,7 +12,19 @@ learning models, hardware abstractions for scheduling and executing operations,
and communication components for distributed training. The Python interface
wraps some CPP data structures and provides additional high-level classes for
neural network training, which makes it convenient to implement complex neural
-network models. Next, we introduce the software stack in a bottom-up manner.
+network models.
+
+SINGA's programming model enjoys the advantages of imperative programming and
+declarative programming. Users define the network structure and the training
+procedure (data flow) via imperative programming like PyTorch.
+Different to PyTorch which recreates the operations in every iteration, SINGA
+buffers the operations to create a computational graph implicitly (when this
+feature is enabled) after the first iteration. The graph is similar to that
+created by libraries using declarative programming, e.g., TensorFlow. Therefore,
+SINGA can apply the memory and speed optimization techniques over the
+computational graph.
+
+Next, we introduce the software stack in a bottom-up manner.
 <br/> **Figure 1 - SINGA V3
software stack.**
@@ -126,11 +138,11 @@ backward functions automatically in the reverse order. All functions can be
buffered by the `Scheduler` to create a [computational graph](./graph) for
efficiency and memory optimization.
-### Module
+### Model
-`Module` provides an easy interface to implement new network models. You just
-need to inherit `Module` and define the forward propagation of the model by
-creating and calling the layers or operators. `Module` will do autograd and
+[Model](./graph) provides an easy interface to implement new network models. You
+just need to inherit `Model` and define the forward propagation of the model by
+creating and calling the layers or operators. `Model` will do autograd and
update the parameters via `Opt` automatically when training data is fed into it.
### ONNX
[singa-doc] 01/07: add the page for installation using pip and wheel
Posted by wa...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
wangwei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/singa-doc.git
commit 089110accaf6f8847a615244cc8771929851dd8b
Author: wang wei <wa...@gmail.com>
AuthorDate: Mon Jul 20 13:31:05 2020 +0800
add the page for installation using pip and wheel
---
docs-site/docs/wheel.md | 50 +++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 50 insertions(+)
diff --git a/docs-site/docs/wheel.md b/docs-site/docs/wheel.md
new file mode 100644
index 0000000..47259e8
--- /dev/null
+++ b/docs-site/docs/wheel.md
@@ -0,0 +1,50 @@
+---
+id: wheel
+title: Wheel
+---
+
+<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agree [...]
+
+## Installation Instructions
+
+The wheel package of SINGA is installed via `pip`. Depending on the hardware
+environment, here are two ways to install the wheel package.
+
+- CPU only
+
+```bash
+pip install singa-<version> -f http://singa.apache.org/docs/next/wheel.html
+```
+
+The `<version>` field should be replaced, e.g., `3.0.0.dev1`. The available
+SINGA versions are listed below.
+
+- With CUDA and cuDNN
+
+```bash
+pip install singa-<version>+cuda<cu_version> -f http://singa.apache.org/docs/next/wheel.html
+```
+
+The `<version>` field should be replaced with specific SINGA version; The
+`<cu_version>` field should be replaced with specific CUDA version; For example,
+`singa-3.0.0.dev1+cuda10.2` is the full version specification for one package.
+The available combinations of SINGA version and CUDA version are listed below.
+
+Note: the Python version of your local Python environment will be used to find
+the corresponding wheel package. For example, if your local Python is 3.6, then
+the wheel package compiled on Python 3.6 will be selected by pip and installed.
+In fact, the wheel file's name include SINGA version, CUDA version and Python
+version. Therefore, `pip` knows which wheel file to download and install.
+
+## Building Instructions
+
+Refer to the comments at the top of the `setup.py` file.
+
+## SINGA-3.0.0.dev1
+
+- [CPU only, Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp36-cp36m-manylinux2014_x86_64.whl)
+- [CPU only, Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp37-cp37m-manylinux2014_x86_64.whl)
+- [CPU only, Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp38-cp38-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.6](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1%2Bcuda10.2-cp36-cp36m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.7](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp37-cp37m-manylinux2014_x86_64.whl)
+- [CUDA10.2, cuDNN 7.6.5, Python 3.8](https://singa-wheel.s3-ap-southeast-1.amazonaws.com/singa-3.0.0.dev1-cp38-cp38-manylinux2014_x86_64.whl)