You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@singa.apache.org by GitBox <gi...@apache.org> on 2020/03/30 10:08:20 UTC
[GitHub] [singa-doc] joddiy opened a new pull request #7: add autograd doc
joddiy opened a new pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7
Add doc for new operators
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403456022
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,863 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported Operations
Review comment:
operation list here is different from autograd, we do not support all operators in the autograd.
By the way, I reformat it now, I guess maybe more clear now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403457401
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,863 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported Operations
Review comment:
and, the name of operators may be different between ONNX and SINGA, the list here is ONNX operators' name.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on a change in pull request #7: add
onnx doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #7: add onnx doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403682215
##########
File path: docs-site/docs/onnx.md
##########
@@ -391,178 +535,227 @@ autograd.training = False
test(new_model, valid_x, valid_y, dev=dev)
```
-## Example: ONNX tiny_yolov2 on singa
+## ONNX model zoo
-Now, the onnx of Singa supports importing models from
-[Onnx Model Zoo](https://github.com/onnx/models). We will show you how to
-inmport a Tiny-Yolo-V2 model and verify the correctness of the model by using
-its test dataset.
+The [ONNX Model Zoo](https://github.com/onnx/models) is a collection of
+pre-trained, state-of-the-art models in the ONNX format contributed by community
+members. SINGA has supported several CV and NLP models now. More models are
+going to be supported soon.
-### Load model
+### Image Classification
-Firstly, we try to download the Tiny-Yolo-V2 model from the Onnx Model Zoo if it
-doesn't exist already, and then load this model:
-
-```python
-def load_model():
- url = 'https://onnxzoo.blob.core.windows.net/models/opset_8/tiny_yolov2/tiny_yolov2.tar.gz'
- download_dir = '/tmp/'
- filename = os.path.join(download_dir, 'tiny_yolov2', '.', 'Model.onnx')
- with tarfile.open(check_exist_or_download(url), 'r') as t:
- t.extractall(path=download_dir)
- return filename
-
-def check_exist_or_download(url):
- download_dir = '/tmp/'
- name = url.rsplit('/', 1)[-1]
- filename = os.path.join(download_dir, name)
- if not os.path.isfile(filename):
- print("Downloading %s" % url)
- urllib.request.urlretrieve(url, filename)
- return filename
+This collection of models take images as input, then classifies the major
+objects in the images into 1000 object categories such as keyboard, mouse,
+pencil, and many animals.
-dev = device.create_cuda_gpu()
-model_path = load_model()
-onnx_model = onnx.load(model_path)
-```
+| Model Class | Reference | Description | Link |
+| ---------------------------------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[MobileNet](https://github.com/onnx/models/tree/master/vision/classification/mobilenet)</b> | [Sandler et al.](https://arxiv.org/abs/1801.04381) | Light-weight deep neural network best suited for mobile and embedded vision applications. <br>Top-5 error from paper - ~10% | [](https://colab.research.google.com/drive/1HsixqJMIpKyEPhkbB8jy7NwNEFEAUWAf) |
+| <b>[ResNet18](https://github.com/onnx/models/tree/master/vision/classification/resnet)</b> | [He et al.](https://arxiv.org/abs/1512.03385) | A CNN model (up to 152 layers). Uses shortcut connections to achieve higher accuracy when classifying images. <br> Top-5 error from paper - ~3.6% | [](https://colab.research.google.com/drive/1u1RYefSsVbiP4I-5wiBKHjsT9L0FxLm9) |
+| <b>[VGG16](https://github.com/onnx/models/tree/master/vision/classification/vgg)</b> | [Simonyan et al.](https://arxiv.org/abs/1409.1556) | Deep CNN model(up to 19 layers). Similar to AlexNet but uses multiple smaller kernel-sized filters that provides more accuracy when classifying images. <br>Top-5 error from paper - ~8% | [](https://colab.research.google.com/drive/14kxgRKtbjPCKKsDJVNi3AvTev81Gp_Ds) |
-### Set batchsize and prepare model
+### Object Detection
-Then since lots of example models don't indicate its batch size, we need to
-update it. After that, we can parse the onnx model into singa model:
+Object detection models detect the presence of multiple objects in an image and
+segment out areas of the image where the objects are detected.
-```python
-def update_batch_size(onnx_model, batch_size):
- model_input = onnx_model.graph.input[0]
- model_input.type.tensor_type.shape.dim[0].dim_value = batch_size
- return onnx_model
+| Model Class | Reference | Description | Link |
+| ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[Tiny YOLOv2](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/tiny_yolov2)</b> | [Redmon et al.](https://arxiv.org/pdf/1612.08242.pdf) | A real-time CNN for object detection that detects 20 different classes. A smaller version of the more complex full YOLOv2 network. | [](https://colab.research.google.com/drive/11V4I6cRjIJNUv5ZGsEGwqHuoQEie6b1T) |
-# set batch size
-onnx_model = update_batch_size(onnx_model, 1)
-sg_ir = sonnx.prepare(onnx_model, device=dev)
-```
+### Face Analysis
-### Define inference
+Face detection models identify and/or recognize human faces and emotions in
+given images.
-For clearness, we define a Infer functin to hold the model's forward process:
+| Model Class | Reference | Description | Link |
+| --------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[ArcFace](https://github.com/onnx/models/tree/master/vision/body_analysis/arcface)</b> | [Deng et al.](https://arxiv.org/abs/1801.07698) | A CNN based model for face recognition which learns discriminative features of faces and produces embeddings for input face images. | [](https://colab.research.google.com/drive/1qanaqUKGIDtifdzEzJOHjEj4kYzA9uJC) |
+| <b>[Emotion FerPlus](https://github.com/onnx/models/tree/master/vision/body_analysis/emotion_ferplus)</b> | [Barsoum et al.](https://arxiv.org/abs/1608.01041) | Deep CNN for emotion recognition trained on images of faces. | [](https://colab.research.google.com/drive/1XHtBQGRhe58PDi4LGYJzYueWBeWbO23r) |
-```python
-class Infer:
- def __init__(self, sg_ir):
- self.sg_ir = sg_ir
- for idx, tens in sg_ir.tensor_map.items():
- # allow the tensors to be updated
- tens.requires_grad = True
- tens.stores_grad = True
- sg_ir.tensor_map[idx] = tens
+### Machine Comprehension
- def forward(self, x):
- return sg_ir.run([x])[0]
+This subset of natural language processing models that answer questions about a
+given context paragraph.
-# inference
-autograd.training = False
-model = Infer(sg_ir)
-```
+| Model Class | Reference | Description | Link |
+| ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[BERT-Squad](https://github.com/onnx/models/tree/master/text/machine_comprehension/bert-squad)</b> | [Devlin et al.](https://arxiv.org/pdf/1810.04805.pdf) | This model answers questions based on the context of the given input paragraph. | [](https://colab.research.google.com/drive/1kud-lUPjS_u-TkDAzihBTw0Vqr0FjCE-) |
-### Load dataset, run and verify
+## Supported operators
-Finally, we load the test dataset which is provided by Onnx Model Zoo, do the
-inference and verify its correctness.
+The following operators are supported:
-```python
-def load_dataset(test_data_dir):
- # Load inputs
- inputs = []
- inputs_num = len(glob.glob(os.path.join(test_data_dir, 'input_*.pb')))
- for i in range(inputs_num):
- input_file = os.path.join(test_data_dir, 'input_{}.pb'.format(i))
- tensor = onnx.TensorProto()
- with open(input_file, 'rb') as f:
- tensor.ParseFromString(f.read())
- inputs.append(numpy_helper.to_array(tensor))
-
- # Load reference outputs
- ref_outputs = []
- ref_outputs_num = len(glob.glob(os.path.join(test_data_dir, 'output_*.pb')))
- for i in range(ref_outputs_num):
- output_file = os.path.join(test_data_dir, 'output_{}.pb'.format(i))
- tensor = onnx.TensorProto()
- with open(output_file, 'rb') as f:
- tensor.ParseFromString(f.read())
- ref_outputs.append(numpy_helper.to_array(tensor))
- return inputs, ref_outputs
-
-inputs, ref_outputs = load_dataset(os.path.join('/tmp', 'tiny_yolov2', 'test_data_set_0'))
-x_batch = tensor.Tensor(device=dev, data=inputs[0])
-outputs = model.forward(x_batch)
-
-# Compare the results with reference outputs.
-for ref_o, o in zip(ref_outputs, outputs):
- np.testing.assert_almost_equal(ref_o, o)
-```
+- Conv
+- Relu
+- Constant
+- MaxPool
+- AveragePool
+- Softmax
+- Sigmoid
+- Add
+- MatMul
+- BatchNormalization
+- Concat
+- Flatten
+- Add
+- Gemm
+- Reshape
+- Sum
+- Cos
+- Cosh
+- Sin
+- Sinh
+- Tan
+- Tanh
+- Acos
+- Acosh
+- Asin
+- Asinh
+- Atan
+- Atanh
+- Selu
+- Elu
+- Equal
+- Less
+- Sign
+- Div
+- Sub
+- Sqrt
+- Log
+- Greater
+- HardSigmoid
+- Identity
+- Softplus
+- Softsign
+- Mean
+- Pow
+- Clip
+- PRelu
+- Mul
+- Transpose
+- Max
+- Min
+- Shape
+- And
+- Or
+- Xor
+- Not
+- Neg
+- Reciprocal
+- LeakyRelu
+- GlobalAveragePool
+- ConstantOfShape
+- Dropout
+- ReduceSum
+- ReduceMean
+- LeakyRelu
+- GlobalAveragePool
+- Squeeze
+- Unsqueeze
+- Slice
+- Ceil
+- Split
+- Gather
+- Tile
+- NonZero
+- Cast
+- OneHot
+
+### Special comments for ONNX backend
+
+- Conv, MaxPool and AveragePool
+
+ Input must be 1d`(N*C*H)` and 2d(`N*C*H*W`) shape and `dilation` must be 1.
+
+- BatchNormalization
+
+ `epsilon` is 1e-05 and cannot be changed.
+
+- Cast
+
+ Only support float32 and int32, other types are casted to these two types.
+
+- Squeeze and Unsqueeze
+
+ If you encounter errors when you `Squeeze` or `Unsqueeze` between `Tensor` and
+ Scalar, please report to us.
+
+- Empty tensor Empty tensor is illegal in SINGA.
+
+## Implementation
+
+The code of SINGA ONNX locates at `python/singa/soonx.py`. There are three main
+class, `SingaFrontend` and `SingaBackend` and `SingaRep`. `SingaFrontend`
+translates a SINGA model to ONNX model; `SingaBackend` translates a ONNX model
+to `SingaRep` object which stores all SINGA operators and tensors(the tensor in
+this doc means SINGA `Tensor`); `SingaRep` can be run like a SINGA model.
+
+### SingaFrontend
+
+The entry function of `SingaFrontend` is `singa_to_onnx_model` which also is
+called `to_onnx`. `singa_to_onnx_model` creates the ONNX model, and it also
+create a ONNX graph by using `singa_to_onnx_graph`.
+
+`singa_to_onnx_graph` accepts the output of the model, and recursively iterate
+the SINGA model's graph from the output to get all operators to form a queue.
+The input and intermediate tensors, i.e, trainable weights, of the SINGA model
+is picked up at the same time. The input is stored in `onnx_model.graph.input`;
+the output is stored in `onnx_model.graph.output`; and the trainable weights are
+stored in `onnx_model.graph.initializer`.
+
+Then the SINGA operator in the queue is translated to ONNX operators one by one.
+`_rename_operators` defines the operators name mapping between SINGA and ONNX.
+`_special_operators` defines which function to be used to translate the
+operator.
+
+In addition, some operators in SINGA has different definition with ONNX, that
+is, ONNX regards some attributes of SINGA operators as input, so
+`_unhandled_operators` defines which function to handle the special operator.
+
+Since the bool type is regarded as int32 in SINGA, `_bool_operators` defines the
+operators to be changed as bool type.
+
+### SingaBackend
+
+The entry function of `SingaBackend` is `prepare` which checks the version of
+ONNX model and call `_onnx_model_to_singa_net` then.
+
+The purpose of `_onnx_model_to_singa_net` is to get SINGA tensors and operators.
+The tensors are stored in a dictionary by their name in ONNX, and operators are
+stored in queue by the form of
+`namedtuple('SingaOps', ['name', 'op', 'handle', 'forward'])`. For each
+operator, `name` is its ONNX node name; `op` is the ONNX node; `forward` is the
+SINGA operator's forward function; `handle` is prepared for some special
+operators such as Conv and Pooling which has `handle` object.
+
+The first step of `_onnx_model_to_singa_net` is to call `_init_graph_parameter`
+to get all tensors within the model. For trainable weights, it can init SINGA
+`Tensor` from `onnx_model.graph.initializer`. Please note, the weights may also
+be stored within graph's input or a ONNX node called `Constant`, SINGA can also
+handle these.
+
+Though all weights are stored within ONNX model, the input of the model is
+unknown but its shape and type. So SINGA support two ways to init input, 1,
+generate random tensor by its shape and type, 2, allow the user to assign the
+input. The first way works fine for most models, however, for some model such as
+bert, the indices of matrix cannot be random generated otherwise it will incurs
+errors.
+
+Then, `_onnx_model_to_singa_net` iterators all nodes within ONNX graph to
+translate it to SIGNA operators. Also, `_rename_operators` defines the operators
+name mapping between SINGA and ONNX. `_special_operators` defines which function
+to be used to translate the operator. `_run_node` runs the generated SINGA model
+by its input tensors and store its output tensors for being used by later
+operators.
-## Supported operators
+This class finally return a `SingaRep` object and stores all SINGA tensors and
+operators within it.
-The following operators are supported:
+### SingaRep
-| Operation | Comments |
-| ------------------ | ----------------------------------------- |
Review comment:
I didn't delete it, just sperate it to two parts, the operator list (L589) and some special operator's comment(L665).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] nudles commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r400205383
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,978 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported operators
+
+#### mean
Review comment:
Copy the comments as the docString for each operator in the python file?
Then we can generate the html for the APIs automatically using Sphinx.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] nudles commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r402965156
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,863 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported Operations
Review comment:
if the descriptions of the operations are already included in the docString, then there is no need to list them here.
Otherwise, we have to maintain both the docString and this file when there is a change to any API.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r402756821
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,978 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported operators
+
+#### mean
+
+Init a element-wise Mean operator.
+
+##### Input
Review comment:
I add a `---` to separate them, it'd be better now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] nudles commented on a change in pull request #7: add
onnx doc
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #7: add onnx doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403645611
##########
File path: docs-site/docs/onnx.md
##########
@@ -391,178 +535,227 @@ autograd.training = False
test(new_model, valid_x, valid_y, dev=dev)
```
-## Example: ONNX tiny_yolov2 on singa
+## ONNX model zoo
-Now, the onnx of Singa supports importing models from
-[Onnx Model Zoo](https://github.com/onnx/models). We will show you how to
-inmport a Tiny-Yolo-V2 model and verify the correctness of the model by using
-its test dataset.
+The [ONNX Model Zoo](https://github.com/onnx/models) is a collection of
+pre-trained, state-of-the-art models in the ONNX format contributed by community
+members. SINGA has supported several CV and NLP models now. More models are
+going to be supported soon.
-### Load model
+### Image Classification
-Firstly, we try to download the Tiny-Yolo-V2 model from the Onnx Model Zoo if it
-doesn't exist already, and then load this model:
-
-```python
-def load_model():
- url = 'https://onnxzoo.blob.core.windows.net/models/opset_8/tiny_yolov2/tiny_yolov2.tar.gz'
- download_dir = '/tmp/'
- filename = os.path.join(download_dir, 'tiny_yolov2', '.', 'Model.onnx')
- with tarfile.open(check_exist_or_download(url), 'r') as t:
- t.extractall(path=download_dir)
- return filename
-
-def check_exist_or_download(url):
- download_dir = '/tmp/'
- name = url.rsplit('/', 1)[-1]
- filename = os.path.join(download_dir, name)
- if not os.path.isfile(filename):
- print("Downloading %s" % url)
- urllib.request.urlretrieve(url, filename)
- return filename
+This collection of models take images as input, then classifies the major
+objects in the images into 1000 object categories such as keyboard, mouse,
+pencil, and many animals.
-dev = device.create_cuda_gpu()
-model_path = load_model()
-onnx_model = onnx.load(model_path)
-```
+| Model Class | Reference | Description | Link |
+| ---------------------------------------------------------------------------------------------- | -------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[MobileNet](https://github.com/onnx/models/tree/master/vision/classification/mobilenet)</b> | [Sandler et al.](https://arxiv.org/abs/1801.04381) | Light-weight deep neural network best suited for mobile and embedded vision applications. <br>Top-5 error from paper - ~10% | [](https://colab.research.google.com/drive/1HsixqJMIpKyEPhkbB8jy7NwNEFEAUWAf) |
+| <b>[ResNet18](https://github.com/onnx/models/tree/master/vision/classification/resnet)</b> | [He et al.](https://arxiv.org/abs/1512.03385) | A CNN model (up to 152 layers). Uses shortcut connections to achieve higher accuracy when classifying images. <br> Top-5 error from paper - ~3.6% | [](https://colab.research.google.com/drive/1u1RYefSsVbiP4I-5wiBKHjsT9L0FxLm9) |
+| <b>[VGG16](https://github.com/onnx/models/tree/master/vision/classification/vgg)</b> | [Simonyan et al.](https://arxiv.org/abs/1409.1556) | Deep CNN model(up to 19 layers). Similar to AlexNet but uses multiple smaller kernel-sized filters that provides more accuracy when classifying images. <br>Top-5 error from paper - ~8% | [](https://colab.research.google.com/drive/14kxgRKtbjPCKKsDJVNi3AvTev81Gp_Ds) |
-### Set batchsize and prepare model
+### Object Detection
-Then since lots of example models don't indicate its batch size, we need to
-update it. After that, we can parse the onnx model into singa model:
+Object detection models detect the presence of multiple objects in an image and
+segment out areas of the image where the objects are detected.
-```python
-def update_batch_size(onnx_model, batch_size):
- model_input = onnx_model.graph.input[0]
- model_input.type.tensor_type.shape.dim[0].dim_value = batch_size
- return onnx_model
+| Model Class | Reference | Description | Link |
+| ----------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[Tiny YOLOv2](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/tiny_yolov2)</b> | [Redmon et al.](https://arxiv.org/pdf/1612.08242.pdf) | A real-time CNN for object detection that detects 20 different classes. A smaller version of the more complex full YOLOv2 network. | [](https://colab.research.google.com/drive/11V4I6cRjIJNUv5ZGsEGwqHuoQEie6b1T) |
-# set batch size
-onnx_model = update_batch_size(onnx_model, 1)
-sg_ir = sonnx.prepare(onnx_model, device=dev)
-```
+### Face Analysis
-### Define inference
+Face detection models identify and/or recognize human faces and emotions in
+given images.
-For clearness, we define a Infer functin to hold the model's forward process:
+| Model Class | Reference | Description | Link |
+| --------------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[ArcFace](https://github.com/onnx/models/tree/master/vision/body_analysis/arcface)</b> | [Deng et al.](https://arxiv.org/abs/1801.07698) | A CNN based model for face recognition which learns discriminative features of faces and produces embeddings for input face images. | [](https://colab.research.google.com/drive/1qanaqUKGIDtifdzEzJOHjEj4kYzA9uJC) |
+| <b>[Emotion FerPlus](https://github.com/onnx/models/tree/master/vision/body_analysis/emotion_ferplus)</b> | [Barsoum et al.](https://arxiv.org/abs/1608.01041) | Deep CNN for emotion recognition trained on images of faces. | [](https://colab.research.google.com/drive/1XHtBQGRhe58PDi4LGYJzYueWBeWbO23r) |
-```python
-class Infer:
- def __init__(self, sg_ir):
- self.sg_ir = sg_ir
- for idx, tens in sg_ir.tensor_map.items():
- # allow the tensors to be updated
- tens.requires_grad = True
- tens.stores_grad = True
- sg_ir.tensor_map[idx] = tens
+### Machine Comprehension
- def forward(self, x):
- return sg_ir.run([x])[0]
+This subset of natural language processing models that answer questions about a
+given context paragraph.
-# inference
-autograd.training = False
-model = Infer(sg_ir)
-```
+| Model Class | Reference | Description | Link |
+| ----------------------------------------------------------------------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| <b>[BERT-Squad](https://github.com/onnx/models/tree/master/text/machine_comprehension/bert-squad)</b> | [Devlin et al.](https://arxiv.org/pdf/1810.04805.pdf) | This model answers questions based on the context of the given input paragraph. | [](https://colab.research.google.com/drive/1kud-lUPjS_u-TkDAzihBTw0Vqr0FjCE-) |
-### Load dataset, run and verify
+## Supported operators
-Finally, we load the test dataset which is provided by Onnx Model Zoo, do the
-inference and verify its correctness.
+The following operators are supported:
-```python
-def load_dataset(test_data_dir):
- # Load inputs
- inputs = []
- inputs_num = len(glob.glob(os.path.join(test_data_dir, 'input_*.pb')))
- for i in range(inputs_num):
- input_file = os.path.join(test_data_dir, 'input_{}.pb'.format(i))
- tensor = onnx.TensorProto()
- with open(input_file, 'rb') as f:
- tensor.ParseFromString(f.read())
- inputs.append(numpy_helper.to_array(tensor))
-
- # Load reference outputs
- ref_outputs = []
- ref_outputs_num = len(glob.glob(os.path.join(test_data_dir, 'output_*.pb')))
- for i in range(ref_outputs_num):
- output_file = os.path.join(test_data_dir, 'output_{}.pb'.format(i))
- tensor = onnx.TensorProto()
- with open(output_file, 'rb') as f:
- tensor.ParseFromString(f.read())
- ref_outputs.append(numpy_helper.to_array(tensor))
- return inputs, ref_outputs
-
-inputs, ref_outputs = load_dataset(os.path.join('/tmp', 'tiny_yolov2', 'test_data_set_0'))
-x_batch = tensor.Tensor(device=dev, data=inputs[0])
-outputs = model.forward(x_batch)
-
-# Compare the results with reference outputs.
-for ref_o, o in zip(ref_outputs, outputs):
- np.testing.assert_almost_equal(ref_o, o)
-```
+- Conv
+- Relu
+- Constant
+- MaxPool
+- AveragePool
+- Softmax
+- Sigmoid
+- Add
+- MatMul
+- BatchNormalization
+- Concat
+- Flatten
+- Add
+- Gemm
+- Reshape
+- Sum
+- Cos
+- Cosh
+- Sin
+- Sinh
+- Tan
+- Tanh
+- Acos
+- Acosh
+- Asin
+- Asinh
+- Atan
+- Atanh
+- Selu
+- Elu
+- Equal
+- Less
+- Sign
+- Div
+- Sub
+- Sqrt
+- Log
+- Greater
+- HardSigmoid
+- Identity
+- Softplus
+- Softsign
+- Mean
+- Pow
+- Clip
+- PRelu
+- Mul
+- Transpose
+- Max
+- Min
+- Shape
+- And
+- Or
+- Xor
+- Not
+- Neg
+- Reciprocal
+- LeakyRelu
+- GlobalAveragePool
+- ConstantOfShape
+- Dropout
+- ReduceSum
+- ReduceMean
+- LeakyRelu
+- GlobalAveragePool
+- Squeeze
+- Unsqueeze
+- Slice
+- Ceil
+- Split
+- Gather
+- Tile
+- NonZero
+- Cast
+- OneHot
+
+### Special comments for ONNX backend
+
+- Conv, MaxPool and AveragePool
+
+ Input must be 1d`(N*C*H)` and 2d(`N*C*H*W`) shape and `dilation` must be 1.
+
+- BatchNormalization
+
+ `epsilon` is 1e-05 and cannot be changed.
+
+- Cast
+
+ Only support float32 and int32, other types are casted to these two types.
+
+- Squeeze and Unsqueeze
+
+ If you encounter errors when you `Squeeze` or `Unsqueeze` between `Tensor` and
+ Scalar, please report to us.
+
+- Empty tensor Empty tensor is illegal in SINGA.
+
+## Implementation
+
+The code of SINGA ONNX locates at `python/singa/soonx.py`. There are three main
+class, `SingaFrontend` and `SingaBackend` and `SingaRep`. `SingaFrontend`
+translates a SINGA model to ONNX model; `SingaBackend` translates a ONNX model
+to `SingaRep` object which stores all SINGA operators and tensors(the tensor in
+this doc means SINGA `Tensor`); `SingaRep` can be run like a SINGA model.
+
+### SingaFrontend
+
+The entry function of `SingaFrontend` is `singa_to_onnx_model` which also is
+called `to_onnx`. `singa_to_onnx_model` creates the ONNX model, and it also
+create a ONNX graph by using `singa_to_onnx_graph`.
+
+`singa_to_onnx_graph` accepts the output of the model, and recursively iterate
+the SINGA model's graph from the output to get all operators to form a queue.
+The input and intermediate tensors, i.e, trainable weights, of the SINGA model
+is picked up at the same time. The input is stored in `onnx_model.graph.input`;
+the output is stored in `onnx_model.graph.output`; and the trainable weights are
+stored in `onnx_model.graph.initializer`.
+
+Then the SINGA operator in the queue is translated to ONNX operators one by one.
+`_rename_operators` defines the operators name mapping between SINGA and ONNX.
+`_special_operators` defines which function to be used to translate the
+operator.
+
+In addition, some operators in SINGA has different definition with ONNX, that
+is, ONNX regards some attributes of SINGA operators as input, so
+`_unhandled_operators` defines which function to handle the special operator.
+
+Since the bool type is regarded as int32 in SINGA, `_bool_operators` defines the
+operators to be changed as bool type.
+
+### SingaBackend
+
+The entry function of `SingaBackend` is `prepare` which checks the version of
+ONNX model and call `_onnx_model_to_singa_net` then.
+
+The purpose of `_onnx_model_to_singa_net` is to get SINGA tensors and operators.
+The tensors are stored in a dictionary by their name in ONNX, and operators are
+stored in queue by the form of
+`namedtuple('SingaOps', ['name', 'op', 'handle', 'forward'])`. For each
+operator, `name` is its ONNX node name; `op` is the ONNX node; `forward` is the
+SINGA operator's forward function; `handle` is prepared for some special
+operators such as Conv and Pooling which has `handle` object.
+
+The first step of `_onnx_model_to_singa_net` is to call `_init_graph_parameter`
+to get all tensors within the model. For trainable weights, it can init SINGA
+`Tensor` from `onnx_model.graph.initializer`. Please note, the weights may also
+be stored within graph's input or a ONNX node called `Constant`, SINGA can also
+handle these.
+
+Though all weights are stored within ONNX model, the input of the model is
+unknown but its shape and type. So SINGA support two ways to init input, 1,
+generate random tensor by its shape and type, 2, allow the user to assign the
+input. The first way works fine for most models, however, for some model such as
+bert, the indices of matrix cannot be random generated otherwise it will incurs
+errors.
+
+Then, `_onnx_model_to_singa_net` iterators all nodes within ONNX graph to
+translate it to SIGNA operators. Also, `_rename_operators` defines the operators
+name mapping between SINGA and ONNX. `_special_operators` defines which function
+to be used to translate the operator. `_run_node` runs the generated SINGA model
+by its input tensors and store its output tensors for being used by later
+operators.
-## Supported operators
+This class finally return a `SingaRep` object and stores all SINGA tensors and
+operators within it.
-The following operators are supported:
+### SingaRep
-| Operation | Comments |
-| ------------------ | ----------------------------------------- |
Review comment:
are all onnx operators supported?
if not, we still need this list.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] nudles commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403473964
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,863 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported Operations
Review comment:
If these operations are onnx operators, then we just need to list these operators in onnx.md and refer to the onnx website for the descriptions.
This is autograd.md, which should only include descriptions about autograd.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on a change in pull request #7: add
onnx doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #7: add onnx doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403505370
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,978 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported operators
+
+#### mean
Review comment:
has fixed at https://github.com/apache/singa/pull/654
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r403499196
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,863 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported Operations
Review comment:
Sorry, I misunderstood what you said. I will delete this part and add a link to the API reference pages.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] joddiy commented on issue #7: add onnx doc
Posted by GitBox <gi...@apache.org>.
joddiy commented on issue #7: add onnx doc
URL: https://github.com/apache/singa-doc/pull/7#issuecomment-609425945
@nudles just add the Python API link, and ready to merge
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] nudles merged pull request #7: add onnx doc
Posted by GitBox <gi...@apache.org>.
nudles merged pull request #7: add onnx doc
URL: https://github.com/apache/singa-doc/pull/7
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [singa-doc] nudles commented on a change in pull request #7: add
autograd doc
Posted by GitBox <gi...@apache.org>.
nudles commented on a change in pull request #7: add autograd doc
URL: https://github.com/apache/singa-doc/pull/7#discussion_r400203424
##########
File path: docs-site/docs/autograd.md
##########
@@ -148,3 +148,978 @@ for epoch in range(epochs):
for p, gp in autograd.backward(loss): # auto backward
sgd.update(p, gp)
```
+
+### Supported operators
+
+#### mean
+
+Init a element-wise Mean operator.
+
+##### Input
Review comment:
remove '####'?
when I view this file on github, the `mean` and `Input` are of the same size. Not easy to read.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services