[GitHub] aaronmarkham commented on a change in pull request #12340: Add a tutorial for control flow operators.

[GitBox <gi...@apache.org>]

aaronmarkham commented on a change in pull request #12340: Add a tutorial for control flow operators.
URL: https://github.com/apache/incubator-mxnet/pull/12340#discussion_r212778901
 
 

 ##########
 File path: docs/tutorials/control_flow/ControlFlowTutorial.md
 ##########
 @@ -0,0 +1,386 @@
+
+MXNet currently provides three control flow operators: `cond`, `foreach` and `while_loop`. Like other MXNet operators, they all have a version for NDArray and a version for Symbol. These two versions have exactly the same semantics. We can take advantage of this and use them in Gluon to hybridize models.
+
+In this tutorial, we use a few examples to demonstrate the use of control flow operators in Gluon and show how a model that requires control flow is hybridized.
+
+# Prepare running the code
+
+
+```python
+import mxnet as mx
+from mxnet.gluon import HybridBlock
+```
+
+# foreach
+`foreach` is defined with the following signature:
+
+```python
+foreach(body, data, init_states, name) => (outputs, states)
+```
+
+It iterates over the first dimension of the input data (it can be an array or a list of arrays) and run the Python function defined in `body` for every slice from the input arrays. The signature of the `body` function is defined as follows:
+
+```python
+body(data, states) => (outputs, states)
+```
+
+The inputs of the `body` function have two parts: `data` is a slice of an array (if there is only one input array in `foreach`) or a list of slices (if there are a list of input arrays); `states` are the arrays from the previous iteration. The outputs of the `body` function also have two parts: `outputs` is an array or a list of arrays; `states` is the computation states of the current iteration. `outputs` from all iterations are concatenated as the outputs of `foreach`.
+
+The pseudocode below illustrates the execution of `foreach`.
+
+```python
+def foreach(body, data, init_states):
+    states = init_states
+    outs = []
+
+    for i in range(data.shape[0]):
+        s = data[i]
+        out, states = body(s, states)
+        outs.append(out)
+    outs = mx.nd.stack(*outs)
+    return outs, states
+```
+
+### Example 1: foreach works like map
+`foreach` can work like a map function of a functional language. In this case, the states of foreach can be an empty list, which means the computation doesn't carry computation states across iterations.
+
+In this example, we use `foreach` to add each element in an array by one.
+
+
+```python
+data = mx.nd.arange(5)
+print(data)
+```
+
+    
+    [ 0.  1.  2.  3.  4.]
+    <NDArray 5 @cpu(0)>
+
+
+
+```python
+def add1(data, _):
+    return data + 1, []
+
+class Map(HybridBlock):
+    def hybrid_forward(self, F, data):
+        out, _ = F.contrib.foreach(add1, data, [])
+        return out
+    
+map_layer = Map()
+out = map_layer(data)
+print(out)
+```
+
+    
+    [[ 1.]
+     [ 2.]
+     [ 3.]
+     [ 4.]
+     [ 5.]]
+    <NDArray 5x1 @cpu(0)>
+
+
+We can hybridize the block and run the computation again. It should generate the same result.
+
+
+```python
+map_layer.hybridize()
+out = map_layer(data)
+print(out)
+```
+
+    
+    [[ 1.]
+     [ 2.]
+     [ 3.]
+     [ 4.]
+     [ 5.]]
+    <NDArray 5x1 @cpu(0)>
+
+
+### Example 2: foreach works like scan
+`foreach` can work like a scan function in a functional language. In this case, the outputs of the Python function is an empty list.
+
+
+```python
+def sum(data, state):
+    return [], state + data
+
+class Scan(HybridBlock):
+    def hybrid_forward(self, F, data):
+        _, state = F.contrib.foreach(sum, data, F.zeros((1)))
+        return state
+scan_layer = Scan()
+state = scan_layer(data)
+print(data)
+print(state)
+```
+
+    
+    [ 0.  1.  2.  3.  4.]
+    <NDArray 5 @cpu(0)>
+    
+    [ 10.]
+    <NDArray 1 @cpu(0)>
+
+
+
+```python
+scan_layer.hybridize()
+state = scan_layer(data)
+print(state)
+```
+
+    
+    [ 10.]
+    <NDArray 1 @cpu(0)>
+
+
+### Example 3: foreach with both outputs and states
+This is probably the most common use case of `foreach`. We extend the scan example above and return both output and states.
+
+
+```python
+def sum(data, state):
+    return state + data, state + data
+
+class ScanV2(HybridBlock):
+    def hybrid_forward(self, F, data):
+        out, state = F.contrib.foreach(sum, data, F.zeros((1)))
+        return out, state
+scan_layer = ScanV2()
+out, state = scan_layer(data)
+print(out)
+print(state)
+```
+
+    
+    [[  0.]
+     [  1.]
+     [  3.]
+     [  6.]
+     [ 10.]]
+    <NDArray 5x1 @cpu(0)>
+    
+    [ 10.]
+    <NDArray 1 @cpu(0)>
+
+
+
+```python
+scan_layer.hybridize()
+out, state = scan_layer(data)
+print(out)
+print(state)
+```
+
+    
+    [[  0.]
+     [  1.]
+     [  3.]
+     [  6.]
+     [ 10.]]
+    <NDArray 5x1 @cpu(0)>
+    
+    [ 10.]
+    <NDArray 1 @cpu(0)>
+
+
+### Example 4: use foreach to run RNN on a variable-length sequence
+Previous examples illustrate `foreach` with simple use cases. Here I show an example of processing variable-length sequences with `foreach`. The same idea is used by dynamic_rnn in TensorFlow for processing variable-length sequences.
 
 Review comment:
   `dynamic_rnn`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services