[GitHub] xzqjack opened a new issue #7444: define a new Parametrized symbol layer and how to use (bind, init, set learning rate ) it?

[gi...@git.apache.org]

xzqjack opened a new issue #7444: define a new Parametrized symbol layer and  how to use (bind, init, set learning rate ) it?
URL: https://github.com/apache/incubator-mxnet/issues/7444
 
 
   I have been learning to define Parametrized layer, of which parameters will be leared during training time. I start by defining a fc layer with reference to 
   1. [https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/gluon/customop.md"](https://github.com/apache/incubator-mxnet/blob/master/docs/tutorials/gluon/customop.md");
   2. [https://github.com/apache/incubator-mxnet/blob/251ae71a20d8318ab20f4c19520f74b881fdf3ff/example/rcnn/rcnn/symbol/proposal.py](https://github.com/apache/incubator-mxnet/blob/251ae71a20d8318ab20f4c19520f74b881fdf3ff/example/rcnn/rcnn/symbol/proposal.py)
   3. [https://github.com/apache/incubator-mxnet/blob/251ae71a20d8318ab20f4c19520f74b881fdf3ff/example/reinforcement-learning/dqn/operators.py]()
   
   The new parameterized layer definition seems OK, but i dont know how to use it. Eg, how to bind the symbol and how to set myFC parameters learning rate. Here is my code
   ```
   
   import mxnet as mx
   import logging
   from data_iter import get_mnist_iter
   logging.basicConfig(level=logging.INFO)
   
   
   class MyFc(mx.operator.CustomOpProp):
       def __init__ (self):
           super(MyFc, self).__init__()
   
       def forward(self, is_train, req, in_data, out_data, aut):
           x = in_data[0]
           w = in_data[1]
           b = in_data[2]
           y = mx.nd.batch_dot(x, w.T) + b
           self.assign(out_data[0], req[0], y)
   
       def backward(self, out_grad, in_data, out_data, in_grad):
           dy = out_grad[0]
           w = in_data[1]
           x = in_data[0]
           b = in_data[2]
           dx = mx.nd.batch_dot(dy, w)
           self.assign(in_grad[0], req[0], dx)
           self.assing(in_grad[1], req[0], mx.nd.batch_dot(dy, x))
           self.assign(in_grad[2], req[0], mx.nd.ones(shape=b.shape))
   
   @mx.operator.register("myFc")
   class MyFcProp(mx.operator.CustomOpProp):
       def __init__(self):
           super(MyFcProp, self).__init__(need_top_grad=True)
   
       def list_arguments(self):
           return ['data', 'w', 'b']
   
       def list_outputs(self):
           return ['output']
   
       def infer_shape(self, in_shape):
           data_shape = in_shape[0]
           w_shape = in_shape[1]
           b_shape = in_shape[2]
           output_shape = (data_shape[0], 10)
           return [data_shape, w_shape, b_shape], [output_shape],[]
   
       def create_operator(self, ctx, shapes, dtypes):
           return MyFc()
   
   def get_symbol(num_classes=10, add_stn=False, perturbation_flag=False, **kwargs):
       data = mx.symbol.Variable('data')
       weight_2 = mx.sym.Variable('custom0_w', shape=(10, 200))
       bias_2 = mx.symbol.Variable('custom0_b', shape=(10, ))
       if add_stn:
           data = mx.sym.SpatialTransformer(data=data, loc=get_loc(data), target_shape = (28,28),
                                            transform_type="affine", sampler_type="bilinear")
       # first conv
       # conv1 = mx.symbol.Convolution(data=data, kernel=(5,5), num_filter=20)
       conv1 = conv_pertub(data=data, kernel=(5,5), num_filter=20, perturbation_flag=False)
       tanh1 = mx.symbol.Activation(data=conv1, act_type="tanh")
       pool1 = mx.symbol.Pooling(data=tanh1, pool_type="max",
                                 kernel=(2,2), stride=(2,2))
       # second conv
       conv2 = mx.symbol.Convolution(data=pool1, kernel=(5,5), num_filter=50)
       tanh2 = mx.symbol.Activation(data=conv2, act_type="tanh")
       pool2 = mx.symbol.Pooling(data=tanh2, pool_type="max",
                                 kernel=(2,2), stride=(2,2))
       # first fullc
       flatten = mx.symbol.Flatten(data=pool2)
       # fc1 = mx.symbol.Custom(data=flatten, )
       fc1 = fc_pertub(data=flatten, num_hidden=500, perturbation_flag=False)
       tanh3 = mx.symbol.Activation(data=fc1, act_type="tanh")
       # second fullc
       fc2 = mx.symbol.Custom(data=tanh3, op_type="myFc")
       # loss
       lenet = mx.symbol.SoftmaxOutput(data=fc2, name='softmax')
       return lenet
   
   num_classes = 10
   perturbation_flag = True
   batch_size = 5
   net = get_symbol(num_classes, add_stn=False, perturbation_flag=perturbation_flag)
   train, val = get_mnist_iter(batch_size=batch_size )
   print train.provide_label
   mod = mx.mod.Module(net, data_names=['data','custom0_w','custom0_b'])
   mod.bind(data_shapes=[DataDesc('data', (batch_size ,1,28,28)), DataDesc('custom0_w', (10, 200)), DataDesc('custom0_b',(10,))],
                           label_shapes=train.provide_label)
   ```
   
   ## What have you tried to solve it?
   
   1. `a, b, c = net.infer_shape(data=(5,1,28,28), custom0_w=(10, 200), custom0_b=(10,))` infer_shape operation is ok.
   
   Can anyone show me how to use (bind, init, set learning rate) a new parameterizd layer ? Thanks.
   
   
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services