[GitHub] [incubator-mxnet] hassonofer opened a new issue #18164: Making _internal._plus_scalar and _internal._mul_scalar normal operators

[GitBox <gi...@apache.org>]

hassonofer opened a new issue #18164:
URL: https://github.com/apache/incubator-mxnet/issues/18164


   ## Description
   I suggest making 2 internal API's into symbol API.
   mx.sym._internal._plus_scalar and mx.sym._internal._mul_scalar.
   
   They are both already available in Gluon API.
   for example:
   ```python
   class Foo(mx.gluon.HybridBlock):
       def __init__(self):
           super().__init__()
   
       def hybrid_forward(self, F, x):
           return x + 6
   
   foo = Foo()
   foo.initialize(mx.cpu())
   foo.hybridize()
   data = mx.sym.Variable("data")
   print(foo(data).tojson())
   ```
   
   ```json
   {
     "nodes": [
       {
         "op": "null", 
         "name": "data", 
         "inputs": []
       }, 
       {
         "op": "_plus_scalar", 
         "name": "foo0__plusscalar0", 
         "attrs": {"scalar": "6"}, 
         "inputs": [[0, 0, 0]]
       }
     ], 
     "arg_nodes": [0], 
     "node_row_ptr": [0, 1, 2], 
     "heads": [[1, 0, 0]], 
     "attrs": {"mxnet_version": ["int", 10600]}
   }
   ```
   
   The _plus_scalar operator is been used.
   
   To my understanding they are required for ONNX support anyway.
   
   At the moment the only reasonable way to add scalar is either by adding him as an input or using those internal API's like:
   ```python
   x = mx.sym.Variable("x")
   x = mx.sym._internal._plus_scalar(x, 6)
   x = mx.sym._internal._mul_scalar(x, 2)
   ```
   
   Those operators are pretty basic requirements to implement end to end model[1] where the pre-processing is part of the model itself (added to an already trained model in my case).
   
   This will also make for easier and smoother creation of custom activation functions using symbol API, for example the "swish" implementation[2]
   
   ## References
   - [1] https://cwiki.apache.org/confluence/display/MXNET/MXNet+end+to+end+models
   - [2] https://beta.mxnet.io/_modules/mxnet/gluon/nn/activations.html#Swish
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-mxnet] hassonofer commented on issue #18164: Making _internal._plus_scalar and _internal._mul_scalar normal operators

[GitBox <gi...@apache.org>]

hassonofer commented on issue #18164:
URL: https://github.com/apache/incubator-mxnet/issues/18164#issuecomment-619150531


   Not sure how I missed that, thanks


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-mxnet] leezu edited a comment on issue #18164: Making _internal._plus_scalar and _internal._mul_scalar normal operators

[GitBox <gi...@apache.org>]

leezu edited a comment on issue #18164:
URL: https://github.com/apache/incubator-mxnet/issues/18164#issuecomment-619149130


   If they are available in the Gluon API, they are also available in the symbol API. There is nothing special about the `F` in Gluon API. It's just the standard symbol namespace.
   
   Ie just do `x + 6` in the symbol API


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-mxnet] leezu commented on issue #18164: Making _internal._plus_scalar and _internal._mul_scalar normal operators

[GitBox <gi...@apache.org>]

leezu commented on issue #18164:
URL: https://github.com/apache/incubator-mxnet/issues/18164#issuecomment-619149130


   If they are available in the Gluon API, they are also available in the symbol API. There is nothing special about the `F` in Gluon API. It's just the standard symbol namespace.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org