[GitHub] [incubator-tvm] masahi commented on a change in pull request #5249: [PYTORCH]LayerNorm support added

[GitBox <gi...@apache.org>]

masahi commented on a change in pull request #5249: [PYTORCH]LayerNorm support added
URL: https://github.com/apache/incubator-tvm/pull/5249#discussion_r404391408
 
 

 ##########
 File path: python/tvm/relay/frontend/pytorch.py
 ##########
 @@ -503,6 +503,34 @@ def _impl(inputs, input_types):
                                     scale=scale)
     return _impl
 
+def _get_dims(data):
+    import torch
+    if isinstance(data, _expr.Expr):
+        dims = _infer_shape(data)
+    elif isinstance(data, list):
+        dims = data
+    elif isinstance(data, (torch.Tensor, np.ndarray)):
+        dims = data.shape
+    else:
+        msg = "Data type %s could not be parsed" % type(data)
+        raise AssertionError(msg)
+    return dims
+
+def _layer_norm():
+    def _impl(inputs, input_types):
+        data = inputs[0]
+        ndims = len(_get_dims(inputs[1]))
+        assert ndims == 1, "Support only normalization over last one dimension."
+
+        return _op.nn.layer_norm(data,
+                                 gamma=inputs[1],
+                                 beta=inputs[2],
+                                 axis=-1,
 
 Review comment:
   it seems gamma is inputs[2] and beta is inputs[3]. @siju-samuel Can you confirm?
   I get an error using this new conversion in my test. Above change fixed it.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services