You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/07/30 11:43:28 UTC
[GitHub] [tvm] vvchernov opened a new pull request #8599: Vc/lstm perf
vvchernov opened a new pull request #8599:
URL: https://github.com/apache/tvm/pull/8599
LSTM cell was unified and transferred to common place for all frontends. Here it is simultaneously used by onnx and pytorch frontends of TVM. LSTM cell was analyzed and modified to remove excess memory and other manipulations which potentially can not be fixed by compiler on its side. Performance tests for different modification of LSTM before and after were carried out. The results are collected in the tables:
Table 1. Average time per run (microsec) for 10000 runs. The following parameters are used (small input size): with biases = True, batch first = True, feature size = 5, hidden size = 10, number of stacked layers = 2, sequence length = 3, batch size = 1, trials number = 100
Frontend name/LSTM type | uni | b | s | sb |
---------------------------------------------------------------
Onnx | 26.8 | 55.3 | 50.7 | 112.7 |
Onnx dev | 20.1 | 40.5 | 37.7 | 81.7 |
Onnx tuned | 5.1 | 5.8 | 7.1 | 11.1 |
Onnx dev tuned | 4.7 | 6.0 | 6.2 | 10.2 |
--------------------------------------------------------------
Pytorch | 12.1 | 19.9 | 20.5 | 37.2 |
Pytorch dev | 8.9 | 14.1 | 14.9 | 27.5 |
Pytorch tuned | 4.8 | 6.0 | 6.4 | 9.9 |
Pytorch dev tuned | 4.7 | 6.1 | 6.4 | 9.8 |
--------------------------------------------------------------
Onnxruntime | 16.0 | 21.1 | 24.8 | 36.7 |
There are several LSTM types: uni – unidirectional, b – bidirectional, s – stacked (2 layers are used in the tests), sb - stacked bidirectional. Suffix "dev" means implementation in this patch. We have strong difference for performance between implementation on onnx and pytorch without tuning (onnx one is slower). With tuning onnx implementation was slightly worse than pytorch. This patch fixed performance differences for LSTM with tuning and imporved results without tuning for both onnx and pytorch.
Table 2. Average time per run (ms) for 1000 runs. The following parameters are used (big input size): with biases = True, batch first = True, feature size = 40, hidden size = 256, number of stacked layers = 3, sequence length = 160, batch size = 1, trials number = 100
Frontend name/LSTM type | uni | b | s | sb |
--------------------------------------------------------------
Onnx | 47.3 | | 205 | |
Onnx dev | | | | |
Onnx tuned | 8.74 | | 31.8 | |
Onnx dev tuned | | | | |
--------------------------------------------------------------
Pytorch | 7.77 | | 27.2 | |
Pytorch dev | | | | |
Pytorch tuned | 7.71 | | 27.3 | |
Pytorch dev tuned | 7.61 | | | |
--------------------------------------------------------------
Onnxruntime | 1.50 | | 4.69 | |
@masahi @jwfromm please review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#issuecomment-894093350
thanks @vvchernov
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi merged pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi merged pull request #8599:
URL: https://github.com/apache/tvm/pull/8599
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681630689
##########
File path: python/tvm/relay/op/layers.py
##########
@@ -0,0 +1,115 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
I think it is better not to introduce a new file and move its content to frontend/common.py
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681741284
##########
File path: python/tvm/relay/op/layers.py
##########
@@ -0,0 +1,115 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681630173
##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -2142,58 +2142,72 @@ class LSTM(RNN):
"""Operator converter for LSTM"""
@classmethod
- def generate_lstm(
- cls, X_steps, H_t, C_t, W, R, B, p_i, p_f, p_o, f_act, g_act, h_act, backwards=False
- ):
- """Create an unrolled lstm loop.
-
- See https://github.com/onnx/onnx/blob/master/docs/Operators.md for math.
+ def unbind(cls, data, axis=0):
Review comment:
Move this to frontend/common.py if possible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681724290
##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -2142,58 +2142,72 @@ class LSTM(RNN):
"""Operator converter for LSTM"""
@classmethod
- def generate_lstm(
- cls, X_steps, H_t, C_t, W, R, B, p_i, p_f, p_o, f_act, g_act, h_act, backwards=False
- ):
- """Create an unrolled lstm loop.
-
- See https://github.com/onnx/onnx/blob/master/docs/Operators.md for math.
+ def unbind(cls, data, axis=0):
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682306900
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -2094,21 +2094,8 @@ def deform_conv2d(self, inputs, input_types):
def unbind(self, inputs, input_types):
Review comment:
It is used. It is wrapper for pytorch frontend which supports given format of op arguments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681797370
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -2588,23 +2505,92 @@ def lstm(self, inputs, input_types):
else:
layers_c = self.unbind((c_0, 0), X_dtype)
- hiddens = []
- for i in range(num_layers):
+ layer_weights_dicts = []
+ k = 0 # layer counter
+ if has_biases:
if bidirectional:
- hiddens.append(
- ((layers_h[2 * i], layers_c[2 * i]), (layers_h[2 * i + 1], layers_c[2 * i + 1]))
- )
+ rsd = len(_weights) % (2 * weights_num)
+ assert rsd == 0, "got an incorrect number of LSTM weights"
+ for i in range(0, len(_weights), 2 * weights_num):
+ fw_weights_dict = {}
+ fw_weights_dict["hidden_state"] = layers_h[2 * k]
+ fw_weights_dict["cell_state"] = layers_c[2 * k]
+ fw_weights_dict["w_inp"] = _weights[i]
+ fw_weights_dict["w_hid"] = _weights[i + 1]
+ fw_weights_dict["b_inp"] = _weights[i + 2]
+ fw_weights_dict["b_hid"] = _weights[i + 3]
+ if has_proj:
+ fw_weights_dict["proj"] = _weights[i + 4]
+ rev_weights_dict = {}
+ j = i + weights_num
+ rev_weights_dict["hidden_state"] = layers_h[2 * k + 1]
+ rev_weights_dict["cell_state"] = layers_c[2 * k + 1]
+ rev_weights_dict["w_inp"] = _weights[j]
+ rev_weights_dict["w_hid"] = _weights[j + 1]
+ rev_weights_dict["b_inp"] = _weights[j + 2]
+ rev_weights_dict["b_hid"] = _weights[j + 3]
+ if has_proj:
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682200709
##########
File path: python/tvm/relay/frontend/common.py
##########
@@ -624,3 +624,128 @@ def to_int_list(np_array):
cause problems in relay/TOPI.
"""
return [int(x) for x in np_array]
+
+
+def unbind(data, axis=0):
+ """
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
Review comment:
`already without it` -> `with specified axis removed`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682195329
##########
File path: python/tvm/relay/frontend/common.py
##########
@@ -624,3 +624,128 @@ def to_int_list(np_array):
cause problems in relay/TOPI.
"""
return [int(x) for x in np_array]
+
+
+def unbind(data, axis=0):
+ """
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
+ TODO (vvchernov): It needs such operation on relay side to reduce time consumption
+ on squeeze operation.
+
+ Parameters
+ ----------
+ data : relay.Expr
+ Input tensor
+ axis : int
+ Axis along which tensor is splited. Tensors in the list do not have this axis.
Review comment:
`splited` -> `split`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681631886
##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -2142,58 +2142,72 @@ class LSTM(RNN):
"""Operator converter for LSTM"""
@classmethod
- def generate_lstm(
- cls, X_steps, H_t, C_t, W, R, B, p_i, p_f, p_o, f_act, g_act, h_act, backwards=False
- ):
- """Create an unrolled lstm loop.
-
- See https://github.com/onnx/onnx/blob/master/docs/Operators.md for math.
+ def unbind(cls, data, axis=0):
"""
- h_list = []
- seq_length = len(X_steps)
- for i in range(seq_length):
- step = X_steps[i] if not backwards else X_steps[seq_length - (i + 1)]
- step = _op.squeeze(step, axis=[0])
- gates = _op.nn.dense(step, W) + _op.nn.dense(H_t, R)
- if B is not None:
- WB, RB = _op.split(B, 2)
- gates += WB + RB
- i, o, f, c = _op.split(gates, 4, axis=-1)
-
- if p_i != 0:
- i = f_act(i + p_i * C_t)
- else:
- i = f_act(i)
-
- if p_f != 0:
- f = f_act(f + p_f * C_t)
- else:
- f = f_act(f)
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
+ TODO (vvchernov): It needs such operation on relay side to reduce time consumption
+ on squeeze operation.
- c = g_act(c)
- C = f * C_t + i * c
- if p_o != 0:
- o = f_act(o + p_o * C)
- else:
- o = f_act(o)
-
- H = o * h_act(C)
+ Parameters
+ ----------
+ data : relay.Expr
+ Input tensor
+ axis : int
+ Axis along which tensor is splited. Tensors in the list has not this axis.
Review comment:
`Axis along which tensor is split.`
Do you mean `Tensors in the list does not have this axis`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#issuecomment-892269049
Thanks @vvchernov I only have minor comments left.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682195329
##########
File path: python/tvm/relay/frontend/common.py
##########
@@ -624,3 +624,128 @@ def to_int_list(np_array):
cause problems in relay/TOPI.
"""
return [int(x) for x in np_array]
+
+
+def unbind(data, axis=0):
+ """
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
+ TODO (vvchernov): It needs such operation on relay side to reduce time consumption
+ on squeeze operation.
+
+ Parameters
+ ----------
+ data : relay.Expr
+ Input tensor
+ axis : int
+ Axis along which tensor is splited. Tensors in the list do not have this axis.
Review comment:
`splited` -> `split`
I get what you mean by `Tensors in the list do not have this axis.`, but this sentence is not really correct. I suggest just remove this sentence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682191113
##########
File path: python/tvm/relay/frontend/common.py
##########
@@ -624,3 +624,128 @@ def to_int_list(np_array):
cause problems in relay/TOPI.
"""
return [int(x) for x in np_array]
+
+
+def unbind(data, axis=0):
+ """
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
+ TODO (vvchernov): It needs such operation on relay side to reduce time consumption
+ on squeeze operation.
+
+ Parameters
+ ----------
+ data : relay.Expr
+ Input tensor
+ axis : int
+ Axis along which tensor is splited. Tensors in the list do not have this axis.
+ Returns
+ -------
+ result : List[relay.Expr]
+ The sequence of computed tensors
+ """
+ shape = infer_shape(data)
+ if axis >= len(shape):
+ msg = "Please check input dim, it shouldn't be greater than or equal to rank."
+ raise AttributeError(msg)
+
+ selections = shape[axis]
+ res_split = _op.split(data, selections, axis)
+ ret = []
+ for i in range(selections):
+ ret.append(_op.squeeze(res_split[i], axis=[axis]))
+ return _expr.TupleWrapper(_expr.Tuple(ret), selections)
+
+
+def lstm_cell(
+ input_seqs,
+ hidden_state,
+ cell_state,
+ w_inp,
+ w_hid,
+ b_inp=None,
+ b_hid=None,
+ proj=None,
+ p_i=None,
+ p_f=None,
+ p_o=None,
+ f_act=_op.sigmoid,
+ g_act=_op.tanh,
+ h_act=_op.tanh,
+ backwards=False,
+):
+ """
+ Common implementation of LSTM cell for all frontends of TVM
+ TODO (vvchernov): currently it is used by onnx and pytorch. Extend for other frontends
+
+ Parameters
+ ----------
+ input_seqs : List[relay.Expr]
+ The sequence of input tensors
+ Input tensor should be 2d while issue #8412 is not resolved
+ Shape = (batch, feature_size)
+ hidden_state : relay.Expr
+ Hidden state. shape = (batch, hidden_size)
+ cell_state : relay.Expr
+ Cell state. shape = (batch, hidden_size)
+ w_inp, w_hid : relay.Expr
+ weight matrices. wi shape = (4 * hidden_size, feature_size)
+ wh shape = (4 * hidden_size, hidden_size or proj_size)
+ NOTE: wi = (w_ii|w_if|w_ig|w_io) for input, forget, cell and output gates.
+ The order is important for correct LSTM calculation!
+ b_inp, b_hid : relay.Expr
+ bias matrices. The same order of internal parts as for weights. shape = (4 * hidden_size)
+ proj : relay.Expr
+ projection matrix. shape = (proj_size, hidden_size)
+ p_i, p_f, p_o : relay.Expr
+ peephole LSTM matrices. shape = (batch, hidden_size)
+ f_act, g_act, h_act : relay.op
+ activation funtions
+ backwards : bool
+ Flag for reverse pass of LSTM
+
+ Returns
+ -------
+ result : List[relay.Expr], relay.Expr, relay.Expr
+ The sequence of computed result, final hidden and cell state
+ """
+
+ outputs_list = []
+ for x_t in input_seqs if not backwards else reversed(input_seqs):
+ # x_t shape = (batch, feature size), step shape = (batch, feature size + hidden_size)
+ step = _op.concatenate([x_t, hidden_state], axis=1)
+ cat_w = _op.concatenate([w_inp, w_hid], axis=1)
+ # Instead of nn.dense(x_t, w_inp) + nn.dense(hidden_state, w_hid)
+ # the nn.dense(step, cat_w) is used
Review comment:
remove the
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#issuecomment-891743490
cc @mbrookhart
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681657427
##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -2142,58 +2142,72 @@ class LSTM(RNN):
"""Operator converter for LSTM"""
@classmethod
- def generate_lstm(
- cls, X_steps, H_t, C_t, W, R, B, p_i, p_f, p_o, f_act, g_act, h_act, backwards=False
- ):
- """Create an unrolled lstm loop.
-
- See https://github.com/onnx/onnx/blob/master/docs/Operators.md for math.
+ def unbind(cls, data, axis=0):
Review comment:
Should I do the same for pytorch frontend? https://github.com/apache/tvm/blob/09e234dfd9dd8adf77cf4b6c39a371dd74530e14/python/tvm/relay/frontend/pytorch.py#L2095
More over just now I'm trying to implement this method (unbind) as relay operation, I follow to split op for this. But I think it will be separated PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] vvchernov commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
vvchernov commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681657107
##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -2142,58 +2142,72 @@ class LSTM(RNN):
"""Operator converter for LSTM"""
@classmethod
- def generate_lstm(
- cls, X_steps, H_t, C_t, W, R, B, p_i, p_f, p_o, f_act, g_act, h_act, backwards=False
- ):
- """Create an unrolled lstm loop.
-
- See https://github.com/onnx/onnx/blob/master/docs/Operators.md for math.
+ def unbind(cls, data, axis=0):
"""
- h_list = []
- seq_length = len(X_steps)
- for i in range(seq_length):
- step = X_steps[i] if not backwards else X_steps[seq_length - (i + 1)]
- step = _op.squeeze(step, axis=[0])
- gates = _op.nn.dense(step, W) + _op.nn.dense(H_t, R)
- if B is not None:
- WB, RB = _op.split(B, 2)
- gates += WB + RB
- i, o, f, c = _op.split(gates, 4, axis=-1)
-
- if p_i != 0:
- i = f_act(i + p_i * C_t)
- else:
- i = f_act(i)
-
- if p_f != 0:
- f = f_act(f + p_f * C_t)
- else:
- f = f_act(f)
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
+ TODO (vvchernov): It needs such operation on relay side to reduce time consumption
+ on squeeze operation.
- c = g_act(c)
- C = f * C_t + i * c
- if p_o != 0:
- o = f_act(o + p_o * C)
- else:
- o = f_act(o)
-
- H = o * h_act(C)
+ Parameters
+ ----------
+ data : relay.Expr
+ Input tensor
+ axis : int
+ Axis along which tensor is splited. Tensors in the list has not this axis.
Review comment:
Yes. You can see https://pytorch.org/docs/stable/generated/torch.unbind.html, it is repeat torch op.
This operation is reverse to relay stack op. For split we have such pair, it is concatinate op.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682195329
##########
File path: python/tvm/relay/frontend/common.py
##########
@@ -624,3 +624,128 @@ def to_int_list(np_array):
cause problems in relay/TOPI.
"""
return [int(x) for x in np_array]
+
+
+def unbind(data, axis=0):
+ """
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
+ and returns a tuple of all slices along a given dimension, already without it.
+ TODO (vvchernov): It needs such operation on relay side to reduce time consumption
+ on squeeze operation.
+
+ Parameters
+ ----------
+ data : relay.Expr
+ Input tensor
+ axis : int
+ Axis along which tensor is splited. Tensors in the list do not have this axis.
Review comment:
`splited` -> `split`
I get what you mean by `Tensors in the list do not have this axis.`, but this sentence is not really correct (they do have axis 0 which previously was axis1). I suggest just remove this sentence.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] junrushao1994 commented on pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
junrushao1994 commented on pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#issuecomment-891392292
CC @masahi if you are interested
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682195059
##########
File path: python/tvm/relay/frontend/common.py
##########
@@ -624,3 +624,128 @@ def to_int_list(np_array):
cause problems in relay/TOPI.
"""
return [int(x) for x in np_array]
+
+
+def unbind(data, axis=0):
+ """
+ Unbind was gotten from pytorch.py and modified. The operation removes a tensor dimension
Review comment:
`Unbind was taken from the PyTorch frontend.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r682192887
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -2094,21 +2094,8 @@ def deform_conv2d(self, inputs, input_types):
def unbind(self, inputs, input_types):
Review comment:
Remove this method if possible (it is no longer used?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] masahi commented on a change in pull request #8599: [Frontend] Unified LSTM cell
Posted by GitBox <gi...@apache.org>.
masahi commented on a change in pull request #8599:
URL: https://github.com/apache/tvm/pull/8599#discussion_r681647443
##########
File path: python/tvm/relay/frontend/pytorch.py
##########
@@ -2588,23 +2505,92 @@ def lstm(self, inputs, input_types):
else:
layers_c = self.unbind((c_0, 0), X_dtype)
- hiddens = []
- for i in range(num_layers):
+ layer_weights_dicts = []
+ k = 0 # layer counter
+ if has_biases:
if bidirectional:
- hiddens.append(
- ((layers_h[2 * i], layers_c[2 * i]), (layers_h[2 * i + 1], layers_c[2 * i + 1]))
- )
+ rsd = len(_weights) % (2 * weights_num)
+ assert rsd == 0, "got an incorrect number of LSTM weights"
+ for i in range(0, len(_weights), 2 * weights_num):
+ fw_weights_dict = {}
+ fw_weights_dict["hidden_state"] = layers_h[2 * k]
+ fw_weights_dict["cell_state"] = layers_c[2 * k]
+ fw_weights_dict["w_inp"] = _weights[i]
+ fw_weights_dict["w_hid"] = _weights[i + 1]
+ fw_weights_dict["b_inp"] = _weights[i + 2]
+ fw_weights_dict["b_hid"] = _weights[i + 3]
+ if has_proj:
+ fw_weights_dict["proj"] = _weights[i + 4]
+ rev_weights_dict = {}
+ j = i + weights_num
+ rev_weights_dict["hidden_state"] = layers_h[2 * k + 1]
+ rev_weights_dict["cell_state"] = layers_c[2 * k + 1]
+ rev_weights_dict["w_inp"] = _weights[j]
+ rev_weights_dict["w_hid"] = _weights[j + 1]
+ rev_weights_dict["b_inp"] = _weights[j + 2]
+ rev_weights_dict["b_hid"] = _weights[j + 3]
+ if has_proj:
Review comment:
Please try cleaning up weight dict initialization. I'd try something like `fw_weights_dict = dict(zip(..))`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org