You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/02/26 19:58:54 UTC
[GitHub] [incubator-tvm] mbrookhart opened a new pull request #4949:
Conv3D_ncdhw and onnx support
mbrookhart opened a new pull request #4949: Conv3D_ncdhw and onnx support
URL: https://github.com/apache/incubator-tvm/pull/4949
This PR adds support for conv3d to the ONNX frontend and ports the conv2d_nchw schedules to conv3d_ncdhw for improved performance.
Thanks!
cc: @jwfromm @kevinthesun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen merged pull request #4949: Conv3D ONNX
support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
tqchen merged pull request #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4949: Conv3D ONNX
support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-597720540
Thanks @mbrookhart @icemelon9 @anijain2305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4949: Conv3D ONNX
support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-594966265
@icemelon9 please followup
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request
#4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
icemelon9 commented on a change in pull request #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384804363
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -73,6 +73,45 @@ def conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype):
return _conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype)
+@autotvm.register_topi_compute("conv3d_ncdhw.x86")
+def conv3d_ncdhw(cfg, data, kernel, strides, padding, dilation, out_dtype):
+ """3D convolution forward operator.
+
+ Parameters
+ ----------
+ input : tvm.Tensor
+ 5-D input data with shapes:
+ [batch, in_channel, in_depth, in_height, in_width] for NCDHW layout
+ [batch, in_depth, in_height, in_width, in_channel] for NDHWC layout
+
+ filter : tvm.Tensor
+ 5-D filter with shape [kernel_depth, kernel_height, kernel_width, in_channels, out_channels]
+
+ strides : int or a list/tuple of three ints
+ stride size, or [stride_depth, stride_height, stride_width]
+
+ padding : int or a list/tuple of three ints
+ padding size, or [pad_depth, pad_height, pad_width]
+
+ dilation: int or a list/tuple of three ints
+ dilation size, or [dilation_depth, dilation_height, dilation_width]
+
+ Returns
+ -------
+ output : tvm.Tensor
+ 5-D with shape [batch, out_depth, out_height, out_width, out_channel] for NDHWC layout
Review comment:
remove this line
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request
#4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
icemelon9 commented on a change in pull request #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384804848
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -197,6 +273,92 @@ def _conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype):
tag='conv3d_ndhwc')
return conv_unpacked
+def _conv3d_ncdhw(cfg, data, kernel, strides, padding, dilation, layout, out_dtype):
+ out_dtype = data.dtype if out_dtype is None else out_dtype
+
+ assert isinstance(dilation, int) or len(dilation) == 3
+ if isinstance(dilation, int):
+ dilation_d, dilation_h, dilation_w = (dilation, dilation, dilation)
+ else:
+ dilation_d, dilation_h, dilation_w = dilation
+
+ DSTR, HSTR, WSTR = strides
+ batch_size, in_channel, in_depth, in_height, in_width = get_const_tuple(data.shape)
+ num_filter, _, kernel_depth, kernel_height, kernel_width = get_const_tuple(kernel.shape)
+
+ dilated_kernel_d = (kernel_depth - 1) * dilation_d + 1
+ dilated_kernel_h = (kernel_height - 1) * dilation_h + 1
+ dilated_kernel_w = (kernel_width - 1) * dilation_w + 1
+
+ pad_front, pad_top, pad_left, pad_back, pad_down, pad_right = get_pad_tuple3d(
+ padding, (dilated_kernel_d, dilated_kernel_h, dilated_kernel_w))
+
+ pad_d = pad_front + pad_back
+ pad_h = pad_top + pad_down
+ pad_w = pad_left + pad_right
+
+ pad_depth = in_depth + pad_d
+ pad_height = in_height + pad_h
+ pad_width = in_width + pad_w
+
+ out_depth = simplify((in_depth + pad_d - dilated_kernel_d) // DSTR + 1)
+ out_height = simplify((in_height + pad_h - dilated_kernel_h) // HSTR + 1)
+ out_width = simplify((in_width + pad_w - dilated_kernel_w) // WSTR + 1)
+
+ # pack data
+ DOPAD = (pad_d != 0 or pad_h != 0 or pad_w != 0)
+ if DOPAD:
+ data_pad = pad(data, (0, 0, pad_front, pad_top, pad_left),
+ (0, 0, pad_back, pad_down, pad_right), name="data_pad")
+ else:
+ data_pad = data
+
+ # fetch schedule
+ ic_bn, oc_bn = cfg["tile_ic"].size[-1], cfg["tile_oc"].size[-1]
+
+ shape = (batch_size, in_channel // ic_bn, pad_depth, pad_height, ic_bn, pad_width)
+ data_vec = tvm.compute(shape,
+ lambda n, C, d, h, c, w: data_pad[n, C * ic_bn + c, d, h, w],
+ name='data_vec')
+
+ # pack kernel
+ shape = (num_filter//oc_bn, in_channel//ic_bn,
+ kernel_depth, kernel_height, kernel_width, ic_bn, oc_bn)
Review comment:
Since this schedule packs the kernel, we should consider adding `alter_op_layout` for conv3d to avoid the overhead of kernel packing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request
#4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384806231
##########
File path: python/tvm/relay/frontend/onnx.py
##########
@@ -99,8 +99,10 @@ def _impl(attr):
return prefix + '1d' + surfix
if len(kernel) == 2:
return prefix + '2d' + surfix
- msg = 'Only 1D and 2D kernels are supported for operator {}.'
- op_name = prefix + '1d/2d'
+ if len(kernel) == 3:
+ return prefix + '3d' + surfix
Review comment:
Would suggest changing surfix -> suffix while you are on it :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request
#4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384806750
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -73,6 +73,45 @@ def conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype):
return _conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype)
+@autotvm.register_topi_compute("conv3d_ncdhw.x86")
+def conv3d_ncdhw(cfg, data, kernel, strides, padding, dilation, out_dtype):
+ """3D convolution forward operator.
+
+ Parameters
+ ----------
+ input : tvm.Tensor
+ 5-D input data with shapes:
+ [batch, in_channel, in_depth, in_height, in_width] for NCDHW layout
+ [batch, in_depth, in_height, in_width, in_channel] for NDHWC layout
Review comment:
Do we need this line as this is NCDHW
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] masahi commented on issue #4949: Conv3D ONNX
support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
masahi commented on issue #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-596896933
hi, I'd like to use this schedule in my next Torch frontend PR (loading resnet 3D from torchvision). Can you merge this? @icemelon9 @anijain2305
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] tqchen commented on issue #4949: Conv3D ONNX
supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
tqchen commented on issue #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-591672116
also cc @yzhliu @icemelon9 @anijain2305 @masahi
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on a change in pull request
#4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on a change in pull request #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384807143
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -110,6 +149,43 @@ def _traverse(op):
traverse_inline(s, outs[0].op, _traverse)
return s
+@autotvm.register_topi_schedule("conv3d_ncdhw.x86")
+def schedule_conv3d_ncdhw(cfg, outs):
+ """TOPI schedule callback for conv3d
+ Parameters
+ ----------
+ outs: Array of Tensor
+ The computation graph description of conv3d
+ in the format of an array of tensors.
+ Returns
Review comment:
An extra line before it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] icemelon9 commented on issue #4949: Conv3D ONNX
support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
icemelon9 commented on issue #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-592150851
Could you rebase to latest master to fix the doc error? @mbrookhart
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbrookhart commented on a change in pull request
#4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on a change in pull request #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384838550
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -197,6 +273,92 @@ def _conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype):
tag='conv3d_ndhwc')
return conv_unpacked
+def _conv3d_ncdhw(cfg, data, kernel, strides, padding, dilation, layout, out_dtype):
+ out_dtype = data.dtype if out_dtype is None else out_dtype
+
+ assert isinstance(dilation, int) or len(dilation) == 3
+ if isinstance(dilation, int):
+ dilation_d, dilation_h, dilation_w = (dilation, dilation, dilation)
+ else:
+ dilation_d, dilation_h, dilation_w = dilation
+
+ DSTR, HSTR, WSTR = strides
+ batch_size, in_channel, in_depth, in_height, in_width = get_const_tuple(data.shape)
+ num_filter, _, kernel_depth, kernel_height, kernel_width = get_const_tuple(kernel.shape)
+
+ dilated_kernel_d = (kernel_depth - 1) * dilation_d + 1
+ dilated_kernel_h = (kernel_height - 1) * dilation_h + 1
+ dilated_kernel_w = (kernel_width - 1) * dilation_w + 1
+
+ pad_front, pad_top, pad_left, pad_back, pad_down, pad_right = get_pad_tuple3d(
+ padding, (dilated_kernel_d, dilated_kernel_h, dilated_kernel_w))
+
+ pad_d = pad_front + pad_back
+ pad_h = pad_top + pad_down
+ pad_w = pad_left + pad_right
+
+ pad_depth = in_depth + pad_d
+ pad_height = in_height + pad_h
+ pad_width = in_width + pad_w
+
+ out_depth = simplify((in_depth + pad_d - dilated_kernel_d) // DSTR + 1)
+ out_height = simplify((in_height + pad_h - dilated_kernel_h) // HSTR + 1)
+ out_width = simplify((in_width + pad_w - dilated_kernel_w) // WSTR + 1)
+
+ # pack data
+ DOPAD = (pad_d != 0 or pad_h != 0 or pad_w != 0)
+ if DOPAD:
+ data_pad = pad(data, (0, 0, pad_front, pad_top, pad_left),
+ (0, 0, pad_back, pad_down, pad_right), name="data_pad")
+ else:
+ data_pad = data
+
+ # fetch schedule
+ ic_bn, oc_bn = cfg["tile_ic"].size[-1], cfg["tile_oc"].size[-1]
+
+ shape = (batch_size, in_channel // ic_bn, pad_depth, pad_height, ic_bn, pad_width)
+ data_vec = tvm.compute(shape,
+ lambda n, C, d, h, c, w: data_pad[n, C * ic_bn + c, d, h, w],
+ name='data_vec')
+
+ # pack kernel
+ shape = (num_filter//oc_bn, in_channel//ic_bn,
+ kernel_depth, kernel_height, kernel_width, ic_bn, oc_bn)
Review comment:
:+1: I'm running into a lot of issues with the ONNX model zoo when AlterOpLayout is enabled, so there will be a second PR for that Pass.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request
#4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
icemelon9 commented on a change in pull request #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384841079
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -197,6 +273,92 @@ def _conv3d_ndhwc(cfg, data, kernel, strides, padding, dilation, out_dtype):
tag='conv3d_ndhwc')
return conv_unpacked
+def _conv3d_ncdhw(cfg, data, kernel, strides, padding, dilation, layout, out_dtype):
+ out_dtype = data.dtype if out_dtype is None else out_dtype
+
+ assert isinstance(dilation, int) or len(dilation) == 3
+ if isinstance(dilation, int):
+ dilation_d, dilation_h, dilation_w = (dilation, dilation, dilation)
+ else:
+ dilation_d, dilation_h, dilation_w = dilation
+
+ DSTR, HSTR, WSTR = strides
+ batch_size, in_channel, in_depth, in_height, in_width = get_const_tuple(data.shape)
+ num_filter, _, kernel_depth, kernel_height, kernel_width = get_const_tuple(kernel.shape)
+
+ dilated_kernel_d = (kernel_depth - 1) * dilation_d + 1
+ dilated_kernel_h = (kernel_height - 1) * dilation_h + 1
+ dilated_kernel_w = (kernel_width - 1) * dilation_w + 1
+
+ pad_front, pad_top, pad_left, pad_back, pad_down, pad_right = get_pad_tuple3d(
+ padding, (dilated_kernel_d, dilated_kernel_h, dilated_kernel_w))
+
+ pad_d = pad_front + pad_back
+ pad_h = pad_top + pad_down
+ pad_w = pad_left + pad_right
+
+ pad_depth = in_depth + pad_d
+ pad_height = in_height + pad_h
+ pad_width = in_width + pad_w
+
+ out_depth = simplify((in_depth + pad_d - dilated_kernel_d) // DSTR + 1)
+ out_height = simplify((in_height + pad_h - dilated_kernel_h) // HSTR + 1)
+ out_width = simplify((in_width + pad_w - dilated_kernel_w) // WSTR + 1)
+
+ # pack data
+ DOPAD = (pad_d != 0 or pad_h != 0 or pad_w != 0)
+ if DOPAD:
+ data_pad = pad(data, (0, 0, pad_front, pad_top, pad_left),
+ (0, 0, pad_back, pad_down, pad_right), name="data_pad")
+ else:
+ data_pad = data
+
+ # fetch schedule
+ ic_bn, oc_bn = cfg["tile_ic"].size[-1], cfg["tile_oc"].size[-1]
+
+ shape = (batch_size, in_channel // ic_bn, pad_depth, pad_height, ic_bn, pad_width)
+ data_vec = tvm.compute(shape,
+ lambda n, C, d, h, c, w: data_pad[n, C * ic_bn + c, d, h, w],
+ name='data_vec')
+
+ # pack kernel
+ shape = (num_filter//oc_bn, in_channel//ic_bn,
+ kernel_depth, kernel_height, kernel_width, ic_bn, oc_bn)
Review comment:
sounds good
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] mbrookhart commented on issue #4949: Conv3D ONNX
supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
mbrookhart commented on issue #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-591705734
Thanks, @anijain2305 and @icemelon9! I missed the change in testing strategy when I rebased ontop of @icemelon9 's recent refactor, should be good now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] icemelon9 commented on a change in pull request
#4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
icemelon9 commented on a change in pull request #4949: Conv3D ONNX support and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#discussion_r384841548
##########
File path: topi/python/topi/x86/conv3d.py
##########
@@ -110,6 +146,44 @@ def _traverse(op):
traverse_inline(s, outs[0].op, _traverse)
return s
+@autotvm.register_topi_schedule("conv3d_ncdhw.x86")
+def schedule_conv3d_ncdhw(cfg, outs):
+ """TOPI schedule callback for conv3d
+ Parameters
Review comment:
One empty line above this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-tvm] anijain2305 commented on issue #4949: Conv3D ONNX
supprot and conv3D_ncdhw x86 schedules
Posted by GitBox <gi...@apache.org>.
anijain2305 commented on issue #4949: Conv3D ONNX supprot and conv3D_ncdhw x86 schedules
URL: https://github.com/apache/incubator-tvm/pull/4949#issuecomment-591681153
@alexgl-github You might be interested in this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services