You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2020/07/23 06:27:39 UTC
[GitHub] [incubator-tvm] zhanghaohit opened a new pull request #6125: add device_annot support in graphpack
zhanghaohit opened a new pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125
This is related to #5840 and split from PR #5842
This PR depends on #6124 to work.
The new API allows to tag ext_dev device type from annot_start_name and annot_end_name
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit removed a comment on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit removed a comment on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-682350954
> #4178
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-682350954
> #4178
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on a change in pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#discussion_r477013469
##########
File path: vta/python/vta/top/graphpack.py
##########
@@ -174,6 +174,103 @@ def _operator_idx_inc(expr, count_meta, operator_current_idx):
operator_current_idx = operator_current_idx + 1
return operator_current_idx
+
+class ExprDeviceAnnot(ExprMutator):
+ """Visitor to perform graph annotation on an AST.
+
+ Parameters
+ ----------
+ start: int
+ the start location to mark run on vta (inclusive)
+ end: int
+ the end location to mark run on vta (exclusive)
+
+ Returns
+ ---------
+ None
+ """
+ def __init__(self, start=-1, end=-1):
+ self.ext_ctx = tvm.context("ext_dev")
+ self.cpu_ctx = tvm.context("cpu")
+ self.cast = op.op.get("cast")
+ self.counter = -1
+ self.start = start
+ self.end = end
+ super().__init__()
+
+ def visit_call(self, call):
+ """ Visit the children. """
+ # First visit the children.
+ args = [self.visit(arg) for arg in call.args]
+
+ self.counter += 1
+ if self.counter == self.start:
+ ret = relay.Call(call.op, args, call.attrs)
+ ret = relay.annotation.on_device(ret, self.ext_ctx)
+ return ret
+
+ if self.counter == self.end:
+ ret = relay.Call(call.op, args, call.attrs)
+ ret = relay.annotation.on_device(ret, self.cpu_ctx)
+ return ret
+
+ if self.counter > self.start and self.counter < self.end:
+ ret = relay.Call(call.op, args, call.attrs)
+
+ # skip the float op, i.e., float->int cast
+ if self.is_float_op(call):
+ return ret
+
+ return relay.annotation.on_device(ret, self.ext_ctx)
+
+ return relay.Call(self.visit(call.op), args, call.attrs)
+
+ def is_float_op(self, call):
+ """check if this op belongs to a float op
+ in general, float op's odtype is float;
+ a special case is float->int cast, which follow this op sequence:
+ multiply(float) -> round(float) -> clip(float) -> cast(int);
+ """
+ args = call.args
+ odtype = _get_tensor_type(call)
+
+ if odtype == "float32":
+ return True
+
+ if call.op == self.cast:
+ idtype = _get_tensor_type(args[0])
+ if idtype == "float32":
+ return True
+
+ return False
+
+
+class ExprLocater(ExprMutator):
Review comment:
ExprLocater -> ExprLocator
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on a change in pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#discussion_r477013670
##########
File path: python/tvm/relay/op/_tensor.py
##########
@@ -87,6 +87,10 @@
register_broadcast_schedule("fast_exp")
register_broadcast_schedule("fast_tanh")
register_broadcast_schedule("fast_erf")
+# a fake on_device schedule.
+# this will not be used in actual computation
+# as on_device will be removed during DeviceAnnotation pass
+register_injective_schedule("on_device")
Review comment:
@zhanghaohit what happens if we don't register this annotation op schedule?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-678934451
> To prevent accidental PR merged given that this depends on #6124, I'm setting it to "changes requested"
@tmoreau89 Thanks for reviewing #6124. Shall we start review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on a change in pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on a change in pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#discussion_r477013567
##########
File path: vta/python/vta/top/graphpack.py
##########
@@ -449,18 +549,46 @@ def graph_pack(expr,
'expr.astext(show_meta_data=False)'. When count_meta is True, the operator increase
logic would count the meta.
+ device_annot: boolean, optional
+ if we want to annoate the device_type
+
+ annot_start_name: str, optional
+ device annotation start node, from which we mark the nodes as `ext_dev`
+
+ annot_end_name: str, optional
+ device annotation end node, after which we mark the nodes as 'cpu'
+
Returns
-------
expr : Expr
The transformed expression.
"""
assert isinstance(expr, relay.Function)
- assert ((start_name != stop_name) or (start_name_idx < stop_name_idx))
+ assert ((start_name != stop_name) or (start_name_idx is None != stop_name_idx is None) or \
+ (not (start_name_idx is None and stop_name_idx is None)) \
+ or (start_name_idx < stop_name_idx))
expr = get_subgraph(expr, start_name, stop_name, start_name_idx, stop_name_idx, count_meta)
expr = run_opt_pass(expr, transform.InferType())
packer = ExprPack(
bfactor, cfactor,
weight_bits)
expr = packer.visit(expr)
assert not packer.start_pack
- return run_opt_pass(expr, transform.InferType())
+ expr = run_opt_pass(expr, transform.InferType())
+
+ if device_annot:
+ expr_locator = ExprLocater()
Review comment:
ExprLocator
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit closed pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit closed pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-682351681
> I am wondering whether VTA's graph annotation can be unified into the relay's heterogeneous execution feature: #4178
I think #4178 is for VM, while graph annotation is for graph runtime?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-719454103
ping @ZihengJiang
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-720508527
> Please fix the CI issue @zhanghaohit
Fixed. Thanks @ZihengJiang @tmoreau89 for the review.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] zhanghaohit commented on a change in pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
zhanghaohit commented on a change in pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#discussion_r478050263
##########
File path: python/tvm/relay/op/_tensor.py
##########
@@ -87,6 +87,10 @@
register_broadcast_schedule("fast_exp")
register_broadcast_schedule("fast_tanh")
register_broadcast_schedule("fast_erf")
+# a fake on_device schedule.
+# this will not be used in actual computation
+# as on_device will be removed during DeviceAnnotation pass
+register_injective_schedule("on_device")
Review comment:
It will raise
`
AssertionError: on_device doesn't have FTVMStrategy registered
`
during build_module.cc::Optimize before we do the `RunDeviceAnnotationPass`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] ZihengJiang edited a comment on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
ZihengJiang edited a comment on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-681502427
I am wondering whether VTA's graph annotation can be unified into the relay's heterogeneous execution feature: https://github.com/apache/incubator-tvm/issues/4178
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] tmoreau89 commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
tmoreau89 commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-680508143
@zhanghaohit thanks for the ping, @ZihengJiang can you help me review this PR?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-tvm] ZihengJiang commented on pull request #6125: [VTA][OpenCL] add device_annot support in graphpack
Posted by GitBox <gi...@apache.org>.
ZihengJiang commented on pull request #6125:
URL: https://github.com/apache/incubator-tvm/pull/6125#issuecomment-681502427
I am wondering whether VTA's graph annotation can be unified into the relay's heterogeneous execution feature
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org