You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/07/15 17:15:39 UTC

[GitHub] [tvm] mbs-octoml opened a new pull request, #12107: [Relay] Allow partial virtual device annotations.

mbs-octoml opened a new pull request, #12107:
URL: https://github.com/apache/tvm/pull/12107

   Previously CompilationConfig::CanonicalVirtualDevice required
   the argument virtual device to contain a device type. However
   now that virtual devices may contain memory scopes that's
   unnecessarily strict.
   
   With this change it is possible to write virtual device
   annotations with just memory scopes, and let PlanDevices
   flow those constraints along with the usual device constraints.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] driazati commented on a diff in pull request #12107: [Relay] Allow partial virtual device annotations.

Posted by GitBox <gi...@apache.org>.
driazati commented on code in PR #12107:
URL: https://github.com/apache/tvm/pull/12107#discussion_r922394353


##########
python/tvm/target/virtual_device.py:
##########
@@ -27,7 +27,9 @@ class VirtualDevice(Object):
     """A compile time representation for where data is to be stored at runtime,
     and how to compile code to compute it."""
 
-    def __init__(self, device, target=None, memory_scope="") -> None:
+    def __init__(self, device=None, target=None, memory_scope="") -> None:
+        if device is None:
+            device = tvm.device(-1, -1)

Review Comment:
   does `device(-1, -1)` mean anything special or is it just a stand in for `kInvalidDeviceType`?



##########
src/target/compilation_config.cc:
##########
@@ -98,19 +98,30 @@ Target CompilationConfigNode::CanonicalTarget(const Target& target) const {
 
 VirtualDevice CompilationConfigNode::CanonicalVirtualDevice(
     const VirtualDevice& virtual_device) const {
-  DLDeviceType device_type = virtual_device->device_type();
+  // Targets need special handling.
   Target target = virtual_device->target;
   if (target.defined()) {
-    target = CanonicalTarget(target);
-  } else {
-    // Find the (unique) target matching the device's device type.
-    // TODO(mbs): Proper diagnostics.
-    CHECK(device_type != kInvalidDeviceType)
-        << "VirtualDevice annotations must include at least a device_type";
-    target = FindPrimitiveTargetForDeviceOrFail(device_type);
+    // It is possible the given target object was constructed by the user, but was then
+    // rewritten on the way into the CompilationConfig. So 'canonicalize' it by replacing
+    // the given target with one structurally equal to one already known in the config if
+    // possible.
+    Target canon_target = CanonicalTarget(target);
+    if (canon_target != target) {
+      VLOG(1) << "Canonicalized target " << canon_target->ToDebugString();
+    }
+    target = canon_target;
+  } else if (virtual_device->device_type() != kInvalidDeviceType) {
+    // Since no target was given, choose one with a matching device type.
+    // This is the one place where we allow device types to imply targets.
+    target = FindPrimitiveTargetForDeviceOrFail(virtual_device->device_type());
+    VLOG(1) << "Defaulted to target " << target->ToDebugString();
   }
-  return virtual_device_cache_.Unique(VirtualDevice(device_type, virtual_device->virtual_device_id,
-                                                    target, virtual_device->memory_scope));
+  // else: the target will remain unknown.

Review Comment:
   I don't know the code around these parts to well but just wondering with this change is it possible to get a virtual device that has neither a memory scope or a concrete device type? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] mbs-octoml commented on a diff in pull request #12107: [Relay] Allow partial virtual device annotations.

Posted by GitBox <gi...@apache.org>.
mbs-octoml commented on code in PR #12107:
URL: https://github.com/apache/tvm/pull/12107#discussion_r922413056


##########
src/target/compilation_config.cc:
##########
@@ -98,19 +98,30 @@ Target CompilationConfigNode::CanonicalTarget(const Target& target) const {
 
 VirtualDevice CompilationConfigNode::CanonicalVirtualDevice(
     const VirtualDevice& virtual_device) const {
-  DLDeviceType device_type = virtual_device->device_type();
+  // Targets need special handling.
   Target target = virtual_device->target;
   if (target.defined()) {
-    target = CanonicalTarget(target);
-  } else {
-    // Find the (unique) target matching the device's device type.
-    // TODO(mbs): Proper diagnostics.
-    CHECK(device_type != kInvalidDeviceType)
-        << "VirtualDevice annotations must include at least a device_type";
-    target = FindPrimitiveTargetForDeviceOrFail(device_type);
+    // It is possible the given target object was constructed by the user, but was then
+    // rewritten on the way into the CompilationConfig. So 'canonicalize' it by replacing
+    // the given target with one structurally equal to one already known in the config if
+    // possible.
+    Target canon_target = CanonicalTarget(target);
+    if (canon_target != target) {
+      VLOG(1) << "Canonicalized target " << canon_target->ToDebugString();
+    }
+    target = canon_target;
+  } else if (virtual_device->device_type() != kInvalidDeviceType) {
+    // Since no target was given, choose one with a matching device type.
+    // This is the one place where we allow device types to imply targets.
+    target = FindPrimitiveTargetForDeviceOrFail(virtual_device->device_type());
+    VLOG(1) << "Defaulted to target " << target->ToDebugString();
   }
-  return virtual_device_cache_.Unique(VirtualDevice(device_type, virtual_device->virtual_device_id,
-                                                    target, virtual_device->memory_scope));
+  // else: the target will remain unknown.

Review Comment:
   Good point -- it is now possible to ask for CanonicalVirtualDevice of a 'fully unconstrained' virtual device (ie one with no fields define at all), which should be object equal to VirtualDevice::FullyUnconstrained(). Fixed that, thanks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] mbs-octoml commented on a diff in pull request #12107: [Relay] Allow partial virtual device annotations.

Posted by GitBox <gi...@apache.org>.
mbs-octoml commented on code in PR #12107:
URL: https://github.com/apache/tvm/pull/12107#discussion_r922406486


##########
python/tvm/target/virtual_device.py:
##########
@@ -27,7 +27,9 @@ class VirtualDevice(Object):
     """A compile time representation for where data is to be stored at runtime,
     and how to compile code to compute it."""
 
-    def __init__(self, device, target=None, memory_scope="") -> None:
+    def __init__(self, device=None, target=None, memory_scope="") -> None:
+        if device is None:
+            device = tvm.device(-1, -1)

Review Comment:
   Added a comment. Yes, it just means 'nothing known about the device type or id, it may be filled in during device planning based on how the contraints work out'.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [tvm] AndrewZhaoLuo merged pull request #12107: [Relay] Allow partial virtual device annotations.

Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo merged PR #12107:
URL: https://github.com/apache/tvm/pull/12107


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org