You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2021/08/05 08:09:05 UTC
[GitHub] [tvm] kueitang opened a new pull request #8668: [Draft PR] Visualization Relay IR on terminal
kueitang opened a new pull request #8668:
URL: https://github.com/apache/tvm/pull/8668
Hi,
This utility is designed for users who are prone to snap-shot the relay IR graph on terminal.
The AST structure format covers basic relay function/expressions such as _Function, Call, Let, Var, GlobalVar, If, Tuple, Constant_ currently.
For example:
```
A Relay IRModule:
x = relay.var("x")
y = relay.const(1)
z = relay.add(x, y)
z = relay.multiply(x, z)
mod = tvm.ir.IRModule().from_expr(z)
```
with:
```
viz_res = retv.ASTVisualization()
mod = viz_res(mod)
res = viz_res.get_output()
```
will be showed as:
```
== The AST view of the IRModule is ==
@main([Var(x)])
`--(call)
|--multiply
|--x
`--(call)
|--add
|--x
`--1
```
It is welcome to have your thoughts on the utility.
Anyways, thanks. :)
Thanks for contributing to TVM! Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from [Reviewers](https://github.com/apache/incubator-tvm/blob/master/CONTRIBUTORS.md#reviewers) by @ them in the pull request thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-895672531
hi @tqchen,
would you like to give it any thoughts? Thanks a lot! :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo closed pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo closed pull request #8668:
URL: https://github.com/apache/tvm/pull/8668
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a change in pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
areusch commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r713507093
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+
+ self._plotter, self._render_rules = get_plotter_and_render_rules(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
Review comment:
could you say why?
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+
+ self._plotter, self._render_rules = get_plotter_and_render_rules(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ if type(node) in self._render_rules: # pylint: disable=unidiomatic-typecheck
+ graph_info, edge_info = self._render_rules[type(node)](
+ node, relay_param, node_to_id
+ )
+ if graph_info:
+ graph.node(*graph_info)
+ for edge in edge_info:
+ graph.edge(*edge)
+ else:
+ unknown_info = "Unknown node: {}".format(type(node))
+ _LOGGER.warning(unknown_info)
+ graph.node(node_id, unknown_type, unknown_info)
+
+ def render(self, filename):
+ return self._plotter.render(filename=filename)
+
+
+def get_plotter_and_render_rules(backend):
+ """Specify the Plottor and its render rules
+
+ Parameters
+ ----------
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+ if type(backend) is tuple and len(backend) == 2: # pylint: disable=unidiomatic-typecheck
+ if not isinstance(backend[0], Plotter):
+ raise ValueError("First elemnet of the backend should be a plotter")
+ plotter = backend[0]
+ if not isinstance(backend[1], RenderCallback):
+ raise ValueError("Second elemnet of the backend should be a callback")
+ render = backend[1]
+ render_rules = render.get_rules()
+ return plotter, render_rules
+
+ if backend in PlotterBackend:
+ if backend == PlotterBackend.BOKEH:
+ # pylint: disable=import-outside-toplevel
Review comment:
why do you late-import? add a comment explaining why if you need to do this.
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
Review comment:
@kueitang if you have time, would be awesome to add type annotations here
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+
+ self._plotter, self._render_rules = get_plotter_and_render_rules(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
Review comment:
any reason to carry this as a class variable?
##########
File path: python/tvm/contrib/relay_viz/README.md
##########
@@ -0,0 +1,60 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+# IR Visualization
+
+This tool target to visualize Relay IR.
+
+# Table of Contents
+1. [Requirement](#Requirement)
+2. [Usage](#Usage)
+3. [Credits](#Credits)
+
+## Requirement
+
+1. TVM
+2. graphviz
+2. pydot
+3. bokeh >= 2.3.1
+
+```
+# To install TVM, please refer to https://tvm.apache.org/docs/install/from_source.html
+
+# requirements of pydot
Review comment:
i'd ideally like to add these to `python/gen_requirements.py`, but i'm not sure it's the best idea. `bokeh` in particular is pretty heavyweight. before we can do that, we'll need to split the IR parsing stuff into another python package which can be depended on from both this utility and TVM. so for now let's leave them out of `gen_requirements.py`.
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+
+ self._plotter, self._render_rules = get_plotter_and_render_rules(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ if type(node) in self._render_rules: # pylint: disable=unidiomatic-typecheck
+ graph_info, edge_info = self._render_rules[type(node)](
+ node, relay_param, node_to_id
+ )
+ if graph_info:
+ graph.node(*graph_info)
+ for edge in edge_info:
+ graph.edge(*edge)
+ else:
+ unknown_info = "Unknown node: {}".format(type(node))
+ _LOGGER.warning(unknown_info)
+ graph.node(node_id, unknown_type, unknown_info)
+
+ def render(self, filename):
+ return self._plotter.render(filename=filename)
+
+
+def get_plotter_and_render_rules(backend):
+ """Specify the Plottor and its render rules
+
+ Parameters
+ ----------
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+ if type(backend) is tuple and len(backend) == 2: # pylint: disable=unidiomatic-typecheck
+ if not isinstance(backend[0], Plotter):
+ raise ValueError("First elemnet of the backend should be a plotter")
Review comment:
nit: element
##########
File path: python/tvm/contrib/relay_viz/README.md
##########
@@ -0,0 +1,60 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+# IR Visualization
Review comment:
@kueitang any interest in adding a tutorial for this in `tutorials/`?
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+
+ self._plotter, self._render_rules = get_plotter_and_render_rules(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ if type(node) in self._render_rules: # pylint: disable=unidiomatic-typecheck
+ graph_info, edge_info = self._render_rules[type(node)](
+ node, relay_param, node_to_id
+ )
+ if graph_info:
+ graph.node(*graph_info)
+ for edge in edge_info:
+ graph.edge(*edge)
+ else:
+ unknown_info = "Unknown node: {}".format(type(node))
+ _LOGGER.warning(unknown_info)
+ graph.node(node_id, unknown_type, unknown_info)
+
+ def render(self, filename):
+ return self._plotter.render(filename=filename)
+
+
+def get_plotter_and_render_rules(backend):
+ """Specify the Plottor and its render rules
+
+ Parameters
+ ----------
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+ if type(backend) is tuple and len(backend) == 2: # pylint: disable=unidiomatic-typecheck
+ if not isinstance(backend[0], Plotter):
+ raise ValueError("First elemnet of the backend should be a plotter")
+ plotter = backend[0]
+ if not isinstance(backend[1], RenderCallback):
+ raise ValueError("Second elemnet of the backend should be a callback")
+ render = backend[1]
+ render_rules = render.get_rules()
+ return plotter, render_rules
+
+ if backend in PlotterBackend:
Review comment:
rather than handle the good case here, handle the bad case and bail:
```
if backend not in PlotterBackend:
raise ValueError(...)
```
then the rest of the function can un-indent.
##########
File path: python/tvm/contrib/relay_viz/_bokeh.py
##########
@@ -0,0 +1,484 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Bokeh backend for Relay IR Visualizer."""
+import html
+import logging
+import functools
+
+import numpy as np
+import pydot
+
+from bokeh.io import output_file, save
+from bokeh.models import (
+ ColumnDataSource,
+ CustomJS,
+ Text,
+ Rect,
+ HoverTool,
+ MultiLine,
+ Legend,
+ Scatter,
+ Plot,
+ TapTool,
+ PanTool,
+ ResetTool,
+ WheelZoomTool,
+ SaveTool,
+)
+from bokeh.palettes import (
+ d3,
+)
+from bokeh.layouts import column
+
+from .plotter import (
+ Plotter,
+ Graph,
+)
+
+from .render_callback import RenderCallback # pylint: disable=import-outside-toplevel
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class BokehRenderCallback(RenderCallback):
+ pass
+
+
+class NodeDescriptor:
+ """Descriptor used by Bokeh plotter."""
+
+ def __init__(self, node_id, node_type, node_detail):
+ self._node_id = node_id
+ self._node_type = node_type
+ self._node_detail = node_detail
+
+ @property
+ def node_id(self):
+ return self._node_id
+
+ @property
+ def node_type(self):
+ return self._node_type
+
+ @property
+ def detail(self):
+ return self._node_detail
+
+
+class GraphShaper:
+ """Provide the bounding-box, and node location, height, width given by pygraphviz."""
+
+ # defined by graphviz.
+ _px_per_inch = 72
Review comment:
i think style convention is `_PX_PER_INCH`
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from enum import Enum
+from tvm import relay
+from .plotter import Plotter
+from .render_callback import RenderCallback
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+
+ self._plotter, self._render_rules = get_plotter_and_render_rules(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ if type(node) in self._render_rules: # pylint: disable=unidiomatic-typecheck
+ graph_info, edge_info = self._render_rules[type(node)](
+ node, relay_param, node_to_id
+ )
+ if graph_info:
+ graph.node(*graph_info)
+ for edge in edge_info:
+ graph.edge(*edge)
+ else:
+ unknown_info = "Unknown node: {}".format(type(node))
+ _LOGGER.warning(unknown_info)
+ graph.node(node_id, unknown_type, unknown_info)
+
+ def render(self, filename):
+ return self._plotter.render(filename=filename)
+
+
+def get_plotter_and_render_rules(backend):
+ """Specify the Plottor and its render rules
+
+ Parameters
+ ----------
+ backend: PlotterBackend or a tuple
+ PlotterBackend: The backend of plotting. Default "bokeh"
+ Tuple: A tuple with two arguments. First is user-defined Plotter, \
+ the second is user-defined RenderCallback
+ """
+ if type(backend) is tuple and len(backend) == 2: # pylint: disable=unidiomatic-typecheck
Review comment:
isinstance(backend, tuple), no? also, should assert the tuple length is 2 rather than allowing tuples of length != 2 right?
##########
File path: python/tvm/contrib/relay_viz/_bokeh.py
##########
@@ -0,0 +1,484 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Bokeh backend for Relay IR Visualizer."""
+import html
+import logging
+import functools
+
+import numpy as np
+import pydot
+
+from bokeh.io import output_file, save
+from bokeh.models import (
+ ColumnDataSource,
+ CustomJS,
+ Text,
+ Rect,
+ HoverTool,
+ MultiLine,
+ Legend,
+ Scatter,
+ Plot,
+ TapTool,
+ PanTool,
+ ResetTool,
+ WheelZoomTool,
+ SaveTool,
+)
+from bokeh.palettes import (
+ d3,
+)
+from bokeh.layouts import column
+
+from .plotter import (
+ Plotter,
+ Graph,
+)
+
+from .render_callback import RenderCallback # pylint: disable=import-outside-toplevel
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class BokehRenderCallback(RenderCallback):
+ pass
+
+
+class NodeDescriptor:
+ """Descriptor used by Bokeh plotter."""
+
+ def __init__(self, node_id, node_type, node_detail):
+ self._node_id = node_id
+ self._node_type = node_type
+ self._node_detail = node_detail
+
+ @property
+ def node_id(self):
+ return self._node_id
+
+ @property
+ def node_type(self):
+ return self._node_type
+
+ @property
+ def detail(self):
+ return self._node_detail
+
+
+class GraphShaper:
+ """Provide the bounding-box, and node location, height, width given by pygraphviz."""
+
+ # defined by graphviz.
+ _px_per_inch = 72
+
+ def __init__(self, pydot_graph, prog="dot", args=None):
+ if args is None:
+ args = []
+ # call the graphviz program to get layout
+ pydot_graph_str = pydot_graph.create([prog] + args, format="dot").decode()
+ # remember original nodes
+ self._nodes = [n.get_name() for n in pydot_graph.get_nodes()]
+ # parse layout
+ pydot_graph = pydot.graph_from_dot_data(pydot_graph_str)
+ if len(pydot_graph) != 1:
+ # should be unlikely.
+ _LOGGER.warning(
+ "Got %d pydot graphs. Only the first one will be used.", len(pydot_graph)
+ )
+ self._pydot_graph = pydot_graph[0]
+
+ def get_nodes(self):
+ return self._nodes
+
+ @functools.lru_cache()
+ def get_edge_path(self, start_node_id, end_node_id):
+ """Get explicit path points for MultiLine."""
+ edge = self._pydot_graph.get_edge(str(start_node_id), str(end_node_id))
+ if len(edge) != 1:
+ _LOGGER.warning(
+ "Got %d edges between %s and %s. Only the first one will be used.",
+ len(edge),
+ start_node_id,
+ end_node_id,
+ )
+ edge = edge[0]
+ # filter out quotes and newline
+ pos_str = edge.get_pos().strip('"').replace("\\\n", "")
+ tokens = pos_str.split(" ")
+ s_token = None
+ e_token = None
+ ret_x_pts = []
+ ret_y_pts = []
+ for token in tokens:
+ if token.startswith("e,"):
+ e_token = token
+ elif token.startswith("s,"):
+ s_token = token
+ else:
+ x_str, y_str = token.split(",")
+ ret_x_pts.append(float(x_str))
+ ret_y_pts.append(float(y_str))
+ if s_token is not None:
+ _, x_str, y_str = s_token.split(",")
+ ret_x_pts.insert(0, float(x_str))
+ ret_y_pts.insert(0, float(y_str))
+ if e_token is not None:
+ _, x_str, y_str = e_token.split(",")
+ ret_x_pts.append(float(x_str))
+ ret_y_pts.append(float(y_str))
+
+ return ret_x_pts, ret_y_pts
+
+ @functools.lru_cache()
+ def get_node_pos(self, node_name):
+ pos_str = self._get_node_attr(node_name, "pos", "0,0")
+ return list(map(float, pos_str.split(",")))
+
+ def get_node_height(self, node_name):
+ height_str = self._get_node_attr(node_name, "height", "20")
+ return float(height_str) * self._px_per_inch
+
+ def get_node_width(self, node_name):
+ width_str = self._get_node_attr(node_name, "width", "20")
+ return float(width_str) * self._px_per_inch
+
+ def _get_node_attr(self, node_name, attr_name, default_val):
+
+ node = self._pydot_graph.get_node(str(node_name))
+ if len(node) > 1:
+ _LOGGER.error(
+ "There are %d nodes with the name %s. Randomly choose one.", len(node), node_name
+ )
+ if len(node) == 0:
+ _LOGGER.warning(
+ "%s does not exist in the graph. Use default %s for attribute %s",
+ node_name,
+ default_val,
+ attr_name,
+ )
+ return default_val
+
+ node = node[0]
+ try:
+ val = node.obj_dict["attributes"][attr_name].strip('"')
+ except KeyError:
+ _LOGGER.warning(
+ "%s don't exist in node %s. Use default %s", attr_name, node_name, default_val
+ )
+ val = default_val
+ return val
+
+
+class BokehGraph(Graph):
+ """Use Bokeh library to plot Relay IR."""
+
+ def __init__(self):
+ self._pydot_digraph = pydot.Dot(graph_type="digraph")
+ self._id_to_node = {}
+
+ def node(self, node_id, node_type, node_detail):
+ # need string for pydot
+ node_id = str(node_id)
+ if node_id in self._id_to_node:
+ _LOGGER.warning("node_id %s already exists.", node_id)
+ return
+ self._pydot_digraph.add_node(pydot.Node(node_id, label=node_detail))
+ self._id_to_node[node_id] = NodeDescriptor(node_id, node_type, node_detail)
+
+ def edge(self, id_start, id_end):
+ # need string to pydot
+ id_start, id_end = str(id_start), str(id_end)
+ self._pydot_digraph.add_edge(pydot.Edge(id_start, id_end))
+
+ def render(self, plot):
+ """To draw a Bokeh Graph"""
+ shaper = GraphShaper(
+ self._pydot_digraph,
+ prog="dot",
+ args=["-Grankdir=TB", "-Gsplines=ortho", "-Gfontsize=14", "-Nordering=in"],
+ )
+
+ self._create_graph(plot, shaper)
+
+ self._add_scalable_glyph(plot, shaper)
+ return plot
+
+ def _get_type_to_color_map(self):
+ category20 = d3["Category20"][20]
+ # FIXME: a problem is, for different network we have different color
+ # for the same type.
+ all_types = list({v.node_type for v in self._id_to_node.values()})
+ all_types.sort()
+ if len(all_types) > 20:
+ _LOGGER.warning(
+ "The number of types %d is larger than 20. "
+ "Some colors are re-used for different types.",
+ len(all_types),
+ )
+ type_to_color = {}
+ for idx, t in enumerate(all_types):
+ type_to_color[t] = category20[idx % 20]
+ return type_to_color
+
+ def _create_graph(self, plot, shaper):
+
+ # Add edge first
+ edges = self._pydot_digraph.get_edges()
+ x_path_list = []
+ y_path_list = []
+ for edge in edges:
+ id_start = edge.get_source()
+ id_end = edge.get_destination()
+ x_pts, y_pts = shaper.get_edge_path(id_start, id_end)
+ x_path_list.append(x_pts)
+ y_path_list.append(y_pts)
+
+ multi_line_source = ColumnDataSource({"xs": x_path_list, "ys": y_path_list})
+ edge_line_color = "#888888"
+ edge_line_width = 3
+ multi_line_glyph = MultiLine(line_color=edge_line_color, line_width=edge_line_width)
+ plot.add_glyph(multi_line_source, multi_line_glyph)
+
+ # Then add nodes
+ type_to_color = self._get_type_to_color_map()
+
+ def cnvt_to_html(s):
+ return html.escape(s).replace("\n", "<br>")
+
+ label_to_ids = {}
+ for node_id in shaper.get_nodes():
+ label = self._id_to_node[node_id].node_type
+ if label not in label_to_ids:
+ label_to_ids[label] = []
+ label_to_ids[label].append(node_id)
+
+ renderers = []
+ legend_itmes = []
+ for label, id_list in label_to_ids.items():
+ source = ColumnDataSource(
+ {
+ "x": [shaper.get_node_pos(n)[0] for n in id_list],
+ "y": [shaper.get_node_pos(n)[1] for n in id_list],
+ "width": [shaper.get_node_width(n) for n in id_list],
+ "height": [shaper.get_node_height(n) for n in id_list],
+ "node_detail": [cnvt_to_html(self._id_to_node[n].detail) for n in id_list],
+ "node_type": [label] * len(id_list),
+ }
+ )
+ glyph = Rect(fill_color=type_to_color[label])
+ renderer = plot.add_glyph(source, glyph)
+ # set glyph for interactivity
+ renderer.nonselection_glyph = Rect(fill_color=type_to_color[label])
+ renderer.hover_glyph = Rect(
+ fill_color=type_to_color[label], line_color="firebrick", line_width=3
+ )
+ renderer.selection_glyph = Rect(
+ fill_color=type_to_color[label], line_color="firebrick", line_width=3
+ )
+ # Though it is called "muted_glyph", we actually use it
+ # to emphasize nodes in this renderer.
+ renderer.muted_glyph = Rect(
+ fill_color=type_to_color[label], line_color="firebrick", line_width=3
+ )
+ name = f"{self._get_graph_name(plot)}_{label}"
+ renderer.name = name
+ renderers.append(renderer)
+ legend_itmes.append((label, [renderer]))
+
+ # add legend
+ legend = Legend(
+ items=legend_itmes,
+ title="Click to highlight",
+ inactive_fill_color="firebrick",
+ inactive_fill_alpha=0.2,
+ )
+ legend.click_policy = "mute"
+ legend.location = "top_right"
+ plot.add_layout(legend)
+
+ # add tooltips
+ tooltips = [
+ ("node_type", "@node_type"),
+ ("description", "@node_detail{safe}"),
+ ]
+ inspect_tool = WheelZoomTool()
+ # only render nodes
+ hover_tool = HoverTool(tooltips=tooltips, renderers=renderers)
+ plot.add_tools(PanTool(), TapTool(), inspect_tool, hover_tool, ResetTool(), SaveTool())
+ plot.toolbar.active_scroll = inspect_tool
+
+ def _add_scalable_glyph(self, plot, shaper):
+ nodes = shaper.get_nodes()
+
+ def populate_detail(n_type, n_detail):
+ if n_detail:
+ return f"{n_type}\n{n_detail}"
+ return n_type
+
+ text_source = ColumnDataSource(
+ {
+ "x": [shaper.get_node_pos(n)[0] for n in nodes],
+ "y": [shaper.get_node_pos(n)[1] for n in nodes],
+ "text": [self._id_to_node[n].node_type for n in nodes],
+ "detail": [
+ populate_detail(self._id_to_node[n].node_type, self._id_to_node[n].detail)
+ for n in nodes
+ ],
+ "box_w": [shaper.get_node_width(n) for n in nodes],
+ "box_h": [shaper.get_node_height(n) for n in nodes],
+ }
+ )
+
+ text_glyph = Text(
+ x="x",
+ y="y",
+ text="text",
+ text_align="center",
+ text_baseline="middle",
+ text_font_size={"value": "14px"},
+ )
+ node_annotation = plot.add_glyph(text_source, text_glyph)
+
+ def get_scatter_loc(x_start, x_end, y_start, y_end, end_node):
+ """return x, y, angle as a tuple"""
+ node_x, node_y = shaper.get_node_pos(end_node)
+ node_w = shaper.get_node_width(end_node)
+ node_h = shaper.get_node_height(end_node)
+
+ # only 4 direction
+ if x_end - x_start > 0:
+ return node_x - node_w / 2, y_end, -np.pi / 2
+ if x_end - x_start < 0:
+ return node_x + node_w / 2, y_end, np.pi / 2
+ if y_end - y_start < 0:
+ return x_end, node_y + node_h / 2, np.pi
+ return x_end, node_y - node_h / 2, 0
+
+ scatter_source = {"x": [], "y": [], "angle": []}
+ for edge in self._pydot_digraph.get_edges():
+ id_start = edge.get_source()
+ id_end = edge.get_destination()
+ x_pts, y_pts = shaper.get_edge_path(id_start, id_end)
+ x_loc, y_loc, angle = get_scatter_loc(
+ x_pts[-2], x_pts[-1], y_pts[-2], y_pts[-1], id_end
+ )
+ scatter_source["angle"].append(angle)
+ scatter_source["x"].append(x_loc)
+ scatter_source["y"].append(y_loc)
+
+ scatter_glyph = Scatter(
+ x="x",
+ y="y",
+ angle="angle",
+ size=5,
+ marker="triangle",
+ fill_color="#AAAAAA",
+ fill_alpha=0.8,
+ )
+ edge_end_arrow = plot.add_glyph(ColumnDataSource(scatter_source), scatter_glyph)
+
+ plot.y_range.js_on_change(
+ "start",
+ CustomJS(
+ args=dict(
+ plot=plot,
+ node_annotation=node_annotation,
+ text_source=text_source,
+ edge_end_arrow=edge_end_arrow,
+ ),
+ code="""
+ // fontsize is in px
+ var fontsize = 14
+ // ratio = data_point/px
+ var ratio = (this.end - this.start)/plot.height
+ var text_list = text_source.data["text"]
+ var detail_list = text_source.data["detail"]
+ var box_h_list = text_source.data["box_h"]
+ for(var i = 0; i < text_list.length; i++) {
+ var line_num = Math.floor((box_h_list[i]/ratio) / (fontsize*1.5))
+ if(line_num <= 0) {
+ // relieve for the first line
+ if(Math.floor((box_h_list[i]/ratio) / (fontsize)) > 0) {
+ line_num = 1
+ }
+ }
+ var lines = detail_list[i].split("\\n")
+ lines = lines.slice(0, line_num)
+ text_list[i] = lines.join("\\n")
+ }
+ text_source.change.emit()
+
+ node_annotation.glyph.text_font_size = {value: `${fontsize}px`}
+
+ var new_scatter_size = Math.round(fontsize / ratio)
+ edge_end_arrow.glyph.size = {value: new_scatter_size}
+ """,
+ ),
+ )
+
+ @staticmethod
+ def _get_graph_name(plot):
+ return plot.title
+
+
+class BokehPlotter(Plotter):
+ """Use Bokeh library to plot Relay IR."""
+
+ def __init__(self):
+ self._name_to_graph = {}
+
+ def create_graph(self, name):
+ if name in self._name_to_graph:
+ _LOGGER.warning("Graph name %s exists. ")
Review comment:
%s but no params
##########
File path: python/tvm/contrib/relay_viz/_bokeh.py
##########
@@ -0,0 +1,484 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Bokeh backend for Relay IR Visualizer."""
+import html
+import logging
+import functools
+
+import numpy as np
+import pydot
+
+from bokeh.io import output_file, save
+from bokeh.models import (
+ ColumnDataSource,
+ CustomJS,
+ Text,
+ Rect,
+ HoverTool,
+ MultiLine,
+ Legend,
+ Scatter,
+ Plot,
+ TapTool,
+ PanTool,
+ ResetTool,
+ WheelZoomTool,
+ SaveTool,
+)
+from bokeh.palettes import (
+ d3,
+)
+from bokeh.layouts import column
+
+from .plotter import (
+ Plotter,
+ Graph,
+)
+
+from .render_callback import RenderCallback # pylint: disable=import-outside-toplevel
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class BokehRenderCallback(RenderCallback):
+ pass
+
+
+class NodeDescriptor:
+ """Descriptor used by Bokeh plotter."""
+
+ def __init__(self, node_id, node_type, node_detail):
+ self._node_id = node_id
+ self._node_type = node_type
+ self._node_detail = node_detail
+
+ @property
+ def node_id(self):
+ return self._node_id
+
+ @property
+ def node_type(self):
+ return self._node_type
+
+ @property
+ def detail(self):
+ return self._node_detail
+
+
+class GraphShaper:
+ """Provide the bounding-box, and node location, height, width given by pygraphviz."""
+
+ # defined by graphviz.
+ _px_per_inch = 72
+
+ def __init__(self, pydot_graph, prog="dot", args=None):
Review comment:
can you write docstrings?
##########
File path: python/tvm/contrib/relay_viz/_bokeh.py
##########
@@ -0,0 +1,484 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Bokeh backend for Relay IR Visualizer."""
+import html
+import logging
+import functools
+
+import numpy as np
+import pydot
+
+from bokeh.io import output_file, save
+from bokeh.models import (
+ ColumnDataSource,
+ CustomJS,
+ Text,
+ Rect,
+ HoverTool,
+ MultiLine,
+ Legend,
+ Scatter,
+ Plot,
+ TapTool,
+ PanTool,
+ ResetTool,
+ WheelZoomTool,
+ SaveTool,
+)
+from bokeh.palettes import (
+ d3,
+)
+from bokeh.layouts import column
+
+from .plotter import (
+ Plotter,
+ Graph,
+)
+
+from .render_callback import RenderCallback # pylint: disable=import-outside-toplevel
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class BokehRenderCallback(RenderCallback):
+ pass
+
+
+class NodeDescriptor:
+ """Descriptor used by Bokeh plotter."""
+
+ def __init__(self, node_id, node_type, node_detail):
+ self._node_id = node_id
+ self._node_type = node_type
+ self._node_detail = node_detail
+
+ @property
+ def node_id(self):
+ return self._node_id
+
+ @property
+ def node_type(self):
+ return self._node_type
+
+ @property
+ def detail(self):
+ return self._node_detail
+
+
+class GraphShaper:
+ """Provide the bounding-box, and node location, height, width given by pygraphviz."""
+
+ # defined by graphviz.
+ _px_per_inch = 72
+
+ def __init__(self, pydot_graph, prog="dot", args=None):
+ if args is None:
+ args = []
+ # call the graphviz program to get layout
+ pydot_graph_str = pydot_graph.create([prog] + args, format="dot").decode()
+ # remember original nodes
+ self._nodes = [n.get_name() for n in pydot_graph.get_nodes()]
+ # parse layout
+ pydot_graph = pydot.graph_from_dot_data(pydot_graph_str)
+ if len(pydot_graph) != 1:
+ # should be unlikely.
+ _LOGGER.warning(
+ "Got %d pydot graphs. Only the first one will be used.", len(pydot_graph)
+ )
+ self._pydot_graph = pydot_graph[0]
+
+ def get_nodes(self):
+ return self._nodes
+
+ @functools.lru_cache()
+ def get_edge_path(self, start_node_id, end_node_id):
+ """Get explicit path points for MultiLine."""
+ edge = self._pydot_graph.get_edge(str(start_node_id), str(end_node_id))
+ if len(edge) != 1:
+ _LOGGER.warning(
+ "Got %d edges between %s and %s. Only the first one will be used.",
+ len(edge),
+ start_node_id,
+ end_node_id,
+ )
+ edge = edge[0]
+ # filter out quotes and newline
+ pos_str = edge.get_pos().strip('"').replace("\\\n", "")
+ tokens = pos_str.split(" ")
+ s_token = None
+ e_token = None
+ ret_x_pts = []
+ ret_y_pts = []
+ for token in tokens:
+ if token.startswith("e,"):
+ e_token = token
+ elif token.startswith("s,"):
+ s_token = token
+ else:
+ x_str, y_str = token.split(",")
+ ret_x_pts.append(float(x_str))
+ ret_y_pts.append(float(y_str))
+ if s_token is not None:
+ _, x_str, y_str = s_token.split(",")
+ ret_x_pts.insert(0, float(x_str))
+ ret_y_pts.insert(0, float(y_str))
+ if e_token is not None:
+ _, x_str, y_str = e_token.split(",")
+ ret_x_pts.append(float(x_str))
+ ret_y_pts.append(float(y_str))
+
+ return ret_x_pts, ret_y_pts
+
+ @functools.lru_cache()
+ def get_node_pos(self, node_name):
+ pos_str = self._get_node_attr(node_name, "pos", "0,0")
+ return list(map(float, pos_str.split(",")))
+
+ def get_node_height(self, node_name):
+ height_str = self._get_node_attr(node_name, "height", "20")
+ return float(height_str) * self._px_per_inch
+
+ def get_node_width(self, node_name):
+ width_str = self._get_node_attr(node_name, "width", "20")
+ return float(width_str) * self._px_per_inch
+
+ def _get_node_attr(self, node_name, attr_name, default_val):
+
+ node = self._pydot_graph.get_node(str(node_name))
+ if len(node) > 1:
+ _LOGGER.error(
+ "There are %d nodes with the name %s. Randomly choose one.", len(node), node_name
+ )
+ if len(node) == 0:
+ _LOGGER.warning(
+ "%s does not exist in the graph. Use default %s for attribute %s",
+ node_name,
+ default_val,
+ attr_name,
+ )
+ return default_val
+
+ node = node[0]
+ try:
+ val = node.obj_dict["attributes"][attr_name].strip('"')
+ except KeyError:
+ _LOGGER.warning(
+ "%s don't exist in node %s. Use default %s", attr_name, node_name, default_val
+ )
+ val = default_val
+ return val
+
+
+class BokehGraph(Graph):
+ """Use Bokeh library to plot Relay IR."""
+
+ def __init__(self):
+ self._pydot_digraph = pydot.Dot(graph_type="digraph")
+ self._id_to_node = {}
+
+ def node(self, node_id, node_type, node_detail):
+ # need string for pydot
+ node_id = str(node_id)
+ if node_id in self._id_to_node:
+ _LOGGER.warning("node_id %s already exists.", node_id)
+ return
+ self._pydot_digraph.add_node(pydot.Node(node_id, label=node_detail))
+ self._id_to_node[node_id] = NodeDescriptor(node_id, node_type, node_detail)
+
+ def edge(self, id_start, id_end):
+ # need string to pydot
+ id_start, id_end = str(id_start), str(id_end)
+ self._pydot_digraph.add_edge(pydot.Edge(id_start, id_end))
+
+ def render(self, plot):
+ """To draw a Bokeh Graph"""
+ shaper = GraphShaper(
+ self._pydot_digraph,
+ prog="dot",
+ args=["-Grankdir=TB", "-Gsplines=ortho", "-Gfontsize=14", "-Nordering=in"],
+ )
+
+ self._create_graph(plot, shaper)
+
+ self._add_scalable_glyph(plot, shaper)
+ return plot
+
+ def _get_type_to_color_map(self):
+ category20 = d3["Category20"][20]
+ # FIXME: a problem is, for different network we have different color
+ # for the same type.
+ all_types = list({v.node_type for v in self._id_to_node.values()})
+ all_types.sort()
+ if len(all_types) > 20:
+ _LOGGER.warning(
+ "The number of types %d is larger than 20. "
+ "Some colors are re-used for different types.",
+ len(all_types),
+ )
+ type_to_color = {}
+ for idx, t in enumerate(all_types):
+ type_to_color[t] = category20[idx % 20]
+ return type_to_color
+
+ def _create_graph(self, plot, shaper):
+
+ # Add edge first
+ edges = self._pydot_digraph.get_edges()
+ x_path_list = []
+ y_path_list = []
+ for edge in edges:
+ id_start = edge.get_source()
+ id_end = edge.get_destination()
+ x_pts, y_pts = shaper.get_edge_path(id_start, id_end)
+ x_path_list.append(x_pts)
+ y_path_list.append(y_pts)
+
+ multi_line_source = ColumnDataSource({"xs": x_path_list, "ys": y_path_list})
+ edge_line_color = "#888888"
+ edge_line_width = 3
+ multi_line_glyph = MultiLine(line_color=edge_line_color, line_width=edge_line_width)
+ plot.add_glyph(multi_line_source, multi_line_glyph)
+
+ # Then add nodes
+ type_to_color = self._get_type_to_color_map()
+
+ def cnvt_to_html(s):
+ return html.escape(s).replace("\n", "<br>")
+
+ label_to_ids = {}
+ for node_id in shaper.get_nodes():
+ label = self._id_to_node[node_id].node_type
+ if label not in label_to_ids:
+ label_to_ids[label] = []
+ label_to_ids[label].append(node_id)
+
+ renderers = []
+ legend_itmes = []
+ for label, id_list in label_to_ids.items():
+ source = ColumnDataSource(
+ {
+ "x": [shaper.get_node_pos(n)[0] for n in id_list],
+ "y": [shaper.get_node_pos(n)[1] for n in id_list],
+ "width": [shaper.get_node_width(n) for n in id_list],
+ "height": [shaper.get_node_height(n) for n in id_list],
+ "node_detail": [cnvt_to_html(self._id_to_node[n].detail) for n in id_list],
+ "node_type": [label] * len(id_list),
+ }
+ )
+ glyph = Rect(fill_color=type_to_color[label])
+ renderer = plot.add_glyph(source, glyph)
+ # set glyph for interactivity
+ renderer.nonselection_glyph = Rect(fill_color=type_to_color[label])
+ renderer.hover_glyph = Rect(
+ fill_color=type_to_color[label], line_color="firebrick", line_width=3
+ )
+ renderer.selection_glyph = Rect(
+ fill_color=type_to_color[label], line_color="firebrick", line_width=3
+ )
+ # Though it is called "muted_glyph", we actually use it
+ # to emphasize nodes in this renderer.
+ renderer.muted_glyph = Rect(
+ fill_color=type_to_color[label], line_color="firebrick", line_width=3
+ )
+ name = f"{self._get_graph_name(plot)}_{label}"
+ renderer.name = name
+ renderers.append(renderer)
+ legend_itmes.append((label, [renderer]))
+
+ # add legend
+ legend = Legend(
+ items=legend_itmes,
+ title="Click to highlight",
+ inactive_fill_color="firebrick",
+ inactive_fill_alpha=0.2,
+ )
+ legend.click_policy = "mute"
+ legend.location = "top_right"
+ plot.add_layout(legend)
+
+ # add tooltips
+ tooltips = [
+ ("node_type", "@node_type"),
+ ("description", "@node_detail{safe}"),
+ ]
+ inspect_tool = WheelZoomTool()
+ # only render nodes
+ hover_tool = HoverTool(tooltips=tooltips, renderers=renderers)
+ plot.add_tools(PanTool(), TapTool(), inspect_tool, hover_tool, ResetTool(), SaveTool())
+ plot.toolbar.active_scroll = inspect_tool
+
+ def _add_scalable_glyph(self, plot, shaper):
+ nodes = shaper.get_nodes()
+
+ def populate_detail(n_type, n_detail):
+ if n_detail:
+ return f"{n_type}\n{n_detail}"
+ return n_type
+
+ text_source = ColumnDataSource(
+ {
+ "x": [shaper.get_node_pos(n)[0] for n in nodes],
+ "y": [shaper.get_node_pos(n)[1] for n in nodes],
+ "text": [self._id_to_node[n].node_type for n in nodes],
+ "detail": [
+ populate_detail(self._id_to_node[n].node_type, self._id_to_node[n].detail)
+ for n in nodes
+ ],
+ "box_w": [shaper.get_node_width(n) for n in nodes],
+ "box_h": [shaper.get_node_height(n) for n in nodes],
+ }
+ )
+
+ text_glyph = Text(
+ x="x",
+ y="y",
+ text="text",
+ text_align="center",
+ text_baseline="middle",
+ text_font_size={"value": "14px"},
+ )
+ node_annotation = plot.add_glyph(text_source, text_glyph)
+
+ def get_scatter_loc(x_start, x_end, y_start, y_end, end_node):
+ """return x, y, angle as a tuple"""
+ node_x, node_y = shaper.get_node_pos(end_node)
+ node_w = shaper.get_node_width(end_node)
+ node_h = shaper.get_node_height(end_node)
+
+ # only 4 direction
+ if x_end - x_start > 0:
+ return node_x - node_w / 2, y_end, -np.pi / 2
+ if x_end - x_start < 0:
+ return node_x + node_w / 2, y_end, np.pi / 2
+ if y_end - y_start < 0:
+ return x_end, node_y + node_h / 2, np.pi
+ return x_end, node_y - node_h / 2, 0
+
+ scatter_source = {"x": [], "y": [], "angle": []}
+ for edge in self._pydot_digraph.get_edges():
+ id_start = edge.get_source()
+ id_end = edge.get_destination()
+ x_pts, y_pts = shaper.get_edge_path(id_start, id_end)
+ x_loc, y_loc, angle = get_scatter_loc(
+ x_pts[-2], x_pts[-1], y_pts[-2], y_pts[-1], id_end
+ )
+ scatter_source["angle"].append(angle)
+ scatter_source["x"].append(x_loc)
+ scatter_source["y"].append(y_loc)
+
+ scatter_glyph = Scatter(
+ x="x",
+ y="y",
+ angle="angle",
+ size=5,
+ marker="triangle",
+ fill_color="#AAAAAA",
+ fill_alpha=0.8,
+ )
+ edge_end_arrow = plot.add_glyph(ColumnDataSource(scatter_source), scatter_glyph)
+
+ plot.y_range.js_on_change(
+ "start",
+ CustomJS(
+ args=dict(
+ plot=plot,
+ node_annotation=node_annotation,
+ text_source=text_source,
+ edge_end_arrow=edge_end_arrow,
+ ),
+ code="""
+ // fontsize is in px
+ var fontsize = 14
+ // ratio = data_point/px
+ var ratio = (this.end - this.start)/plot.height
+ var text_list = text_source.data["text"]
+ var detail_list = text_source.data["detail"]
+ var box_h_list = text_source.data["box_h"]
+ for(var i = 0; i < text_list.length; i++) {
+ var line_num = Math.floor((box_h_list[i]/ratio) / (fontsize*1.5))
+ if(line_num <= 0) {
+ // relieve for the first line
+ if(Math.floor((box_h_list[i]/ratio) / (fontsize)) > 0) {
+ line_num = 1
+ }
+ }
+ var lines = detail_list[i].split("\\n")
+ lines = lines.slice(0, line_num)
+ text_list[i] = lines.join("\\n")
+ }
+ text_source.change.emit()
+
+ node_annotation.glyph.text_font_size = {value: `${fontsize}px`}
+
+ var new_scatter_size = Math.round(fontsize / ratio)
+ edge_end_arrow.glyph.size = {value: new_scatter_size}
+ """,
+ ),
+ )
+
+ @staticmethod
+ def _get_graph_name(plot):
+ return plot.title
+
+
+class BokehPlotter(Plotter):
+ """Use Bokeh library to plot Relay IR."""
+
+ def __init__(self):
+ self._name_to_graph = {}
+
+ def create_graph(self, name):
+ if name in self._name_to_graph:
+ _LOGGER.warning("Graph name %s exists. ")
+ else:
+ self._name_to_graph[name] = BokehGraph()
+ return self._name_to_graph[name]
+
+ def render(self, filename):
+ if not filename.endswith(".html"):
+ filename = "{}.html".format(filename)
Review comment:
nit: can use `f"{filename}.html"`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] comaniac commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
comaniac commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-895763722
If the high-level design or the interface needs more feedback, it would be better and more efficient to file a formal RFC at the same time.
https://github.com/apache/tvm-rfcs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-894231259
Yes I consider graphs just as nodes and edges. So the Plotter interface is very simple.
I think depth, or "rank" can be obtained by a topological sort, but indeed maybe we should consider interfaces on different abstraction levels, for easier implementing other kinds of visualizers.
I only have a rough thoughts that we should allow customized visitors in a different interface.
Need some time to chew it over.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] PhilippvK edited a comment on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
PhilippvK edited a comment on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-923930909
@chiwwang This might be helpful for generating BYOC partitioned models: [`test_byoc_microtvm`](https://github.com/apache/tvm/blob/44d3934be5d33590ba63139f9b756b05aec9d5c5/tests/python/relay/aot/test_crt_aot.py#L315)
I have recently written a similar test case wich used a multi-output BYOC subgraph here: [`test_byoc_microtvm_bug`](https://github.com/PhilippvK/tvm/blob/d20ddbd4a6b3a8f4aa5fc1bf467dc4f05c15bb1c/tests/python/relay/aot/test_crt_aot_bug.py#L40)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang edited a comment on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang edited a comment on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-950852024
Hi @areusch, @PhilippvK, @jroesch, @leandron,
I finish an implementation, with tests, docs, and a tutorial.
You might want to take a look. :)
In short, `RelayVisualizer` class serve as a single entry point for different renderer.
All it does is using interfaces defined in `plotter.py` and `node_edge_gen.py`.
A tutorial show a simple use-case and some possible customization.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] PhilippvK commented on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
PhilippvK commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-923930909
@chiwwang This might be helpful for generating BYOC partitioned models: https://github.com/PhilippvK/tvm/blob/d20ddbd4a6b3a8f4aa5fc1bf467dc4f05c15bb1c/tests/python/relay/aot/test_crt_aot_bug.py#L40
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] PhilippvK commented on a change in pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
PhilippvK commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r711815547
##########
File path: python/tvm/contrib/relay_viz/_terminal.py
##########
@@ -0,0 +1,145 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Visualize Relay IR in AST text-form"""
+
+from collections import deque
+import functools
+
+from .plotter import (
+ Plotter,
+ Graph,
+)
+
+from .render_callback import RenderCallback
+
+
+class TermRenderCallback(RenderCallback):
+ """Terminal render callback"""
+
+ def call_node(self, node, relay_param, node_to_id):
+ node_id = node_to_id[node]
+ graph_info = [node_id, "Call", ""]
+ edge_info = [[node_to_id[node.op], node_id]]
+ args = [node_to_id[arg] for arg in node.args]
+ for arg in args:
+ edge_info.append([arg, node_id])
+ return graph_info, edge_info
+
+ def let_node(self, node, relay_param, node_to_id):
+ node_id = node_to_id[node]
+ graph_info = [node_id, "Let", "(var, val, body)"]
+ edge_info = [[node_to_id[node.var], node_id]]
+ edge_info.append([node_to_id[node.value], node_id])
+ edge_info.append([node_to_id[node.body], node_id])
+ return graph_info, edge_info
+
+ def global_var_node(self, node, relay_param, node_to_id):
+ node_id = node_to_id[node]
+ graph_info = [node_id, "GlobalVar", node.name_hint]
+ edge_info = []
+ return graph_info, edge_info
+
+ def if_node(self, node, relay_param, node_to_id):
+ node_id = node_to_id[node]
+ graph_info = [node_id, "If", "(cond, true, false)"]
+ edge_info = [[node_to_id[node.cond], node_id]]
+ edge_info.append([node_to_id[node.true_branch], node_id])
+ edge_info.append([node_to_id[node.false_branch], node_id])
+ return graph_info, edge_info
+
+ def op_node(self, node, relay_param, node_to_id):
+ node_id = node_to_id[node]
+ op_name = node.name
+ graph_info = [node_id, op_name, ""]
+ edge_info = []
+ return graph_info, edge_info
+
+
+class Node:
+ def __init__(self, node_type, other_info):
+ self.type = node_type
+ self.other_info = other_info.replace("\n", ", ")
+
+
+class TermGraph(Graph):
+ """Terminal plot for a relay IR Module"""
+
+ def __init__(self, name):
+ # node_id: [ connected node_id]
+ self._name = name
+ self._graph = {}
+ self._id_to_node = {}
+ # reversed post order
+ self._node_ids_rpo = deque()
+
+ def node(self, node_id, node_type, node_detail):
+ # actually we just need the last one.
+ self._node_ids_rpo.appendleft(node_id)
+
+ if node_id not in self._graph:
+ self._graph[node_id] = []
+
+ node = Node(node_type, node_detail)
+ self._id_to_node[node_id] = node
+
+ def edge(self, id_start, id_end):
+ # want reserved post-order
+ if id_end in self._graph:
+ self._graph[id_end].append(id_start)
+ else:
+ self._graph[id_end] = [id_start]
+
+ def render(self):
+ """To draw a terminal graph"""
+ lines = []
+
+ @functools.lru_cache()
+ def gen_line(indent, n_id):
+ conn_symbol = ["|--", "`--"]
+ last = len(self._graph[n_id]) - 1
+ for i, next_n_id in enumerate(self._graph[n_id]):
+ node = self._id_to_node[next_n_id]
+ lines.append(f"{indent}{conn_symbol[i==last]}{node.type} {node.other_info}")
+ next_indent = indent
+ next_indent += " " if (i == last) else "| "
+ gen_line(next_indent, next_n_id)
+
+ first_node_id = self._node_ids_rpo[0]
+ node = self._id_to_node[first_node_id]
+ lines.append(f"@{self._name}({node.other_info})")
+ gen_line("", first_node_id)
+
+ return "\n".join(lines)
+
+
+class TermPlotter(Plotter):
+ """Terminal plotter"""
+
+ def __init__(self):
+ self._name_to_graph = {}
+
+ def create_graph(self, name):
+ self._name_to_graph[name] = TermGraph(name)
+ return self._name_to_graph[name]
+
+ def render(self, filename):
+ # if filename == "stdio", print to terminal.
+ # Otherwise, print to the file
Review comment:
It looks to me like this will always print to `stdio`, even if a filename ist passed to the `render` function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707142274
##########
File path: python/tvm/contrib/relay_viz/README.md
##########
@@ -0,0 +1,61 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+# IR Visualization
+
+This tool target to visualize Relay IR.
+
+# Table of Contents
+1. [Requirement](#Requirement)
+2. [Usage](#Usage)
+3. [Credits](#Credits)
+3. [TODO](#TODO)
Review comment:
Okay, will remove.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707145621
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+ # self.test_type = set()
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ # self.test_type.add(type(node))
Review comment:
Yes. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-919620384
Hi @chiwwang @leandron @idoudali @comaniac ,
Thanks for those comments, please see if there's anything need to change. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] PhilippvK edited a comment on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
PhilippvK edited a comment on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-923930909
@chiwwang This might be helpful for generating BYOC partitioned models: [`test_byoc_microtvm`](https://github.com/apache/tvm/blob/44d3934be5d33590ba63139f9b756b05aec9d5c5/tests/python/relay/aot/test_crt_aot.py#L315)
I have recently written a similar test case wich used multiple BYOC subgraphs here: [`test_byoc_microtvm_bug`](https://github.com/PhilippvK/tvm/blob/d20ddbd4a6b3a8f4aa5fc1bf467dc4f05c15bb1c/tests/python/relay/aot/test_crt_aot_bug.py#L40)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
areusch commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-933913528
thanks @chiwwang ! ping me when it's ready for review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-950852024
Hi @areusch, @PhilippvK, @jroesch, @leandron,
I finish an implementation, with tests, docs, and a tutorial.
You might want to take a :)
In short, `RelayVisualizer` class serve as a single entry point for different renderer.
All it does is using interfaces defined in `plotter.py` and `node_edge_gen.py`.
A tutorial show a simple use-case and some possible customization.
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-912908596
@kueitang This looks good!
Uses can construct RelayVisualizer() with various PlotterBackend to get different render results.
Who want to implement new PlotterBackend can also leverage some efforts from default `render_callbacks.py`
Just few comments above :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-986889264
Thanks @huajsj for reviewing!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-903155532
Hi @kueitang
Maybe we can re-use this RFC, https://discuss.tvm.apache.org/t/rfc-visualizing-relay-program-as-graph/4825
There are some valuable thoughts there :)
Back to interfaces of Relay visualization, how do you think about a backend-specific render-callback? Specifically, in the constructor of `RelayVisualizer` in #8448 :
```python
class RelayVisualizer:
"""Relay IR Visualizer"""
def __init__(
self, relay_mod, relay_param=None, plotter_be=PlotterBackend.BOKEH, render_cb=_dft_render_cb
):
```
I try to port this PR with the interface in the following commit in the file `_terminal.py`.
https://github.com/chiwwang/tvm/commit/4d1e14a1fa0fdef09ac3c8a7f0556588a276e5f1
(I also make the callback coupled with the backend....but this seems not a good idea. A common render-callback can save efforts of implementing a new backend.)
The output looks like:
```
`@main([Var(x)])
`--Call multiply
|--Var(Input)
`--Call add
|--Var(Input)
`--Constshape: (), dtype: int32`
```
I don't know if I should consider multiply/add as an argument of `Call`, so I put it in `Call` string.
This might be fixed by modifying tmv.ir.Op branch in the callback.
So, currently interfaces of relay-viz in my mind is:
1. class Plotter and class Graph in plotter.py (one plotter can have lots of graphs)
2. If needed, a backend-specific render-callback can overwrite the default one, with the prototype
```python
def render_cb(graph, node_to_id, relay_param)
"""
Parameters
----------
graph : class plotter.Graph
node_to_id : Dict[relay.expr, int]
relay_param : Dict[string, NDarray]
"""
```
Additionally, the dict `node_to_id` ensure its order of keys is in the post-order of a relay `GlobalVar`.
How do you think this kind of integration?
Thanks :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on a change in pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
huajsj commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r760692572
##########
File path: python/tvm/contrib/relay_viz/node_edge_gen.py
##########
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""NodeEdgeGenerator interface for :py:class:`tvm.contrib.relay_viz.plotter.Graph`."""
+import abc
+from typing import (
+ Dict,
+ Union,
+ Tuple,
+ List,
+)
+import tvm
+from tvm import relay
+
+UNKNOWN_TYPE = "unknown"
+
+
+class VizNode:
+ """Node carry information used by `plotter.Graph` interface."""
Review comment:
need parameters docstring
##########
File path: python/tvm/contrib/relay_viz/node_edge_gen.py
##########
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""NodeEdgeGenerator interface for :py:class:`tvm.contrib.relay_viz.plotter.Graph`."""
+import abc
+from typing import (
+ Dict,
+ Union,
+ Tuple,
+ List,
+)
+import tvm
+from tvm import relay
+
+UNKNOWN_TYPE = "unknown"
+
+
+class VizNode:
+ """Node carry information used by `plotter.Graph` interface."""
+
+ def __init__(self, node_id: Union[int, str], node_type: str, node_detail: str):
+ self._id = node_id
+ self._type = node_type
+ self._detail = node_detail
+
+ @property
+ def identity(self) -> Union[int, str]:
+ return self._id
+
+ @property
+ def type_str(self) -> str:
+ return self._type
+
+ @property
+ def detail(self) -> str:
+ return self._detail
+
+
+class VizEdge:
+ """Edges for `plotter.Graph` interface."""
+
Review comment:
ditto
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,143 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+from typing import (
+ Dict,
+ Tuple,
+ Union,
+)
+from enum import Enum
+import tvm
+from tvm import relay
+from .plotter import Plotter
+from .node_edge_gen import NodeEdgeGenerator
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotter backends."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer
+
+ Parameters
+ ----------
+ relay_mod : tvm.IRModule
+ Relay IR module.
+ relay_param: None | Dict[str, tvm.runtime.NDArray]
+ Relay parameter dictionary. Default `None`.
+ backend: PlotterBackend | Tuple[Plotter, NodeEdgeGenerator]
+ The backend used to render graphs. It can be a tuple of an implemented Plotter instance and
+ NodeEdgeGenerator instance to introduce customized parsing and visualization logics.
+ Default ``PlotterBackend.TERMINAL``.
+ """
+
+ def __init__(
+ self,
+ relay_mod: tvm.IRModule,
+ relay_param: Union[None, Dict[str, tvm.runtime.NDArray]] = None,
+ backend: Union[PlotterBackend, Tuple[Plotter, NodeEdgeGenerator]] = PlotterBackend.TERMINAL,
+ ):
+
+ self._plotter, self._ne_generator = get_plotter_and_generator(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ # Then main function can be shown on the top.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ node_to_id = {}
+
+ def traverse_expr(node):
+ if node in node_to_id:
+ return
+ node_to_id[node] = len(node_to_id)
+
+ for name in graph_names:
+ node_to_id.clear()
+ relay.analysis.post_order_visit(relay_mod[name], traverse_expr)
+ graph = self._plotter.create_graph(name)
+ self._add_nodes(graph, node_to_id, self._relay_param)
+
+ def _add_nodes(self, graph, node_to_id, relay_param):
+ """add nodes and to the graph.
+
+ Parameters
+ ----------
+ graph : plotter.Graph
+
+ node_to_id : Dict[relay.expr, str | int]
+
+ relay_param : Dict[str, tvm.runtime.NDarray]
+ """
+ for node in node_to_id:
+ node_info, edge_info = self._ne_generator.get_node_edges(node, relay_param, node_to_id)
+ if node_info is not None:
+ graph.node(node_info.identity, node_info.type_str, node_info.detail)
+ for edge in edge_info:
+ graph.edge(edge.start, edge.end)
+
+ def render(self, filename: str = None) -> None:
+ self._plotter.render(filename=filename)
+
+
+def get_plotter_and_generator(backend):
+ """Specify the Plottor and its NodeEdgeGenerator"""
Review comment:
need "Parameters" and "return".
##########
File path: python/tvm/contrib/relay_viz/node_edge_gen.py
##########
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""NodeEdgeGenerator interface for :py:class:`tvm.contrib.relay_viz.plotter.Graph`."""
+import abc
+from typing import (
+ Dict,
+ Union,
+ Tuple,
+ List,
+)
+import tvm
+from tvm import relay
+
+UNKNOWN_TYPE = "unknown"
+
+
+class VizNode:
+ """Node carry information used by `plotter.Graph` interface."""
+
+ def __init__(self, node_id: Union[int, str], node_type: str, node_detail: str):
+ self._id = node_id
+ self._type = node_type
+ self._detail = node_detail
+
+ @property
+ def identity(self) -> Union[int, str]:
+ return self._id
+
+ @property
+ def type_str(self) -> str:
+ return self._type
+
+ @property
+ def detail(self) -> str:
+ return self._detail
+
+
+class VizEdge:
+ """Edges for `plotter.Graph` interface."""
+
+ def __init__(self, start_node: Union[int, str], end_node: Union[int, str]):
+ self._start_node = start_node
+ self._end_node = end_node
+
+ @property
+ def start(self) -> Union[int, str]:
+ return self._start_node
+
+ @property
+ def end(self) -> Union[int, str]:
+ return self._end_node
+
+
+class NodeEdgeGenerator(abc.ABC):
+ """An interface class to generate nodes and edges information for Graph interfaces."""
+
+ @abc.abstractmethod
+ def get_node_edges(
+ self,
+ node: relay.Expr,
+ relay_param: Dict[str, tvm.runtime.NDArray],
+ node_to_id: Dict[relay.Expr, Union[int, str]],
+ ) -> Tuple[Union[VizNode, None], List[VizEdge]]:
+ """Generate node and edges consumed by Graph interfaces.
+
+ Parameters
+ ----------
+ node : relay.Expr
+ relay.Expr which will be parsed and generate a node and edges.
+
+ relay_param: Dict[str, tvm.runtime.NDArray]
+ relay parameters dictionary.
+
+ node_to_id : Dict[relay.Expr, Union[int, str]]
+ a mapping from relay.Expr to node id which should be unique.
+
+ Returns
+ -------
+ rv1 : Union[VizNode, None]
+ VizNode represent the relay.Expr. If the relay.Expr is not intended to introduce a node
+ to the graph, return None.
+
+ rv2 : List[VizEdge]
+ a list of VizEdge to describe the connectivity of the relay.Expr.
+ Can be empty list to indicate no connectivity.
+ """
+
+
+class DefaultNodeEdgeGenerator(NodeEdgeGenerator):
+ """NodeEdgeGenerator generate for nodes and edges consumed by Graph.
+ This class is a default implementation for common relay types, heavily based on
+ `visualize` function in https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ """
+
+ def __init__(self):
+ self._render_rules = {}
+ self._build_rules()
+
+ def get_node_edges(
+ self,
+ node: relay.Expr,
+ relay_param: Dict[str, tvm.runtime.NDArray],
+ node_to_id: Dict[relay.Expr, Union[int, str]],
+ ) -> Tuple[Union[VizNode, None], List[VizEdge]]:
+ try:
+ node_info, edge_info = self._render_rules[type(node)](node, relay_param, node_to_id)
+ except KeyError:
+ node_info = VizNode(
+ node_to_id[node], UNKNOWN_TYPE, f"don't know how to parse {type(node)}"
+ )
+ edge_info = []
+ return node_info, edge_info
+
+ def _var_node(
+ self,
+ node: relay.Expr,
+ relay_param: Dict[str, tvm.runtime.NDArray],
+ node_to_id: Dict[relay.Expr, Union[int, str]],
+ ) -> Tuple[Union[VizNode, None], List[VizEdge]]:
+ """Render rule for a relay var node"""
+ node_id = node_to_id[node]
+ name_hint = node.name_hint
+ node_detail = f"name_hint: {name_hint}"
+ node_type = "Var(Param)" if name_hint in relay_param else "Var(Input)"
+ if node.type_annotation is not None:
+ if hasattr(node.type_annotation, "shape"):
+ shape = tuple(map(int, node.type_annotation.shape))
+ dtype = node.type_annotation.dtype
+ node_detail = f"name_hint: {name_hint}\nshape: {shape}\ndtype: {dtype}"
+ else:
+ node_detail = f"name_hint: {name_hint}\ntype_annotation: {node.type_annotation}"
+ node_info = VizNode(node_id, node_type, node_detail)
+ edge_info = []
+ return node_info, edge_info
+
+ def _function_node(
+ self,
+ node: relay.Expr,
+ _: Dict[str, tvm.runtime.NDArray], # relay_param
Review comment:
remove relay_param
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
huajsj commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-984220934
Thanks @chiwwang.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] corehalt commented on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
corehalt commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-920608150
Hi @kueitang love this feature, thank you!
This will also allow to visualize a BYOC-partitioned Relay graph? Reading partitioned Relay IR is a bit tedious in some cases and it would be really helpful feature for this use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-1023321581
A follow-up PR: https://github.com/apache/tvm/pull/10085
@kueitang please help to close this thread. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] PhilippvK edited a comment on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
PhilippvK edited a comment on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-923930909
@chiwwang This might be helpful for generating BYOC partitioned models: [test_byoc_microtvm](https://github.com/apache/tvm/blob/44d3934be5d33590ba63139f9b756b05aec9d5c5/tests/python/relay/aot/test_crt_aot.py#L315)
I have recently written a similar test case wich used multiple BYOC subgraphs here: [test_byoc_microtvm_bug](https://github.com/PhilippvK/tvm/blob/d20ddbd4a6b3a8f4aa5fc1bf467dc4f05c15bb1c/tests/python/relay/aot/test_crt_aot_bug.py#L40)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707194947
##########
File path: python/tvm/contrib/relay_viz/plotter.py
##########
@@ -0,0 +1,82 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Abstract class for plotters."""
+import abc
+
+
+class Graph(abc.ABC):
+ """Abstract class for graph.
+
+ Implement this interface for various graph libraries.
+ """
+
+ @abc.abstractmethod
+ def node(self, node_id, node_type, node_detail):
+ """Add a node to the underlying graph.
+
+ Parameters
+ ----------
+ node_id : object
+ Serve as the ID to the node.
+
+ node_type : string
+ the type of the node.
+
+ node_detail : string
+ the description of the node.
+ """
+
+ @abc.abstractmethod
+ def edge(self, id_start, id_end):
+ """Add an edge to the underlying graph.
+
+ Parameters
+ ----------
+ id_start : object
+ the ID to the starting node.
+
+ id_end : object
+ the ID to the ending node.
+ """
+
+class Plotter(abc.ABC):
+ """Abstract class for plotters.
+
+ Implement this interface for various graph libraries.
+ """
+
+ @abc.abstractclassmethod
Review comment:
You are right. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-908956551
1. Users can visualize relay by a single entry point, RelayVisualizer. They could choose different renderers by options(`backend` in `RelayVisualizer` constructor)
2. For those who want to implement new renderer, it is expected to implement Graph, Plotter (in plotter.py). We provide default `RenderCallback` (render_callbak.py) for different relay ops to trigger `Graph` interface.
Specific method for a certain op could be overrided if needed, like [`TermRenderCallback`](https://github.com/apache/tvm/pull/8668/files#diff-a7cebf6faa04dd1c96b95f999d924db02831e272ca32f7b7b4aad117d5ea17c8R33).
@chiwwang @leandron
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r763116588
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,143 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+from typing import (
+ Dict,
+ Tuple,
+ Union,
+)
+from enum import Enum
+import tvm
+from tvm import relay
+from .plotter import Plotter
+from .node_edge_gen import NodeEdgeGenerator
+
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotter backends."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer
+
+ Parameters
+ ----------
+ relay_mod : tvm.IRModule
+ Relay IR module.
+ relay_param: None | Dict[str, tvm.runtime.NDArray]
+ Relay parameter dictionary. Default `None`.
+ backend: PlotterBackend | Tuple[Plotter, NodeEdgeGenerator]
+ The backend used to render graphs. It can be a tuple of an implemented Plotter instance and
+ NodeEdgeGenerator instance to introduce customized parsing and visualization logics.
+ Default ``PlotterBackend.TERMINAL``.
+ """
+
+ def __init__(
+ self,
+ relay_mod: tvm.IRModule,
+ relay_param: Union[None, Dict[str, tvm.runtime.NDArray]] = None,
+ backend: Union[PlotterBackend, Tuple[Plotter, NodeEdgeGenerator]] = PlotterBackend.TERMINAL,
+ ):
+
+ self._plotter, self._ne_generator = get_plotter_and_generator(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ # Then main function can be shown on the top.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ node_to_id = {}
+
+ def traverse_expr(node):
+ if node in node_to_id:
+ return
+ node_to_id[node] = len(node_to_id)
+
+ for name in graph_names:
+ node_to_id.clear()
+ relay.analysis.post_order_visit(relay_mod[name], traverse_expr)
+ graph = self._plotter.create_graph(name)
+ self._add_nodes(graph, node_to_id, self._relay_param)
+
+ def _add_nodes(self, graph, node_to_id, relay_param):
+ """add nodes and to the graph.
+
+ Parameters
+ ----------
+ graph : plotter.Graph
+
+ node_to_id : Dict[relay.expr, str | int]
+
+ relay_param : Dict[str, tvm.runtime.NDarray]
+ """
+ for node in node_to_id:
+ node_info, edge_info = self._ne_generator.get_node_edges(node, relay_param, node_to_id)
+ if node_info is not None:
+ graph.node(node_info.identity, node_info.type_str, node_info.detail)
+ for edge in edge_info:
+ graph.edge(edge.start, edge.end)
+
+ def render(self, filename: str = None) -> None:
+ self._plotter.render(filename=filename)
+
+
+def get_plotter_and_generator(backend):
+ """Specify the Plottor and its NodeEdgeGenerator"""
Review comment:
Done.
##########
File path: python/tvm/contrib/relay_viz/node_edge_gen.py
##########
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""NodeEdgeGenerator interface for :py:class:`tvm.contrib.relay_viz.plotter.Graph`."""
+import abc
+from typing import (
+ Dict,
+ Union,
+ Tuple,
+ List,
+)
+import tvm
+from tvm import relay
+
+UNKNOWN_TYPE = "unknown"
+
+
+class VizNode:
+ """Node carry information used by `plotter.Graph` interface."""
Review comment:
Done.
##########
File path: python/tvm/contrib/relay_viz/node_edge_gen.py
##########
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""NodeEdgeGenerator interface for :py:class:`tvm.contrib.relay_viz.plotter.Graph`."""
+import abc
+from typing import (
+ Dict,
+ Union,
+ Tuple,
+ List,
+)
+import tvm
+from tvm import relay
+
+UNKNOWN_TYPE = "unknown"
+
+
+class VizNode:
+ """Node carry information used by `plotter.Graph` interface."""
+
+ def __init__(self, node_id: Union[int, str], node_type: str, node_detail: str):
+ self._id = node_id
+ self._type = node_type
+ self._detail = node_detail
+
+ @property
+ def identity(self) -> Union[int, str]:
+ return self._id
+
+ @property
+ def type_str(self) -> str:
+ return self._type
+
+ @property
+ def detail(self) -> str:
+ return self._detail
+
+
+class VizEdge:
+ """Edges for `plotter.Graph` interface."""
+
Review comment:
Done.
##########
File path: python/tvm/contrib/relay_viz/node_edge_gen.py
##########
@@ -0,0 +1,250 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""NodeEdgeGenerator interface for :py:class:`tvm.contrib.relay_viz.plotter.Graph`."""
+import abc
+from typing import (
+ Dict,
+ Union,
+ Tuple,
+ List,
+)
+import tvm
+from tvm import relay
+
+UNKNOWN_TYPE = "unknown"
+
+
+class VizNode:
+ """Node carry information used by `plotter.Graph` interface."""
+
+ def __init__(self, node_id: Union[int, str], node_type: str, node_detail: str):
+ self._id = node_id
+ self._type = node_type
+ self._detail = node_detail
+
+ @property
+ def identity(self) -> Union[int, str]:
+ return self._id
+
+ @property
+ def type_str(self) -> str:
+ return self._type
+
+ @property
+ def detail(self) -> str:
+ return self._detail
+
+
+class VizEdge:
+ """Edges for `plotter.Graph` interface."""
+
+ def __init__(self, start_node: Union[int, str], end_node: Union[int, str]):
+ self._start_node = start_node
+ self._end_node = end_node
+
+ @property
+ def start(self) -> Union[int, str]:
+ return self._start_node
+
+ @property
+ def end(self) -> Union[int, str]:
+ return self._end_node
+
+
+class NodeEdgeGenerator(abc.ABC):
+ """An interface class to generate nodes and edges information for Graph interfaces."""
+
+ @abc.abstractmethod
+ def get_node_edges(
+ self,
+ node: relay.Expr,
+ relay_param: Dict[str, tvm.runtime.NDArray],
+ node_to_id: Dict[relay.Expr, Union[int, str]],
+ ) -> Tuple[Union[VizNode, None], List[VizEdge]]:
+ """Generate node and edges consumed by Graph interfaces.
+
+ Parameters
+ ----------
+ node : relay.Expr
+ relay.Expr which will be parsed and generate a node and edges.
+
+ relay_param: Dict[str, tvm.runtime.NDArray]
+ relay parameters dictionary.
+
+ node_to_id : Dict[relay.Expr, Union[int, str]]
+ a mapping from relay.Expr to node id which should be unique.
+
+ Returns
+ -------
+ rv1 : Union[VizNode, None]
+ VizNode represent the relay.Expr. If the relay.Expr is not intended to introduce a node
+ to the graph, return None.
+
+ rv2 : List[VizEdge]
+ a list of VizEdge to describe the connectivity of the relay.Expr.
+ Can be empty list to indicate no connectivity.
+ """
+
+
+class DefaultNodeEdgeGenerator(NodeEdgeGenerator):
+ """NodeEdgeGenerator generate for nodes and edges consumed by Graph.
+ This class is a default implementation for common relay types, heavily based on
+ `visualize` function in https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ """
+
+ def __init__(self):
+ self._render_rules = {}
+ self._build_rules()
+
+ def get_node_edges(
+ self,
+ node: relay.Expr,
+ relay_param: Dict[str, tvm.runtime.NDArray],
+ node_to_id: Dict[relay.Expr, Union[int, str]],
+ ) -> Tuple[Union[VizNode, None], List[VizEdge]]:
+ try:
+ node_info, edge_info = self._render_rules[type(node)](node, relay_param, node_to_id)
+ except KeyError:
+ node_info = VizNode(
+ node_to_id[node], UNKNOWN_TYPE, f"don't know how to parse {type(node)}"
+ )
+ edge_info = []
+ return node_info, edge_info
+
+ def _var_node(
+ self,
+ node: relay.Expr,
+ relay_param: Dict[str, tvm.runtime.NDArray],
+ node_to_id: Dict[relay.Expr, Union[int, str]],
+ ) -> Tuple[Union[VizNode, None], List[VizEdge]]:
+ """Render rule for a relay var node"""
+ node_id = node_to_id[node]
+ name_hint = node.name_hint
+ node_detail = f"name_hint: {name_hint}"
+ node_type = "Var(Param)" if name_hint in relay_param else "Var(Input)"
+ if node.type_annotation is not None:
+ if hasattr(node.type_annotation, "shape"):
+ shape = tuple(map(int, node.type_annotation.shape))
+ dtype = node.type_annotation.dtype
+ node_detail = f"name_hint: {name_hint}\nshape: {shape}\ndtype: {dtype}"
+ else:
+ node_detail = f"name_hint: {name_hint}\ntype_annotation: {node.type_annotation}"
+ node_info = VizNode(node_id, node_type, node_detail)
+ edge_info = []
+ return node_info, edge_info
+
+ def _function_node(
+ self,
+ node: relay.Expr,
+ _: Dict[str, tvm.runtime.NDArray], # relay_param
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] AndrewZhaoLuo commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
AndrewZhaoLuo commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-1027172187
Closing in favor of #10085
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-893386436
Thanks @leandron!
Yes, I have seen #8448. It uses an interface (Plottor) to seperate frontend and backend.
It is believed that the design uses only node and edge information to plot a graph.
As for this PR, there are more information derived from depth that I need to record while traversing.
It seems a little hard to unify the current implementation to that interface, or it needs to be refactored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-923920629
@corehalt Thanks for asking!
I am trying to generate BYOC partitions.
Do you have any material or model/codegen I can refer to?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-1014150738
Sure, I will figure out how to split this PR to smaller patches.
I will raise new PRs and then close this one.
Thanks for suggestions @huajsj !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707146074
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
Review comment:
You're right. Thanks for reminding me. :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
chiwwang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-927778726
After discussing with @kueitang, I will pick up this PR and fix problems according to reviews :)
I have some quick answers here --
Thanks @PhilippvK @corehalt , I tried BYOC partitions.
If I understand correctly, BYOC annotate a segment of codes to use another compiler. The segment of codes is then separated as a `GlobalVar`.
Multiple `GlobalVar` is supported!
So the figure looks like:
In the same html,
Thanks @areusch for reviewing!
I will fix problems in days.
Also, I agree with `bokeh` is too heavyweight to be a dependency.
That's why I use lazy-importing -- `bokeh` can be just optional for users who want to use.
I will add more comments in codes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-908956551
1. Users can visualize relay by a single entry point, RelayVisualizer. They could choose different renderers by options(`backend` in `RelayVisualizer` constructor)
2. For those who want to implement new renderer, it is expected to implement Graph, Plotter (in plotter.py). We provide default `RenderCallback` (render_callbak.py) for different relay ops to trigger `Graph` interface.
Specific method for a certain op could be overrided if needed, like [`TermRenderCallback`](https://github.com/apache/tvm/pull/8668/files#diff-a7cebf6faa04dd1c96b95f999d924db02831e272ca32f7b7b4aad117d5ea17c8R33).
@chiwwang @leandron
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702235001
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+ # self.test_type = set()
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ # self.test_type.add(type(node))
+ if type(node) in self.render_rules:
+ graph_info, edge_info = self.render_rules[type(node)](node, relay_param, node_to_id)
+ if graph_info:
+ graph.node(*graph_info)
+ while edge_info:
+ this_edge = edge_info.pop(0)
Review comment:
Why we need to use "pop" here? Couldn't we just do something like
```python
for edge in edge_info:
graph.edge(*edge)
```
Is there any implication that we need to use `pop`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702234680
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
Review comment:
Maybe change the function name because it is returning .render_rules, not render_cb, right?
Also nit: changing self.render_rules to self._render_rules can indicate this member is private.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] huajsj commented on pull request #8668: Visualization of Relay IR
Posted by GitBox <gi...@apache.org>.
huajsj commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-995160485
@kueitang @chiwwang, do you think it is possible to split this PR into couple small patches? a smaller patch may help to accelerate the process and more review friendly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702235850
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
Review comment:
Maybe we can use collections.OrderedDict() to explicitly show the order is important. (It's post-order of a Relay GlobalVar.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702235184
##########
File path: python/tvm/contrib/relay_viz/plotter.py
##########
@@ -0,0 +1,82 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Abstract class for plotters."""
+import abc
+
+
+class Graph(abc.ABC):
+ """Abstract class for graph.
+
+ Implement this interface for various graph libraries.
+ """
+
+ @abc.abstractmethod
+ def node(self, node_id, node_type, node_detail):
+ """Add a node to the underlying graph.
+
+ Parameters
+ ----------
+ node_id : object
+ Serve as the ID to the node.
+
+ node_type : string
+ the type of the node.
+
+ node_detail : string
+ the description of the node.
+ """
+
+ @abc.abstractmethod
+ def edge(self, id_start, id_end):
+ """Add an edge to the underlying graph.
+
+ Parameters
+ ----------
+ id_start : object
+ the ID to the starting node.
+
+ id_end : object
+ the ID to the ending node.
+ """
+
+class Plotter(abc.ABC):
+ """Abstract class for plotters.
+
+ Implement this interface for various graph libraries.
+ """
+
+ @abc.abstractclassmethod
Review comment:
should be `@abc.abstractmethod`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707142147
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+ # self.test_type = set()
Review comment:
Fogot to remove it. Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702234463
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+ # self.test_type = set()
Review comment:
What is this line for?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron edited a comment on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
leandron edited a comment on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-896016803
> If the high-level design or the interface needs more feedback, it would be better and more efficient to file a formal RFC at the same time.
>
> https://github.com/apache/tvm-rfcs
Agree with @comaniac. With an RFC, perhaps we could also find alignment to have this and other features e.g. #8448, to be built on top of the same interface, and also other parts consuming this (I can imagine `tvmc` as a consumer) to also be able to integrate better with a single and organized interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707275564
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
Review comment:
Thanks @inadob! It's a great idea. I make `backend` could take user-specified plotter/ rendercallback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] leandron commented on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
leandron commented on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-896016803
> If the high-level design or the interface needs more feedback, it would be better and more efficient to file a formal RFC at the same time.
>
> https://github.com/apache/tvm-rfcs
Agree with @comaniac. With an RFC, perhaps we could also find alignment to have this and other features e.g. #8448, to be built on top of the same interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] idoudali commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
idoudali commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r704060404
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
Review comment:
Could we change the interface and allow backend to be a union:
* Either a string of the backend name
* Or a tuple of plotter, render_rules objects ?
This way we can avoid having to implement any new visualizers under this folder and registering them in the get_plotter_and_render_cb function.
A custom visualizer could be written as an external tool to the TVM repo that implements the necessary Plotter/Graph interfaces and imports this code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702234732
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+ # self.test_type = set()
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ # self.test_type.add(type(node))
Review comment:
Should we remove this line?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang edited a comment on pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang edited a comment on pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#issuecomment-912908596
@kueitang This looks good!
Uses can construct `RelayVisualizer` with various `PlotterBackend` to get different render results.
Who want to implement new `PlotterBackend` can also leverage some efforts from default `render_callbacks.py`
Just few comments above :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707233922
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
Review comment:
Dict keeps insertion order after python3.7. Current tvm uses Python 3.6, for the CPython implementation of Python, dictionaries remember the order of items inserted.
Also Python 3.6 ends its life on 23/12/2021. Maybe we could keep it a dictionary right now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kueitang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
kueitang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r707192420
##########
File path: python/tvm/contrib/relay_viz/__init__.py
##########
@@ -0,0 +1,128 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Relay IR Visualizer"""
+import logging
+import copy
+from tvm import relay
+from enum import Enum
+
+_LOGGER = logging.getLogger(__name__)
+
+class PlotterBackend(Enum):
+ """Enumeration for available plotters."""
+
+ BOKEH = "bokeh"
+ TERMINAL = "terminal"
+
+
+class RelayVisualizer:
+ """Relay IR Visualizer"""
+
+ def __init__(
+ self, relay_mod, relay_param=None, backend=PlotterBackend.BOKEH
+ ):
+ """Visualize Relay IR.
+
+ Parameters
+ ----------
+ relay_mod : object
+ Relay IR module
+ relay_param: dict
+ Relay parameter dictionary
+ backend: PlotterBackend.
+ The backend of plotting. Default "bokeh"
+ """
+
+ self._plotter, self.render_rules = get_plotter_and_render_cb(backend)
+ self._relay_param = relay_param if relay_param is not None else {}
+ # This field is used for book-keeping for each graph.
+ self._node_to_id = {}
+ # self.test_type = set()
+
+ global_vars = relay_mod.get_global_vars()
+ graph_names = []
+ # If we have main function, put it to the first.
+ for gv_name in global_vars:
+ if gv_name.name_hint == "main":
+ graph_names.insert(0, gv_name.name_hint)
+ else:
+ graph_names.append(gv_name.name_hint)
+
+ for name in graph_names:
+ # clear previous graph
+ self._node_to_id = {}
+ relay.analysis.post_order_visit(
+ relay_mod[name],
+ lambda node: self._traverse_expr(node), # pylint: disable=unnecessary-lambda
+ )
+ graph = self._plotter.create_graph(name)
+ # shallow copy to prevent callback modify self._node_to_id
+ self._render_cb(graph, copy.copy(self._node_to_id), self._relay_param)
+
+ def _traverse_expr(self, node):
+ # based on https://github.com/apache/tvm/pull/4370
+ if node in self._node_to_id:
+ return
+ self._node_to_id[node] = len(self._node_to_id)
+
+ def _render_cb(self, graph, node_to_id, relay_param):
+ """a callback to Add nodes and edges to the graph.
+
+ Parameters
+ ----------
+ graph : class plotter.Graph
+
+ node_to_id : Dict[relay.expr, int]
+
+ relay_param : Dict[string, NDarray]
+ """
+ # Based on https://tvm.apache.org/2020/07/14/bert-pytorch-tvm
+ unknown_type = "unknown"
+ for node, node_id in node_to_id.items():
+ # self.test_type.add(type(node))
+ if type(node) in self.render_rules:
+ graph_info, edge_info = self.render_rules[type(node)](node, relay_param, node_to_id)
+ if graph_info:
+ graph.node(*graph_info)
+ while edge_info:
+ this_edge = edge_info.pop(0)
Review comment:
Good idea, I will change to go through a for loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] chiwwang commented on a change in pull request #8668: [Draft PR] Visualization Relay IR on terminal
Posted by GitBox <gi...@apache.org>.
chiwwang commented on a change in pull request #8668:
URL: https://github.com/apache/tvm/pull/8668#discussion_r702234416
##########
File path: python/tvm/contrib/relay_viz/README.md
##########
@@ -0,0 +1,61 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+
+# IR Visualization
+
+This tool target to visualize Relay IR.
+
+# Table of Contents
+1. [Requirement](#Requirement)
+2. [Usage](#Usage)
+3. [Credits](#Credits)
+3. [TODO](#TODO)
Review comment:
There is no TODO now. I removed this in a certain commit #8448
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org