You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/07/14 08:47:07 UTC
[GitHub] [tvm] MichaelJKlaiber opened a new pull request, #12087: Uma v1.0
MichaelJKlaiber opened a new pull request, #12087:
URL: https://github.com/apache/tvm/pull/12087
UMA v1:
- UMA Partitioner
- UMA Pipeline
- UMA Lower
- UMA Codegen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925569714
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
It is more intended to be a directory for "templates" / skeleton code to be customizes with uma_cli.py for new accelerator targets
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r926342146
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
Review Comment:
@manupa-arm, I agree in general.
IMHO uma_cli is more a helper for the tutorial in its current form, therefore I think it could be of use for early users of UMA.
A deeper embedding into TVM, e.g. via tvmc is definitely a further discussion that we need to have, e.g. on the RFC level
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r926409727
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
Review Comment:
In that case, Would it be better to move it from here to tutorials or apps/ until we have the discussion how to integrate to back to main compiler ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929975111
##########
python/tvm/relay/backend/contrib/uma/api/utils.py:
##########
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Utility methods for the Universal Modular Accelerator Interface (UMA)"""
+
+from enum import Enum, auto
+import uuid
+import tvm.tir
+from tvm.contrib import utils, clang
+
+
+class PassPhase(Enum):
+ """UMA pass phases."""
+
+ PRE_PARTITIONING = auto()
+ POST_PARTITIONING_0 = auto()
Review Comment:
done: see code
```
"""
UMA pass phases:
PRE_PARTITIONING: prior to UMA partitioning
POST_PARTITIONING_0: after UMA partitioning, before Defunctionalization
POST_PARTITIONING_1: after UMA partitioning and after Defunctionalization
TIR_PHASE_0: Generates the raw IR and loop levels.
TIR_PHASE_1: Flattens the array storage.
TIR_PHASE_2: Transforms loops, like unroll, vectorization and thread-binding.
TIR_PHASE_3: Does some cleanup work.
Reference to TIR phases: src/driver/driver_api.c
"""
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930832276
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
Review Comment:
The decorator transform.prim_func_pass prevents me from using class-level constant, so I would tend to leave them in the _my_ai_hw_conv2d_pass function
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939139200
##########
apps/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,95 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+
Review Comment:
done
##########
apps/uma/_template/passes.py:
##########
@@ -0,0 +1,142 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ _EXTERNAL_FUNCTION_NAME = "my_ai_hw_conv2dnchw"
+ _TVM_BLOCK_MATCH_NAME = "conv2d_nchw"
+
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @classmethod
+ def _my_ai_hw_conv2d_pass(cls, func, mod, ctx):
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+
+ def _has_block(name: str, func: tvm.tir.PrimFunc) -> bool:
+ """
+ Determine of a tir.block with `name` exists in `func`
+ """
+
+ def _hb(op):
+ if isinstance(op, tvm.tir.Block):
+ _found_blocks.append(op.name_hint)
+
+ _found_blocks = []
+ tvm.tir.stmt_functor.post_order_visit(func.body, _hb)
+ return name in _found_blocks
+
+ def _detect_and_replace_conv2d(
+ func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ def _replace_conv2d(op):
+ if op == _entry_node:
+ irb = tvm.tir.ir_builder.create()
+ # Collection of buffer address
+ buffers = [b[1].data for b in _handles]
+ # extraction of loop offsets
+ for k, v in _loops.items():
+ assert v.min.value == 0
+ offset_order = ["co", "w", "h", "ci", "kh", "kw"]
+ offsets = [_loops[i].extent.value for i in offset_order]
+ args = buffers + offsets
+ external_call = tvm.tir.Evaluate(
+ tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args)
+ )
+ ext_calls = tvm.tir.SeqStmt([external_call])
+ irb.emit(ext_calls)
+ irb_result = irb.get()
+ return irb_result
+ elif isinstance(op, tvm.tir.SeqStmt):
+ # Remove that pad block of TOPI's conv2DNCHW by only returning the 2nd statement
+ return op.seq[1]
+ return op
+
+ sch = tir.Schedule(func)
+
+ if _has_block(cls._TVM_BLOCK_MATCH_NAME, func):
+ conv2d_block = sch.get_block(cls._TVM_BLOCK_MATCH_NAME)
+ rv_loops = sch.get_loops(conv2d_block)
+ assert len(rv_loops) == 7
+ loops = dict(
+ n=rv_loops[0],
+ co=rv_loops[1],
+ h=rv_loops[2],
+ w=rv_loops[3],
+ ci=rv_loops[4],
+ kh=rv_loops[5],
+ kw=rv_loops[6],
+ )
+ _entry_node = sch.get(rv_loops[1])
+ _loops = {k: sch.get(v) for k, v in loops.items()}
+ _handles = func.buffer_map.items()
+
+ x = tvm.tir.stmt_functor.ir_transform(
+ func.body, None, _replace_conv2d, ["tir.For", "tir.SeqStmt"]
+ )
+ return func.with_body(x)
+ else:
+ return func
+
+ r = _detect_and_replace_conv2d(func, mod, ctx)
+ return r
+
+
+def tir_call(ib: tvm.tir.ir_builder, extern: bool, name: str, *args):
+ """
+ ib: ir_builder
+ extern: bool
+ True --> tvm.tir.call_extern
+ False --> tvm.tir.call_packed
+ name: str
+ function name
+ *args:
+ arguments for function call
+ """
+
+ def buf_from_array(ib, arr, dtype):
+ # Allocate enough memory to store the whole array
+ var = ib.allocate("int32", (len(arr),), scope="global")
+ for i, v in enumerate(arr):
+ var[i] = v
+ # Declare a buffer, which is basically a view on the chunk of memory that we allocated
+ buf = tvm.tir.decl_buffer((len(arr),), dtype, data=var, scope="global")
+ return buf
+
+ if extern:
+ args = [i.data if isinstance(i, tvm.tir.Buffer) else i for i in args]
+ call = tvm.tir.call_extern("int32", name, *args)
+ else:
+ args = [
+ buf_from_array(ib, i, "int32")
+ if isinstance(i, (tuple, list, tvm.ir.container.Array))
+ else i
+ for i in args
+ ]
+ call = tvm.tir.call_packed(name, *args)
+
+ return call
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939160847
##########
apps/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,95 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+
+ /*!
+ * \brief Conv2D function for mock-accelerator examples. Limited to same-padded Conv2D with
+ * stride (1,1) and datatype float. \param ifmap Pointer to input feature map data of size
+ * iw*ih*ic*sizeof(float). \param weights Pointer to weight data of size
+ * kh*kw*ic**oc*sizeof(float). \param result Pointer to output feature map data of size
+ * iw*ih*oc*sizeof(float). \param oc Number of channels of output feature map. \param iw Width
+ * of input feature map, ifmap. \param ih Height of input feature map, ifmap. \param ic Number
+ * of channels of input feature map. \param kh Height of convolution kernels. \param kw Width of
+ * convolution kernels.
+ *
+ * \return error code
+ *
+ */
+ int
+ my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic,
Review Comment:
I agree that it doesnt look linted. It is what
./docker/lint.sh -i clang_format
gives me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939138720
##########
apps/uma/_template/__init__.py:
##########
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+
+Template files for UMA tutorial
+
+Do not import
Review Comment:
Removed the Do not import comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] cbalint13 commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
cbalint13 commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r922537840
##########
CMakeLists.txt:
##########
@@ -494,6 +494,7 @@ include(cmake/modules/contrib/ArmComputeLib.cmake)
include(cmake/modules/contrib/TensorRT.cmake)
include(cmake/modules/contrib/VitisAI.cmake)
include(cmake/modules/contrib/Verilator.cmake)
+include(cmake/modules/contrib/UMA.cmake)
Review Comment:
Can please enumerate it as cmake option too:
+ Add ```tvm_option(USE_UMA "Build with UMA support" OFF)``` after:
https://github.com/apache/tvm/blob/8b917d71b127a3a0ae02a3ee74b8af4744790d97/CMakeLists.txt#L115
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925589594
##########
tests/python/contrib/test_uma/test_uma_lowering_with_umalower.py:
##########
@@ -0,0 +1,115 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import pytest
+import pathlib
+
+import tvm
+from tests.python.contrib.test_uma.test_uma_utils import _create_schedule, _generate_io_arrays
+from tvm import topi
+from tvm.relay.backend.contrib.uma._template.passes import MyAiHwConv2dPass
+import tvm.testing
+from tvm import te
+from tvm.relay.backend.contrib.uma.api.lower import UMALower
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+
+
+def _conv2d_te_definition(shapes: dict) -> list:
+ n, w, h, ci, kw, kh, co = (
+ shapes["n"],
+ shapes["w"],
+ shapes["h"],
+ shapes["ci"],
+ shapes["kw"],
+ shapes["kh"],
+ shapes["co"],
+ )
+ ifmap = te.placeholder((n, ci, w, h), dtype="float32", name="ifmap")
+ weights = te.placeholder((co, ci, kw, kh), dtype="float32", name="weights")
+ result = topi.nn.conv2d_nchw(ifmap, weights, stride=1, padding=[kw // 2, kh // 2], dilation=1)
+ return [ifmap, weights, result]
+
+
+def _pepare_conv2d_schedule(shapes, use_external_conv2d_impl=True):
+ placeholders = _conv2d_te_definition(shapes)
+
+ uma_path = pathlib.Path(str(tvm.relay.backend.contrib.uma.__file__)).parent.absolute()
+ conv2d_file = uma_path / "_template" / "conv2dnchw.cc"
+
+ with conv2d_file.open() as f:
+ sch_tir = _create_schedule(
+ placeholders, f, use_external_conv2d_impl=use_external_conv2d_impl
+ )
+ return placeholders, sch_tir
+
+
+def _run_external_conv2d(dut_io_arrays, conv2d_shapes, target):
+ # Run conv2d with external function
+ placeholders, schedule = _pepare_conv2d_schedule(conv2d_shapes)
+
+ uma_lower = UMALower("lower_test")
+ uma_lower._tir_passes.append((PassPhase.TIR_PHASE_0, MyAiHwConv2dPass()))
+ with tvm.transform.PassContext():
+ tir_mod = uma_lower._lower_stir_to_nstir(schedule.mod["main"])
+
+ ifmap_data, weight_data, result_data = dut_io_arrays
+
+ llvm_conv2d_mod = tvm.build(tir_mod, placeholders, target=target, name="test_external_conv2d")
+ llvm_conv2d_mod(ifmap_data, weight_data, result_data)
+
+
+def _run_reference_conv2d(reference_io_arrays, conv2d_shapes, target):
+ placeholders, schedule = _pepare_conv2d_schedule(conv2d_shapes)
+ ref_mod = tvm.build(schedule.mod, placeholders, target=target, name="test_reference_conv2d")
+ ifmap, weights, result = reference_io_arrays
+ ref_mod(ifmap, weights, result)
+
+
+def _prepare_io_arrays(conv2d_shapes, dev):
+ dut_io_arrays = _generate_io_arrays(conv2d_shapes, dev)
+ _, _, ref_result = _generate_io_arrays(conv2d_shapes, dev)
+ reference_io_arrays = [dut_io_arrays[0], dut_io_arrays[1], ref_result]
+ return dut_io_arrays, reference_io_arrays
+
+
+@pytest.mark.parametrize(
+ "n, w, h, ci, kw, kh, co",
+ [
+ (1, 224, 224, 3, 3, 3, 4),
+ (1, 224, 224, 3, 5, 5, 4),
+ (1, 224, 224, 3, 7, 7, 4),
+ (1, 224, 320, 3, 7, 7, 4),
+ (1, 224, 224, 3, 7, 7, 4),
+ ],
+)
+def test_lower_with_uma(n, w, h, ci, kw, kh, co):
+ target = tvm.target.Target(target="llvm", host="llvm")
+ dev = tvm.device(target.kind.name, 0)
+ conv2d_shapes = dict(n=n, w=w, h=h, ci=ci, kw=kw, kh=kh, co=co)
+
+ dut_io_arrays, reference_io_arrays = _prepare_io_arrays(conv2d_shapes, dev)
+
+ _run_external_conv2d(dut_io_arrays, conv2d_shapes, target)
+ _run_reference_conv2d(reference_io_arrays, conv2d_shapes, target)
+
+ # compare results
+ dut_results = dut_io_arrays[2].numpy()
+ ref_results = reference_io_arrays[2].numpy()
+ tvm.testing.assert_allclose(dut_results, ref_results, rtol=1e-5)
+
+
+if __name__ == "__main__":
+ test_lower_with_uma(1, 224, 224, 3, 3, 3, 4)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929982499
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
+ # Target configuration
+ #######################################################################
+ self._register_target_attr("dimension")
+
+ #######################################################################
+ # Relay to Relay function registration
+ #######################################################################
+ self._register_pattern("conv2d", conv2d_pattern())
+
+ #######################################################################
+ # Relay to TIR function registration
+ #######################################################################
+ self._register_tir_pass(PassPhase.TIR_PHASE_0, MyAiHwConv2dPass())
Review Comment:
@areusch, we don't have strong opinion here.
If the way you propose is preferred by the others, too, then we can make this change
@PaulPalomeroBernardo @cgerum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r928586644
##########
python/tvm/relay/backend/contrib/uma/backend.py:
##########
@@ -0,0 +1,299 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Backend base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from abc import ABC, abstractmethod
+from typing import Union, Dict, Callable, Optional, Any
+
+import tvm
+from tvm.relay.backend.contrib.uma.api.codegen import UMACodegen
+from tvm.relay.backend.contrib.uma.api.lower import UMALower
+from tvm.relay.backend.contrib.uma.api.partitioner import UMAPartitioner
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+
+
+class UMABackend(ABC):
+ """Backend base class of the Universal Modular Accelerator Interface (UMA)"""
+
+ def __init__(self, merge_compiler_regions: bool = True) -> None:
+ self._target_attrs: Dict = {}
+ self._target_preprocessor: Callable[[str], Dict[str, Any]] = None
+ self._relay_to_relay = UMAPartitioner(self.target_name, merge_compiler_regions)
+ self._relay_to_tir = UMALower(self.target_name)
+ self._tir_to_runtime = UMACodegen(self.target_name)
+
+ @property
+ @abstractmethod
+ def target_name(self) -> str:
+ """Name of the hardware target.
+
+ Returns
+ -------
+ out : str
+ The hardware target name.
+ """
+ ...
+
+ ############################################################################
+ # Target configuration
+ ############################################################################
+ def _register_target_attr(
+ self,
+ name: str,
+ default: Optional[Union[str, int, bool]] = "",
+ ) -> None:
+ """Register a target attribute name that can be used during target instantiation.
+ Parameters
+ ----------
+ name: str
+ The name of the target attribute.
+
+ default: Optional[Union[str, int, bool]]
+ A default value for the attribute.
+ If none is provided, the attribute will be treated as a string.
+
+ Example
+ -------
+ Here is an example of how two attribute options are registered.
+
+ .. code-block:: python
+
+ self._register_target_attr("attrA", default=0)
+ self._register_target_attr("attrB", default=False)
+ """
+ self._target_attrs[name] = default
+
+ ############################################################################
+ # Relay to Relay function registration
+ ############################################################################
+ def _register_relay_pass(self, phase: PassPhase, relay_pass: tvm.transform.Pass) -> None:
+ """Registers a relay pass at the given phase in the lowering process.
+
+ Parameters
+ ----------
+ phase: PassPhase
+ The phase at which the pass is registered.
+
+ relay_pass: tvm.transform.Pass
+ The relay pass to be registered.
+
+ Example
+ -------
+ Here is an example of how two relay passes are registered.
+ Passes of the same phase are executed in the order they are registered.
+
+ .. code-block:: python
+
+ self._register_relay_pass(PassPhase.PRE_PARTITIONING, MyPassA)
+ self._register_relay_pass(PassPhase.POST_PARTITIONING, MyPassB)
+
+ Where a relay pass can look like this:
+
+ .. code-block:: python
+
+ @tvm.ir.transform.module_pass(opt_level=0)
+ class MyPassA:
+ def transform_module(self, mod, ctx):
+ # My pass functionality...
+ return mod
+ """
+ self._relay_to_relay._relay_passes.append((phase, relay_pass))
+
+ def _register_pattern(
+ self,
+ name: str,
+ pattern: tvm.relay.dataflow_pattern.DFPattern,
+ predicate: Optional[Callable] = None,
+ ) -> None:
+ """Registers a dataflow pattern that is used to partition the relay graph.
+
+ Parameters
+ ----------
+ name: str
+ The name of the pattern.
+
+ pattern: tvm.relay.dataflow_pattern.DFPattern
+ The dataflow pattern.
+
+ predicate: Callable Receiving the matched pattern and
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] sunggg commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
sunggg commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r938062243
##########
tests/python/contrib/test_uma/test_uma_pipeline.py:
##########
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+from tvm.relay import transform, testing
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
+
+import tvm
+from test_uma_vanilla_accelerator import VanillaAcceleratorBackend
+from tvm import relay
+import numpy as np
+from collections import OrderedDict
+
+from tvm.relay.backend.contrib.uma.api.utils import uma_available
+
+pytestmark = pytest.mark.skipif(not uma_available(), reason="UMA not available")
+
+
+@pytest.mark.parametrize(
+ "interface_api,use_unpacked_api,test_runner,groups,weight_shape",
+ [("c", True, AOT_DEFAULT_RUNNER, 1, 32)],
+)
+def test_conv2d(interface_api, use_unpacked_api, test_runner, groups, weight_shape):
+ """Test a subgraph with a single conv2d operator."""
+ mod, inputs, output_list, test_runner = create_conv2d(groups, test_runner, weight_shape)
+
+ uma_backend = VanillaAcceleratorBackend()
+ uma_backend.register()
+ mod = uma_backend.partition(mod)
+ target = tvm.target.Target("vanilla_accelerator", host=tvm.target.Target("c"))
+
+ compile_and_run(
+ AOTTestModel(module=mod, inputs=inputs, outputs=output_list),
+ test_runner,
+ interface_api,
+ use_unpacked_api,
+ target=target,
+ )
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
+ dtype = "float32"
+ ishape = (1, 32, 14, 14)
+ wshape = (32, weight_shape, 3, 3)
+ pass_config = {"tir.usmp.enable": True}
+ test_runner = AOTTestRunner(
+ makefile=test_runner.makefile,
+ prologue=test_runner.prologue,
+ epilogue=test_runner.epilogue,
+ includes=test_runner.includes,
+ parameters=test_runner.parameters,
+ pass_config=pass_config,
+ )
+ data0 = relay.var("data", shape=ishape, dtype=dtype)
+ weight0 = relay.var("weight", shape=wshape, dtype=dtype)
+ out = relay.nn.conv2d(data0, weight0, kernel_size=(3, 3), padding=(1, 1), groups=groups)
+ main_f = relay.Function([data0, weight0], out)
+ mod = tvm.IRModule()
+ mod["main"] = main_f
+ mod = transform.InferType()(mod)
+ i_data = np.random.uniform(0, 1, ishape).astype(dtype)
+ w1_data = np.random.uniform(0, 1, wshape).astype(dtype)
+ inputs = OrderedDict([("data", i_data), ("weight", w1_data)])
+ output_list = generate_ref_data(mod, inputs)
+ return mod, inputs, output_list, test_runner
+
+
+def _generate_runtime_data(input_shapes: dict, output_shapes: dict) -> [OrderedDict, OrderedDict]:
+ assert len(input_shapes) == 1
+ assert len(output_shapes) == 1
+
+ iname = list(input_shapes.keys())[0]
+ oname = list(output_shapes.keys())[0]
+ ishape = input_shapes[iname]
+ oshape = output_shapes[oname]
+ i_data = np.random.uniform(0, 1, ishape).astype("float32")
+ o_data = np.random.uniform(0, 1, oshape).astype("float32")
+ oname = "output" # name set by relay.build in executor_codegen_metadata.outputs
+ inputs = OrderedDict([(iname, i_data)])
+ outputs = OrderedDict([(oname, o_data)])
+ return inputs, outputs
+
+
+def test_mobilenet():
+ """Full network test with Mobilenet"""
+ use_unpacked_api = True
+ interface_api = "c"
+ test_runner = AOT_DEFAULT_RUNNER
+
+ mod, params = testing.mobilenet.get_workload(batch_size=1)
+
+ uma_backend = VanillaAcceleratorBackend()
+ uma_backend.register()
+ target = tvm.target.Target("vanilla_accelerator", host=tvm.target.Target("c"))
+ target_c = tvm.target.Target("c")
+
+ data_shape = [int(x) for x in mod["main"].checked_type.arg_types[0].shape]
+ data = np.random.uniform(size=data_shape).astype("float32")
+ input_list = {"data": data}
+ output_list = generate_ref_data(mod, input_list, params)
+ mod = uma_backend.partition(mod)
Review Comment:
Nit: It would be more readable if there is a comment that explains how the model would be partitioned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939481576
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
+ self._tir_passes: List[Tuple[PassPhase, tvm.tir.transform.PrimFuncPass]] = []
+
+ def _lower_relay_to_tir(self, relay_prim_func: relay.Function) -> tvm.tir.PrimFunc:
+ """Lower a Relay primitive function to a S-TIR primitive function.
+
+ Parameters
+ ----------
+ prim_func : tvm.relay.Function
+ The Relay function to lower.
+
+ Returns
+ -------
+ out : tvm.tir.PrimFunc
+ The lowered schedulable TensorIR primitive function.
+
+ """
+
+ def _get_tensors(te_cached_func):
+ outputs = list(te_cached_func.outputs)
+ stack = []
+ visited = set()
+ for output_ in outputs:
+ if output_ not in visited:
+ visited.add(output_)
+ stack.append(output_)
+
+ args = []
+ while len(stack) != 0:
+ tensor = stack.pop()
+ if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+ args.append(tensor)
+ elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+ inputs = tensor.op.input_tensors
+ for input_ in inputs:
+ if input_ not in visited:
+ visited.add(input_)
+ stack.append(input_)
+
+ return args + outputs
+
+ f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
+ te_cached_func = f(relay_prim_func)
+ x = _get_tensors(te_cached_func)
+ tir_prim_func = te.create_prim_func(x)
+ tir_prim_func = tir_prim_func.with_attr(
+ "global_symbol", relay_prim_func.attrs["global_symbol"]
+ )
+ # TODO: The target should probably come from somewhere else instead of being created here.
+ tir_prim_func = tir_prim_func.with_attr("target", tvm.target.Target(self.target_name))
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936306933
##########
python/tvm/relay/backend/contrib/uma/api/partitioner.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Partitioner base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Dict, List, Tuple, Optional
+
+import tvm
+from tvm import relay
+from tvm.relay.build_module import bind_params_by_name
+from tvm.relay.op.contrib.register import register_pattern_table
+from .utils import PassPhase
+
+
+PatternTable = List[Tuple[str, tvm.relay.dataflow_pattern.DFPattern, Callable]]
+
+
+class UMAPartitioner:
+ """Partitioner base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str, merge_compiler_regions: bool = True) -> None:
+ self.target_name = target_name
+ self.merge_compiler_regions = merge_compiler_regions
+
+ self._relay_passes: List[Tuple[PassPhase, tvm.transform.Pass]] = []
+ self._patterns: PatternTable = []
+
+ def add_pattern(
+ self,
+ name: str,
+ pattern: tvm.relay.dataflow_pattern.DFPattern,
+ predicate: Optional[Callable] = None,
+ ) -> None:
+ """Add pattern to UMA partitioner
+
+ Parameters
+ ----------
+ name : str
+ relay name of pattern
+
+ pattern: tvm.relay.dataflow_pattern.DFPattern
+ pattern description as DFPattern
+
+ predicate: Optional[Callable]
+ Optional predicate
+
+ """
+
+ name = self.target_name + "." + name
+ if predicate:
+ self._patterns.append((name, pattern, predicate))
+ else:
+ self._patterns.append((name, pattern))
+
+ def _pattern_table(self) -> PatternTable:
+ return self._patterns
+
+ def register(self) -> None:
+ """Register all relevant relay-to-relay functions."""
+ register_pattern_table(self.target_name, self._pattern_table)
+
+ def partition(
+ self, mod: tvm.IRModule, params: Optional[Dict[str, tvm.runtime.NDArray]] = None
+ ) -> tvm.IRModule:
+ """Partition the relay graph in by the NPU supported and unsupported parts.
+
+ Parameters
+ ----------
+ mod : tvm.IRModule
+ The relay module to be partitioned.
+
+ params: Optional[Dict[str, tvm.runtime.NDArray]]
+
+ Returns
+ -------
+ out : tvm.IRModule
+ The partitioned relay module.
+
+ """
+ if params:
+ mod["main"] = bind_params_by_name(mod["main"], params)
+
+ mod = relay.transform.InferType()(mod)
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939140465
##########
tests/python/contrib/test_uma/test_uma_pipeline.py:
##########
@@ -0,0 +1,136 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+from tvm.relay import transform, testing
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
+
+import tvm
+from test_uma_vanilla_accelerator import VanillaAcceleratorBackend
+from tvm import relay
+import numpy as np
+from collections import OrderedDict
+
+from tvm.relay.backend.contrib.uma.api.utils import uma_available
+
+pytestmark = pytest.mark.skipif(not uma_available(), reason="UMA not available")
+
+
+@pytest.mark.parametrize(
+ "interface_api,use_unpacked_api,test_runner,groups,weight_shape",
+ [("c", True, AOT_DEFAULT_RUNNER, 1, 32)],
+)
+def test_conv2d(interface_api, use_unpacked_api, test_runner, groups, weight_shape):
+ """Test a subgraph with a single conv2d operator."""
+ mod, inputs, output_list, test_runner = create_conv2d(groups, test_runner, weight_shape)
+
+ uma_backend = VanillaAcceleratorBackend()
+ uma_backend.register()
+ mod = uma_backend.partition(mod)
+ target = tvm.target.Target("vanilla_accelerator", host=tvm.target.Target("c"))
+
+ compile_and_run(
+ AOTTestModel(module=mod, inputs=inputs, outputs=output_list),
+ test_runner,
+ interface_api,
+ use_unpacked_api,
+ target=target,
+ )
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
+ dtype = "float32"
+ ishape = (1, 32, 14, 14)
+ wshape = (32, weight_shape, 3, 3)
+ pass_config = {"tir.usmp.enable": True}
+ test_runner = AOTTestRunner(
+ makefile=test_runner.makefile,
+ prologue=test_runner.prologue,
+ epilogue=test_runner.epilogue,
+ includes=test_runner.includes,
+ parameters=test_runner.parameters,
+ pass_config=pass_config,
+ )
+ data0 = relay.var("data", shape=ishape, dtype=dtype)
+ weight0 = relay.var("weight", shape=wshape, dtype=dtype)
+ out = relay.nn.conv2d(data0, weight0, kernel_size=(3, 3), padding=(1, 1), groups=groups)
+ main_f = relay.Function([data0, weight0], out)
+ mod = tvm.IRModule()
+ mod["main"] = main_f
+ mod = transform.InferType()(mod)
+ i_data = np.random.uniform(0, 1, ishape).astype(dtype)
+ w1_data = np.random.uniform(0, 1, wshape).astype(dtype)
+ inputs = OrderedDict([("data", i_data), ("weight", w1_data)])
+ output_list = generate_ref_data(mod, inputs)
+ return mod, inputs, output_list, test_runner
+
+
+def _generate_runtime_data(input_shapes: dict, output_shapes: dict) -> [OrderedDict, OrderedDict]:
+ assert len(input_shapes) == 1
+ assert len(output_shapes) == 1
+
+ iname = list(input_shapes.keys())[0]
+ oname = list(output_shapes.keys())[0]
+ ishape = input_shapes[iname]
+ oshape = output_shapes[oname]
+ i_data = np.random.uniform(0, 1, ishape).astype("float32")
+ o_data = np.random.uniform(0, 1, oshape).astype("float32")
+ oname = "output" # name set by relay.build in executor_codegen_metadata.outputs
+ inputs = OrderedDict([(iname, i_data)])
+ outputs = OrderedDict([(oname, o_data)])
+ return inputs, outputs
+
+
+def test_mobilenet():
+ """Full network test with Mobilenet"""
+ use_unpacked_api = True
+ interface_api = "c"
+ test_runner = AOT_DEFAULT_RUNNER
+
+ mod, params = testing.mobilenet.get_workload(batch_size=1)
+
+ uma_backend = VanillaAcceleratorBackend()
+ uma_backend.register()
+ target = tvm.target.Target("vanilla_accelerator", host=tvm.target.Target("c"))
+ target_c = tvm.target.Target("c")
+
+ data_shape = [int(x) for x in mod["main"].checked_type.arg_types[0].shape]
+ data = np.random.uniform(size=data_shape).astype("float32")
+ input_list = {"data": data}
+ output_list = generate_ref_data(mod, input_list, params)
+ mod = uma_backend.partition(mod)
Review Comment:
Added comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939147157
##########
src/relay/backend/contrib/uma/targets.cc:
##########
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/targets.cc
+ *
+ * \brief this file contains the targets for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+
+namespace tvm {
+
+namespace relay {
+namespace contrib {
+namespace uma {
+tvm::transform::Pass RelayToTIR(String target_name);
+runtime::Module TIRToRuntime(IRModule mod, Target target);
+} // namespace uma
+} // namespace contrib
+} // namespace relay
+
+TVM_REGISTER_GLOBAL("relay.backend.contrib.uma.RegisterTarget")
+ .set_body_typed([](String target_name, Map<String, ObjectRef> attr_options) -> bool {
+ // @todo(cgerum): We probably should get rid of target.register rather sooner than later
+ // And use a proper registry for uma backends
+ for (const String registered_target_name : ::tvm::TargetKindRegEntry::ListTargetKinds()) {
+ if (registered_target_name == target_name) {
+ return false;
+ }
+ }
+
+ auto target_kind =
+ ::tvm::TargetKindRegEntry::RegisterOrGet(target_name)
+ .set_name()
+ .set_device_type(kDLCPU)
+ .add_attr_option<Array<String>>("keys")
+ .add_attr_option<String>("tag")
+ .add_attr_option<String>("device")
+ .add_attr_option<String>("model")
+ .add_attr_option<Array<String>>("libs")
+ .add_attr_option<Target>("host")
+ .add_attr_option<Integer>("from_device")
+ .set_attr<FTVMRelayToTIR>(tvm::attr::kRelayToTIR,
+ relay::contrib::uma::RelayToTIR(target_name))
+ .set_attr<FTVMTIRToRuntime>("TIRToRuntime", relay::contrib::uma::TIRToRuntime);
+
+ for (auto& attr_option : attr_options) {
+ auto option_name = attr_option.first;
+ auto default_value = attr_option.second;
+ if (default_value->IsInstance<StringObj>()) {
+ target_kind.add_attr_option<String>(option_name, Downcast<String>(default_value));
+ } else if (default_value->IsInstance<IntImmNode>()) {
+ target_kind.add_attr_option<Integer>(option_name, Downcast<Integer>(default_value));
+ } else {
+ LOG(FATAL) << "Attribute option of type " << attr_option.second->GetTypeKey()
Review Comment:
added tests in test_target to check these cases
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939140952
##########
include/tvm/relay/transform.h:
##########
@@ -509,6 +509,7 @@ TVM_DLL Pass SimplifyExpr();
*
* \param config All available targets.
*
+ *
Review Comment:
nope. removed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
areusch commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1209618862
thanks @MichaelJKlaiber for all the hard work here!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch merged pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
areusch merged PR #12087:
URL: https://github.com/apache/tvm/pull/12087
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925689353
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review Comment:
Given the out-of-tree usecase of UMA, I personally feel this might be better suited as an apps/ example, where we showcase to a "user" how UMA could be used. WDYT ?
Even if its a template, then it would be still good demonstrate how to use that template in a "user" flow.
*user here means a out-of-tree dev that wants to attach accelerator codegen to TVM
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
Review Comment:
We probably need to discuss about this - (I think we missed this in the RFC)
We need this documented in the guide-level explanation and the rationale to create a bespoke CLI should be discussed, IMO.
cc : @leandron
Also, I think it would be beneficial break this component to another PR for smoother upstreaming of things.
##########
python/tvm/relay/backend/contrib/uma/_template/run.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+
+from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner
+
+import tvm
+from tvm import relay
+from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend
+from tvm.relay import transform
+from collections import OrderedDict
+
+import numpy as np
+import tarfile
+from pathlib import Path
+import onnx
+
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
Review Comment:
This feels like a test source. Should we move this to tests ?
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
It would be better to use naming to the lines of "example_uma_*" to make it explicit that this is an example target
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
cc : @lhutton1 , If its exactly the same and if you feel it useful in a general sense for accelerator flow. I would like to second @areusch, we could move this pass to be generic transform pass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r928574359
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(
+ self,
+ includes: Callable[[], str] = None,
Review Comment:
Doc-string added.
@cgerum : can you answer the other question?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930025468
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+ partitioner = UMAPartitioner("test_partition")
+ assert get_pattern_table("test_partition") is None
+
+ partitioner.register()
+
+ assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+ "workload,backend,merge,expected_partitions",
+ [
+ ("resnet", "dnnl", False, 17),
+ ("resnet", "dnnl", True, 17),
Review Comment:
@areusch can you elaborate what you mean? We don't fully understand what you mean here
##########
python/tvm/relay/backend/contrib/uma/tutorial.md:
##########
@@ -0,0 +1,195 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+Making your hardware accelerator TVM-ready with UMA
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930008614
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
+ _tvm_block_match_name = "conv2d_nchw"
+
+ def _has_block(name: str, func) -> bool:
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930044501
##########
python/tvm/relay/backend/contrib/uma/_template/run.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+
+from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner
+
+import tvm
+from tvm import relay
+from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend
+from tvm.relay import transform
+from collections import OrderedDict
+
+import numpy as np
+import tarfile
+from pathlib import Path
+import onnx
+
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
Review Comment:
@manupa-arm , this is not supposed to be test code. This is example code for the tutorial. We will replace AOTTestRunner by microTVM Project API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930056748
##########
tests/scripts/task_python_uma.sh:
##########
@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euxo pipefail
+
+source tests/scripts/setup-pytest-env.sh
+
+run_pytest ctypes test_uma tests/python/contrib/test_uma
+run_pytest cython3 test_uma tests/python/contrib/test_uma
Review Comment:
we removed this file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925574906
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_replace_call_extern_" + target_str_);
+ if (replace_call_extern_pf == nullptr) {
+ CodeGenCHost::VisitExpr_(op, os);
+ } else {
+ // - funtion type (void) still gets printed before CallNode if extern call is wrapped in
+ // EvaluateNode
+ // - VarNode arguments might have "wrong" name_hints. The correct variable name is determined
+ // in C++ through GetVarID
+ String api_string = (*replace_call_extern_pf)(op->args);
+ os << api_string;
+ }
+ return;
+ }
+};
+
+runtime::Module TIRToRuntime(IRModule mod, Target target) {
+ bool output_ssa = false;
+ bool emit_asserts = false;
+ UMACodegen codegen(target->kind->name);
+ Array<String> function_names;
+ codegen.Init(output_ssa, emit_asserts);
+ for (auto kv : mod->functions) {
+ auto prim_func = Downcast<PrimFunc>(kv.second);
+ auto global_symbol = prim_func->GetAttr<String>(tvm::attr::kGlobalSymbol);
+ function_names.push_back(global_symbol.value());
+ codegen.AddFunction(prim_func);
+ }
+ std::string code = codegen.Finish();
+ return codegen::CSourceModuleCreate(code, "c", function_names);
Review Comment:
I see that UMA can provide this sort of infrastructure e.g. by doing this on TIR level.
An accelerator-specific TIR pass would split a function into DMA calls for transporting data and accelerator calls for the actual computation. In the Strawberry example we plan to demonstrate this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936300831
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936300311
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
+ public:
+ explicit OutlineCompilerFunctionsMutator(const IRModule& mod, const std::string& compiler_name)
+ : mod_(mod), compiler_name_(compiler_name) {}
+
+ Expr VisitExpr_(const LetNode* op) final {
+ auto pre_visit = [this](const LetNode* op) {
+ Expr var = this->VisitExpr(op->var);
+ Expr value = this->VisitExpr(op->value);
+
+ // Outlineable function no longer needs let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[var] = value;
+ }
+ };
+ auto post_visit = [this](const LetNode* op) {
+ // Rely on the Memoizer to cache pre-visit values
+ Expr value = this->VisitExpr(op->value);
+ Expr body = this->VisitExpr(op->body);
+ auto expr = GetRef<Expr>(op);
+
+ // Drop the let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[expr] = this->VisitExpr(op->body);
+ } else {
+ Var var = Downcast<Var>(this->VisitExpr(op->var));
+ if (var.same_as(op->var) && value.same_as(op->value) && body.same_as(op->body)) {
+ this->memo_[expr] = expr;
+ } else {
+ this->memo_[expr] = Let(var, value, body);
+ }
+ }
+ };
+ ExpandANormalForm(op, pre_visit, post_visit);
+ return memo_[GetRef<Expr>(op)];
+ }
+
+ Expr Rewrite_(const CallNode* pre, const Expr& post) override {
+ Call call = Downcast<Call>(post);
+ if (CanOutlineExpr(call->op)) {
+ Function func = Downcast<Function>(call->op);
+ auto gv_name = func->GetAttr<String>("global_symbol").value_or("");
+ ICHECK_NE(gv_name, "")
+ << "Function to be outlined must have global_symbol attribute, but didn't.";
+ GlobalVar gv(gv_name);
+ if (func->checked_type_.defined()) {
+ gv->checked_type_ = func->checked_type();
+ }
+ mod_->Update(gv, func);
+ return Call(gv, call->args, call->attrs, call->type_args);
+ }
+ return post;
+ }
+
+ private:
+ /*!
+ * \brief Check if the expr is a function and has the same
+ * compiler name as compiler_name_.
+ *
+ * \param expr The input expr.
+ * \return True if is outlineable else False.
+ */
+ bool CanOutlineExpr(const Expr& expr) {
+ if (!expr->IsInstance<FunctionNode>()) {
+ return false;
+ }
+ Function func = Downcast<Function>(expr);
+ auto compiler = func->GetAttr<String>(attr::kCompiler);
+ if (!compiler.defined()) {
+ return false;
+ }
+ if (compiler != compiler_name_) {
+ return false;
+ }
+ return true;
+ }
+
+ /*! \brief The module that the pass will run on. */
+ IRModule mod_;
+ /*! \brief The name of the compiler to enable outlining on external functions for. */
+ std::string compiler_name_;
+};
+
+/*!
+ * \brief A pass to outline compiler specific functions.
+ */
+tvm::transform::Pass OutlineCompilerFunctions(const std::string& compiler_name) {
+ runtime::TypedPackedFunc<IRModule(IRModule, transform::PassContext)> pass_func =
+ [=](IRModule mod, transform::PassContext ctx) {
+ GlobalVar gv = mod->GetGlobalVar("main");
+ Function main_func = Downcast<Function>(mod->Lookup("main"));
+ auto new_main_body =
+ OutlineCompilerFunctionsMutator(mod, compiler_name).VisitExpr(main_func->body);
+ if (!new_main_body.same_as(main_func->body)) {
+ Function new_main_func = WithFields(main_func, main_func->params, new_main_body);
+ mod->Update(gv, new_main_func);
+ }
+ return mod;
+ };
+ return tvm::transform::CreateModulePass(pass_func, 0,
+ "relay.backend.contrib.uma.OutlineCompilerFunctions", {});
+}
+
+TVM_REGISTER_GLOBAL("relay.ext.uma.OutlineCompilerFunctions")
+ .set_body_typed(OutlineCompilerFunctions);
+
+/*!
+ * \brief This pass will lower NPU functions in a Relay module to scheduled TIR prim functions.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939139928
##########
apps/uma/_template/passes.py:
##########
@@ -0,0 +1,142 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ _EXTERNAL_FUNCTION_NAME = "my_ai_hw_conv2dnchw"
+ _TVM_BLOCK_MATCH_NAME = "conv2d_nchw"
+
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @classmethod
+ def _my_ai_hw_conv2d_pass(cls, func, mod, ctx):
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+
+ def _has_block(name: str, func: tvm.tir.PrimFunc) -> bool:
+ """
+ Determine of a tir.block with `name` exists in `func`
+ """
+
+ def _hb(op):
+ if isinstance(op, tvm.tir.Block):
+ _found_blocks.append(op.name_hint)
+
+ _found_blocks = []
+ tvm.tir.stmt_functor.post_order_visit(func.body, _hb)
+ return name in _found_blocks
+
+ def _detect_and_replace_conv2d(
+ func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ def _replace_conv2d(op):
+ if op == _entry_node:
+ irb = tvm.tir.ir_builder.create()
+ # Collection of buffer address
+ buffers = [b[1].data for b in _handles]
+ # extraction of loop offsets
+ for k, v in _loops.items():
+ assert v.min.value == 0
+ offset_order = ["co", "w", "h", "ci", "kh", "kw"]
+ offsets = [_loops[i].extent.value for i in offset_order]
+ args = buffers + offsets
+ external_call = tvm.tir.Evaluate(
+ tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args)
+ )
+ ext_calls = tvm.tir.SeqStmt([external_call])
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939142883
##########
apps/uma/_template/strategies.py:
##########
@@ -0,0 +1,17 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Strategies for the my_ai_hw accelerator"""
Review Comment:
Added a comment with an example. Later examples in the tutorial will have detailed code with explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939145045
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Optional
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(self, includes: Optional[Callable[[], str]] = None) -> None:
+ """Registration of UMA helper functions, e.g. includes and replace_call_extern.
+
+ Parameters
+ ----------
+ includes : OptionalCallable[[], str]]
+ user-defined function that adds C-#include statement to UMA C-Code.
+ """
+ if includes is not None:
Review Comment:
I agree with your finding for "includes".
It is optional, therefore I removed the ICHECK from the cc code.
I also agree that it is important, therefore I added a separate parameter instead of kwargs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1203599874
Thanks to all the reviewers for the thorough reviews and the great suggestions and comment that improved UMA!
@manupa-arm @areusch @sunggg @cbalint13 @kslavka @lhutton1
We integrated the vast majority of the requests in this review into the PR.
There were many **great suggestion** that we hadn't considered before, but improved the overall quality of the UMA contribution, e.g.
- Tutorial as RST for gallery
- Adding a case test case where the TVM default pipeline and the UMA pipeline are both used
- ...
This PR has been open for a quite long time now, it has improved steadily and we feel that it is ready now. **Please let us know you thoughts.**
There are a couple of (small) things we left out for the is PR, and would ask for you opinion to move it to (very near future) PRs:
- move OutlineCompilerFunctionsMutator out of UMA @areusch @manupa-arm
- replace AOTTestRunner by project API @manupa-arm
- annotations of UMALower._operator_strategies @lhutton1
(these steps will be moved to the tracking issue, for the case that the PR is granted)
Further steps see current tracking issue https://github.com/apache/tvm/issues/11260
CC: @SebastianBoblest @UlrikHjort
It is also great that WE as a community identified items where further discussion is required, e.g.:
- UMA CLI / tvmc integration
- Demo use-cases, e.g. running BERT on Ethos-U with UMA
- ...
I'd move them to an updated version of the UMA RFC or the discussion forum
Thanks to everyone contributing to UMA, esp @cgerum @PaulPalomeroBernardo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r926637840
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r923152921
##########
CMakeLists.txt:
##########
@@ -494,6 +494,7 @@ include(cmake/modules/contrib/ArmComputeLib.cmake)
include(cmake/modules/contrib/TensorRT.cmake)
include(cmake/modules/contrib/VitisAI.cmake)
include(cmake/modules/contrib/Verilator.cmake)
+include(cmake/modules/contrib/UMA.cmake)
Review Comment:
CC @manupa-arm @areusch @sunggg @Mousius @lhutton1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930040959
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
Review Comment:
We discussed this again and reviewed the options.
The conclusion is to remove this and also remove codegen_c_replace_call_extern_{target_str} from Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930047570
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review Comment:
We will move it to apps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929982909
##########
python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,76 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+ int
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930000397
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
Review Comment:
`self` is not used here. the Pycharm lint therefore detects a smell and proposes to make it a staticmethod.
We could go both ways here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
areusch commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r924081353
##########
tests/python/contrib/test_uma/test_uma_lowering_with_umalower.py:
##########
@@ -0,0 +1,115 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+import pytest
+import pathlib
+
+import tvm
+from tests.python.contrib.test_uma.test_uma_utils import _create_schedule, _generate_io_arrays
+from tvm import topi
+from tvm.relay.backend.contrib.uma._template.passes import MyAiHwConv2dPass
+import tvm.testing
+from tvm import te
+from tvm.relay.backend.contrib.uma.api.lower import UMALower
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+
+
+def _conv2d_te_definition(shapes: dict) -> list:
+ n, w, h, ci, kw, kh, co = (
+ shapes["n"],
+ shapes["w"],
+ shapes["h"],
+ shapes["ci"],
+ shapes["kw"],
+ shapes["kh"],
+ shapes["co"],
+ )
+ ifmap = te.placeholder((n, ci, w, h), dtype="float32", name="ifmap")
+ weights = te.placeholder((co, ci, kw, kh), dtype="float32", name="weights")
+ result = topi.nn.conv2d_nchw(ifmap, weights, stride=1, padding=[kw // 2, kh // 2], dilation=1)
+ return [ifmap, weights, result]
+
+
+def _pepare_conv2d_schedule(shapes, use_external_conv2d_impl=True):
+ placeholders = _conv2d_te_definition(shapes)
+
+ uma_path = pathlib.Path(str(tvm.relay.backend.contrib.uma.__file__)).parent.absolute()
+ conv2d_file = uma_path / "_template" / "conv2dnchw.cc"
+
+ with conv2d_file.open() as f:
+ sch_tir = _create_schedule(
+ placeholders, f, use_external_conv2d_impl=use_external_conv2d_impl
+ )
+ return placeholders, sch_tir
+
+
+def _run_external_conv2d(dut_io_arrays, conv2d_shapes, target):
+ # Run conv2d with external function
+ placeholders, schedule = _pepare_conv2d_schedule(conv2d_shapes)
+
+ uma_lower = UMALower("lower_test")
+ uma_lower._tir_passes.append((PassPhase.TIR_PHASE_0, MyAiHwConv2dPass()))
+ with tvm.transform.PassContext():
+ tir_mod = uma_lower._lower_stir_to_nstir(schedule.mod["main"])
+
+ ifmap_data, weight_data, result_data = dut_io_arrays
+
+ llvm_conv2d_mod = tvm.build(tir_mod, placeholders, target=target, name="test_external_conv2d")
+ llvm_conv2d_mod(ifmap_data, weight_data, result_data)
+
+
+def _run_reference_conv2d(reference_io_arrays, conv2d_shapes, target):
+ placeholders, schedule = _pepare_conv2d_schedule(conv2d_shapes)
+ ref_mod = tvm.build(schedule.mod, placeholders, target=target, name="test_reference_conv2d")
+ ifmap, weights, result = reference_io_arrays
+ ref_mod(ifmap, weights, result)
+
+
+def _prepare_io_arrays(conv2d_shapes, dev):
+ dut_io_arrays = _generate_io_arrays(conv2d_shapes, dev)
+ _, _, ref_result = _generate_io_arrays(conv2d_shapes, dev)
+ reference_io_arrays = [dut_io_arrays[0], dut_io_arrays[1], ref_result]
+ return dut_io_arrays, reference_io_arrays
+
+
+@pytest.mark.parametrize(
+ "n, w, h, ci, kw, kh, co",
+ [
+ (1, 224, 224, 3, 3, 3, 4),
+ (1, 224, 224, 3, 5, 5, 4),
+ (1, 224, 224, 3, 7, 7, 4),
+ (1, 224, 320, 3, 7, 7, 4),
+ (1, 224, 224, 3, 7, 7, 4),
+ ],
+)
+def test_lower_with_uma(n, w, h, ci, kw, kh, co):
+ target = tvm.target.Target(target="llvm", host="llvm")
+ dev = tvm.device(target.kind.name, 0)
+ conv2d_shapes = dict(n=n, w=w, h=h, ci=ci, kw=kw, kh=kh, co=co)
+
+ dut_io_arrays, reference_io_arrays = _prepare_io_arrays(conv2d_shapes, dev)
+
+ _run_external_conv2d(dut_io_arrays, conv2d_shapes, target)
+ _run_reference_conv2d(reference_io_arrays, conv2d_shapes, target)
+
+ # compare results
+ dut_results = dut_io_arrays[2].numpy()
+ ref_results = reference_io_arrays[2].numpy()
+ tvm.testing.assert_allclose(dut_results, ref_results, rtol=1e-5)
+
+
+if __name__ == "__main__":
+ test_lower_with_uma(1, 224, 224, 3, 3, 3, 4)
Review Comment:
tvm.testing.main()
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
Review Comment:
this might be a bit of a personal preference, but i often find long comment separators can make the code harder to parse as a whole. my personal vote is to remove these, though i don't know if there is a specific line in the style guide about this
##########
include/tvm/relay/transform.h:
##########
@@ -509,6 +509,8 @@ TVM_DLL Pass SimplifyExpr();
*
* \param config All available targets.
*
+ * \param config All available targets.
Review Comment:
nit: remove
##########
cmake/config.cmake:
##########
@@ -296,9 +296,6 @@ set(USE_VTA_FPGA OFF)
# Whether use Thrust
set(USE_THRUST OFF)
-# Whether use cuRAND
Review Comment:
why this change?
##########
python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,76 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+ int
Review Comment:
if possible could you write a docstring for this? could be helpful for folks to understand the parameters at this level, since C++ is often a familiar starting point
##########
python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,76 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+ int
+ my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic,
+ int kh, int kw) {
+
+ int kw_low = kw / 2;
+ int kh_low = kh / 2;
+ int kw_high = iw + kw / 2;
+ int kh_high = ih + kh / 2;
+
+ int padded_iw = iw + 2 * kw_low;
+ int padded_ih = ih + 2 * kh_low;
+
+ float* pad_temp = (float*)malloc(
+ (((ic * padded_iw * padded_ih) + (padded_ih * padded_iw)) + padded_iw) * sizeof(float));
+
+ if (pad_temp == NULL) {
+ return -1;
+ }
+
+ for (int i1 = 0; i1 < ic; ++i1) {
+ for (int i2 = 0; i2 < padded_ih; ++i2) {
+ for (int i3 = 0; i3 < padded_iw; ++i3) {
+ ((float*)pad_temp)[(((i1 * padded_iw * padded_ih) + (i2 * padded_iw)) + i3)] =
+ (((((kh_low <= i2) && (i2 < kh_high)) && (kw_low <= i3)) && (i3 < kw_high))
+ ? ifmap[((((i1 * iw * ih) + ((i2 - kh_low) * iw)) + i3 - kw_low))]
+ : 0.000000e+00f);
+ }
+ }
+ }
+ for (int i11 = 0; i11 < oc; ++i11) {
+ for (int i21 = 0; i21 < ih; ++i21) {
+ for (int i31 = 0; i31 < iw; ++i31) {
+ for (int i4 = 0; i4 < ic; ++i4) {
+ for (int i5 = 0; i5 < kh; ++i5) {
+ for (int i6 = 0; i6 < kw; ++i6) {
+ int cse_var_1 = (((i11 * iw * ih) + (i21 * iw)) + i31);
+ if (((i4 == 0) && (i5 == 0)) && (i6 == 0)) {
+ result[cse_var_1] = 0.000000e+00f;
+ }
+ result[cse_var_1] =
+ (result[cse_var_1] +
+ (((float*)
+ pad_temp)[i4 * padded_iw * padded_ih + (i21 + i5) * padded_iw + i31 + i6] *
+ weights[((((i11 * ic * kh * kw) + (i4 * kh * kw)) + (i5 * kw)) + i6)]));
+ }
+ }
+ }
+ }
+ }
+ }
+ free(pad_temp);
+ return 0;
+}
Review Comment:
nit: newline at end of file
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
Review Comment:
what's the benefit of declaring this as staticmethod?
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
+ # Target configuration
+ #######################################################################
+ self._register_target_attr("dimension")
+
+ #######################################################################
+ # Relay to Relay function registration
+ #######################################################################
+ self._register_pattern("conv2d", conv2d_pattern())
+
+ #######################################################################
+ # Relay to TIR function registration
+ #######################################################################
+ self._register_tir_pass(PassPhase.TIR_PHASE_0, MyAiHwConv2dPass())
Review Comment:
do we need to make the phase numbers part of an enum, or should it be more like
```
self._register_tir_pass(PassPhase.TIR_PHASE, 0, MyAiHwConv2dPass())
```
##########
python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,76 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+ int
+ my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic,
+ int kh, int kw) {
+
+ int kw_low = kw / 2;
+ int kh_low = kh / 2;
+ int kw_high = iw + kw / 2;
+ int kh_high = ih + kh / 2;
+
+ int padded_iw = iw + 2 * kw_low;
+ int padded_ih = ih + 2 * kh_low;
+
+ float* pad_temp = (float*)malloc(
Review Comment:
should this be `TVMBackendAllocWorkspace`? or do we intend to show use of a standard memory allocator here?
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
Review Comment:
suggest docstring here since this is an example
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
+ _tvm_block_match_name = "conv2d_nchw"
+
+ def _has_block(name: str, func) -> bool:
+ """
+ Determine of a tir.block with `name` exists in `func`
+ """
+
+ def _hb(op):
+ if isinstance(op, tvm.tir.Block):
+ _found_blocks.append(op.name_hint)
+
+ _found_blocks = []
+ tvm.tir.stmt_functor.post_order_visit(func.body, _hb)
+ return name in _found_blocks
+
+ def _transform_function(
+ func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ def _replace_conv2d(op):
+ if op == _entry_node:
+ irb = tvm.tir.ir_builder.create()
+ # Collection of buffer address
+ buffers = [b[1].data for b in _handles]
+ # extraction of loop offsets
+ for k, v in _loops.items():
+ assert v.min.value == 0
+ offset_order = ["co", "w", "h", "ci", "kh", "kw"]
+ offsets = [_loops[i].extent.value for i in offset_order]
+ args = buffers + offsets
+ external_call = tvm.tir.Evaluate(
+ tir_call(irb, True, _external_function_name, *args)
+ )
+ mac_calls = tvm.tir.SeqStmt([external_call])
+ irb.emit(mac_calls)
+ irb_result = irb.get()
+ return irb_result
+ return op
+
+ sch = tir.Schedule(func)
+
+ if _has_block(_tvm_block_match_name, func):
+ conv2d_block = sch.get_block(_tvm_block_match_name)
+ rv_loops = sch.get_loops(conv2d_block)
+ assert len(rv_loops) == 7
+ loops = dict(
+ n=rv_loops[0],
+ co=rv_loops[1],
+ h=rv_loops[2],
+ w=rv_loops[3],
+ ci=rv_loops[4],
+ kh=rv_loops[5],
+ kw=rv_loops[6],
+ )
+ _entry_node = sch.get(rv_loops[1])
+ _loops = {k: sch.get(v) for k, v in loops.items()}
+ _handles = func.buffer_map.items()
+
+ x = tvm.tir.stmt_functor.ir_transform(func.body, None, _replace_conv2d, ["tir.For"])
+ return func.with_body(x)
+ else:
+ return func
+
+ r = _transform_function(func, mod, ctx)
+ return r
+
+
+def tir_call(ib: tvm.tir.ir_builder, extern: bool, name: str, *args):
+ """
+ ib: ir_builder
+ extern: bool
+ True --> tvm.tir.call_extern
+ False --> tvm.tir.call_packed
+ name: str
+ function name
+ *args:
+ arguments for function call
+ """
+
+ def buf_from_array(ib, arr, dtype):
+ # Allocate enough memory to store the whole array
+ var = ib.allocate("int32", (len(arr),), scope="global")
+ for i, v in enumerate(arr):
+ var[i] = v
+ # Declare a buffer, which is basically a view on the chunk of memory that we allocated previously
+ buf = tvm.tir.decl_buffer((len(arr),), dtype, data=var, scope="global")
Review Comment:
should we use arr.shape?
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
Review Comment:
this one and the one below could be a class-level constant, e.g. `_EXTERNAL_FUNCTION_NAME = ...`
##########
python/tvm/relay/backend/contrib/uma/_template/run.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+
+from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner
+
+import tvm
+from tvm import relay
+from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend
+from tvm.relay import transform
+from collections import OrderedDict
+
+import numpy as np
+import tarfile
+from pathlib import Path
+import onnx
+
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
Review Comment:
is it possible to use this with Project API rather than AOT_DEFAULT_RUNNER? e.g. following `test_crt.py`.
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
+ _tvm_block_match_name = "conv2d_nchw"
+
+ def _has_block(name: str, func) -> bool:
Review Comment:
can `func` be typed?
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
+
+ Tool to create code skeletons for an easy integration of
+ new AI hardware accelerators/libraries into TVM using UMA
+"""
+
+import argparse
+import os
+import shutil
+import sys
+from inflection import camelize, underscore
+
+
+def _parse_args():
+ parser = argparse.ArgumentParser(description="UMA Interface command line interface")
+ parser.add_argument(
+ "--add_hardware",
+ type=str,
+ required=True,
+ )
+ parser.add_argument(
+ "--tutorial",
+ type=str,
+ )
+ args = parser.parse_args()
+ return args
+
+
+def replace_template_name(
+ files: list, template_name: str, add_hw_name: str, template_source: str = "_template"
+) -> None:
+ """
+ Replace names in template skeleton code by new name
+ """
+ for f in files:
+ with open(f) as read_file:
+ data = read_file.read()
+ for case in [underscore, camelize]:
+ data = data.replace(case(template_name), case(add_hw_name))
+ data = data.replace(template_source, underscore(add_hw_name))
+ with open(f, "w") as write_file:
+ write_file.write(data)
+
+
+def main():
Review Comment:
just also wondering if this can be integrated with tvmc instead?
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(
+ self,
+ includes: Callable[[], str] = None,
+ replace_call_extern: Callable[[tvm.ir.container.Array], str] = None,
+ ) -> None:
+ if includes is not None:
+ tvm._ffi.register_func(
+ "relay.ext.uma.codegen_c_includes_{}".format(self.target_name), includes
Review Comment:
suggest f-string e.g. `f"relay.ext.uma.codegen_c_includes{self.target_name}"`
##########
src/relay/backend/contrib/uma/targets.cc:
##########
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/targets.cc
+ *
+ * \brief this file contains the targets for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+
+namespace tvm {
+
+namespace relay {
+namespace contrib {
+namespace uma {
+tvm::transform::Pass RelayToTIR(String target_name);
+runtime::Module TIRToRuntime(IRModule mod, Target target);
+} // namespace uma
+} // namespace contrib
+} // namespace relay
+
+TVM_REGISTER_GLOBAL("relay.backend.contrib.uma.RegisterTarget")
+ .set_body_typed([](String target_name, Map<String, ObjectRef> attr_options) {
+ auto target_kind =
+ ::tvm::TargetKindRegEntry::RegisterOrGet(target_name)
+ .set_name()
+ .set_device_type(kDLCPU)
+ .add_attr_option<Array<String>>("keys")
+ .add_attr_option<String>("tag")
+ .add_attr_option<String>("device")
+ .add_attr_option<String>("model")
+ .add_attr_option<Array<String>>("libs")
+ .add_attr_option<Target>("host")
+ .add_attr_option<Integer>("from_device")
+ .set_attr<FTVMRelayToTIR>(tvm::attr::kRelayToTIR,
+ relay::contrib::uma::RelayToTIR(target_name))
+ .set_attr<FTVMTIRToRuntime>("TIRToRuntime", relay::contrib::uma::TIRToRuntime);
+
+ for (auto& attr_option : attr_options) {
+ try {
+ target_kind.add_attr_option<String>(attr_option.first,
+ Downcast<String>(attr_option.second));
+ continue;
+ } catch (...) {
Review Comment:
rather than catch, can you just attempt to downcast e.g.
```
if (String s = attr_option.as<String>() != nullptr) {
target_kind.add_attr_option<String>(attr_option.first, s);
} ...
```
etc
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+ partitioner = UMAPartitioner("test_partition")
+ assert get_pattern_table("test_partition") is None
+
+ partitioner.register()
+
+ assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+ "workload,backend,merge,expected_partitions",
+ [
+ ("resnet", "dnnl", False, 17),
+ ("resnet", "dnnl", True, 17),
Review Comment:
possible to express expected_partitions as something slightly easier to port, in case these models change? e.g. expected_partition_layers and then compute the # of expected partitions by looking for layers named that in the model and asserting there is at least one?
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
i think this already exists in ethosu, can we factor it out instead of duplicating?
##########
python/tvm/relay/backend/contrib/uma/tutorial.md:
##########
@@ -0,0 +1,195 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+Making your hardware accelerator TVM-ready with UMA
Review Comment:
could this be a rst tutorial in gallery/?
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
+ public:
+ explicit OutlineCompilerFunctionsMutator(const IRModule& mod, const std::string& compiler_name)
+ : mod_(mod), compiler_name_(compiler_name) {}
+
+ Expr VisitExpr_(const LetNode* op) final {
+ auto pre_visit = [this](const LetNode* op) {
+ Expr var = this->VisitExpr(op->var);
+ Expr value = this->VisitExpr(op->value);
+
+ // Outlineable function no longer needs let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[var] = value;
+ }
+ };
+ auto post_visit = [this](const LetNode* op) {
+ // Rely on the Memoizer to cache pre-visit values
+ Expr value = this->VisitExpr(op->value);
+ Expr body = this->VisitExpr(op->body);
+ auto expr = GetRef<Expr>(op);
+
+ // Drop the let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[expr] = this->VisitExpr(op->body);
+ } else {
+ Var var = Downcast<Var>(this->VisitExpr(op->var));
+ if (var.same_as(op->var) && value.same_as(op->value) && body.same_as(op->body)) {
+ this->memo_[expr] = expr;
+ } else {
+ this->memo_[expr] = Let(var, value, body);
+ }
+ }
+ };
+ ExpandANormalForm(op, pre_visit, post_visit);
+ return memo_[GetRef<Expr>(op)];
+ }
+
+ Expr Rewrite_(const CallNode* pre, const Expr& post) override {
+ Call call = Downcast<Call>(post);
+ if (CanOutlineExpr(call->op)) {
+ Function func = Downcast<Function>(call->op);
+ auto gv_name = func->GetAttr<String>("global_symbol").value_or("");
+ ICHECK_NE(gv_name, "")
+ << "Function to be outlined must have global_symbol attribute, but didn't.";
+ GlobalVar gv(gv_name);
+ if (func->checked_type_.defined()) {
+ gv->checked_type_ = func->checked_type();
+ }
+ mod_->Update(gv, func);
+ return Call(gv, call->args, call->attrs, call->type_args);
+ }
+ return post;
+ }
+
+ private:
+ /*!
+ * \brief Check if the expr is a function and has the same
+ * compiler name as compiler_name_.
+ *
+ * \param expr The input expr.
+ * \return True if is outlineable else False.
+ */
+ bool CanOutlineExpr(const Expr& expr) {
+ if (!expr->IsInstance<FunctionNode>()) {
+ return false;
+ }
+ Function func = Downcast<Function>(expr);
+ auto compiler = func->GetAttr<String>(attr::kCompiler);
+ if (!compiler.defined()) {
+ return false;
+ }
+ if (compiler != compiler_name_) {
+ return false;
+ }
+ return true;
+ }
+
+ /*! \brief The module that the pass will run on. */
+ IRModule mod_;
+ /*! \brief The name of the compiler to enable outlining on external functions for. */
+ std::string compiler_name_;
+};
+
+/*!
+ * \brief A pass to outline compiler specific functions.
+ */
+tvm::transform::Pass OutlineCompilerFunctions(const std::string& compiler_name) {
+ runtime::TypedPackedFunc<IRModule(IRModule, transform::PassContext)> pass_func =
+ [=](IRModule mod, transform::PassContext ctx) {
+ GlobalVar gv = mod->GetGlobalVar("main");
+ Function main_func = Downcast<Function>(mod->Lookup("main"));
+ auto new_main_body =
+ OutlineCompilerFunctionsMutator(mod, compiler_name).VisitExpr(main_func->body);
+ if (!new_main_body.same_as(main_func->body)) {
+ Function new_main_func = WithFields(main_func, main_func->params, new_main_body);
+ mod->Update(gv, new_main_func);
+ }
+ return mod;
+ };
+ return tvm::transform::CreateModulePass(pass_func, 0,
+ "relay.backend.contrib.uma.OutlineCompilerFunctions", {});
+}
+
+TVM_REGISTER_GLOBAL("relay.ext.uma.OutlineCompilerFunctions")
+ .set_body_typed(OutlineCompilerFunctions);
+
+/*!
+ * \brief This pass will lower NPU functions in a Relay module to scheduled TIR prim functions.
Review Comment:
nit: fix comment
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+ partitioner = UMAPartitioner("test_partition")
+ assert get_pattern_table("test_partition") is None
+
+ partitioner.register()
+
+ assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+ "workload,backend,merge,expected_partitions",
+ [
+ ("resnet", "dnnl", False, 17),
+ ("resnet", "dnnl", True, 17),
+ ("mlp", "dnnl", False, 1),
+ ("resnet", "cutlass", False, 2),
+ ("resnet", "cutlass", True, 2),
+ ("mlp", "cutlass", False, 4),
+ ("mlp", "cutlass", True, 2),
+ ],
+)
+def test_existing_pattern_tables(workload, backend, merge, expected_partitions):
+ partitioner = UMAPartitioner(backend + "_uma", merge)
+ pattern_table = get_pattern_table(backend)
+
+ for entry in pattern_table:
+ partitioner.add_pattern(*entry)
+
+ if workload == "resnet":
+ net = resnet.get_net(1, 10)
+ elif workload == "mlp":
+ net = mlp.get_net(1, 10)
Review Comment:
add
```
else:
assert False, f"don't know how to find workload for {workload}"
```
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
i think maybe we could fix this with a dir-level comment in an `__init__.py`. did you guys exclude this file to avoid folks importing from `_template`?
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
Review Comment:
this might be a bit hacky (e.g. how many PackedFuncs do we have to register per codegen?). possible to leverage ExprVisitor instead?
##########
python/tvm/relay/backend/contrib/uma/_template/run.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+
+from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner
+
+import tvm
+from tvm import relay
+from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend
+from tvm.relay import transform
+from collections import OrderedDict
+
+import numpy as np
+import tarfile
+from pathlib import Path
+import onnx
+
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
+ dtype = "float32"
+ ishape = (1, 32, 14, 14)
+ wshape = (32, weight_shape, 3, 3)
+ pass_config = {"tir.usmp.enable": True}
+ test_runner = AOTTestRunner(
+ makefile=test_runner.makefile,
+ prologue=test_runner.prologue,
+ epilogue=test_runner.epilogue,
+ includes=test_runner.includes,
+ parameters=test_runner.parameters,
+ pass_config=pass_config,
+ )
+ data0 = relay.var("data", shape=ishape, dtype=dtype)
+ weight0 = relay.var("weight", shape=wshape, dtype=dtype)
+ out = relay.nn.conv2d(data0, weight0, kernel_size=(3, 3), padding=(1, 1), groups=groups)
+ main_f = relay.Function([data0, weight0], out)
+ mod = tvm.IRModule()
+ mod["main"] = main_f
+ mod = transform.InferType()(mod)
+ i_data = np.random.uniform(0, 1, ishape).astype(dtype)
+ w1_data = np.random.uniform(0, 1, wshape).astype(dtype)
+ inputs = OrderedDict([("data", i_data), ("weight", w1_data)])
+ output_list = generate_ref_data(mod, inputs)
+ return mod, inputs, output_list, test_runner
+
+
+def main():
+ mod, inputs, output_list, test_runner = create_conv2d()
+
+ uma_backend = MyAiHwBackend()
+ uma_backend.register()
+ mod = uma_backend.partition(mod)
+ target = tvm.target.Target("my_ai_hw", host=tvm.target.Target("c"))
+
+ export_directory = tvm.contrib.utils.tempdir(keep_for_debug=True).path
+ print(f"Generated files are in {export_directory}")
+ compile_and_run(
+ AOTTestModel(module=mod, inputs=inputs, outputs=output_list),
+ test_runner,
+ interface_api="c",
+ use_unpacked_api=True,
+ target=target,
+ test_dir=str(export_directory),
+ )
+
+
+if __name__ == "__main__":
+ main()
Review Comment:
is it possible to use this from `tvmc`? just curious whether you need this script or not
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_replace_call_extern_" + target_str_);
+ if (replace_call_extern_pf == nullptr) {
+ CodeGenCHost::VisitExpr_(op, os);
+ } else {
+ // - funtion type (void) still gets printed before CallNode if extern call is wrapped in
+ // EvaluateNode
+ // - VarNode arguments might have "wrong" name_hints. The correct variable name is determined
+ // in C++ through GetVarID
+ String api_string = (*replace_call_extern_pf)(op->args);
+ os << api_string;
+ }
+ return;
+ }
+};
+
+runtime::Module TIRToRuntime(IRModule mod, Target target) {
+ bool output_ssa = false;
+ bool emit_asserts = false;
+ UMACodegen codegen(target->kind->name);
+ Array<String> function_names;
+ codegen.Init(output_ssa, emit_asserts);
+ for (auto kv : mod->functions) {
+ auto prim_func = Downcast<PrimFunc>(kv.second);
+ auto global_symbol = prim_func->GetAttr<String>(tvm::attr::kGlobalSymbol);
+ function_names.push_back(global_symbol.value());
+ codegen.AddFunction(prim_func);
+ }
+ std::string code = codegen.Finish();
+ return codegen::CSourceModuleCreate(code, "c", function_names);
Review Comment:
this is a great question. in C runtime, we're building out a facility to invoke accelerator drivers e.g. before/after the whole inference run and before/after each call to an accelerator-offloaded layer. in C++ runtime, it's assumed that any such setup is handled via an appropriate runtime::Module subclass either in GetFunction, constructor, or in a custom PackedFunc override returned from GetFunction. However, AOT doesn't currently support calling out to those functions yet so the c++ strategy is not quite ready to be discussed in this PR in detail.
could you provide a bit more detail about the kinds of drivers you want to invoke?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] sunggg commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
sunggg commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1189214887
@cbalint13, appreciate sharing your thoughts! I also think this is a great out-of-tree approach and look forward to its actual usage.
> Could you add more documentation in general (e.g., explain what parameters are)?
>> Current sample (_template) & doc are fine for a contrib module.
>> It greatly enables "out-of-tree" maintenance towards custom HW accelarator design.
>> Example parts Vanilla, Strawberry & Chocolate are nice levels of introductory & guidance.
I agree that the overall docstring structure (e.g., Vanilla, Strawberry & Chocolate) would be a great introduction/tutorial.
My issue was more like a code-level docstring since some of them are missing and I wasn't sure what each function or parameter exactly means.
https://github.com/apache/tvm/blob/bc211fdc91932ef9b77f7edd2ff65151ee5faaff/python/tvm/relay/backend/contrib/uma/_template/codegen.py
https://github.com/apache/tvm/blob/bc211fdc91932ef9b77f7edd2ff65151ee5faaff/python/tvm/relay/backend/contrib/uma/_template/codegen.py
> It would be great to have a testing for end-to-end model-level testing. (e.g., run BERT with UMA on ARM EthosU)
>> Sounds beyond of scope, same way we could "dream" a fully verilated BERT running verilator-contrib module (!) @tmoreau89 , @vegaluisjose
>
>> As for ARM-Ethos ⟨™⟩, my understanding is a quite different (not so "out-of-tree" principled) way to connect things inside, but @manupa-arm can comment more on this.
>
>> Would be possible in the future to decouple ARM-Ethos⟨™⟩ (or Vitis-AI) to be more "out-of-tree" by porting it over UMA, but this would be quite bit of another task.
I see. Thank you for the clarification! Maybe my example was not relevant. I just thought it would be nice to have some model-level test cases since we seem to have all necessary components (UMA Partitioner, UMA Pipeline, UMA Lower, UMA Codegen) in this PR. I just wanted to see if current PR also covers a case where we lower some parts with UMA while handling other unsupported parts via the conventional pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929962126
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(
+ self,
+ includes: Callable[[], str] = None,
Review Comment:
To generate dynamic strings more easily we would be in favor of passing a Callable
@PaulPalomeroBernardo @cgerum
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930017268
##########
python/tvm/relay/backend/contrib/uma/_template/run.py:
##########
@@ -0,0 +1,88 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+from tvm.micro.testing.aot_test_utils import AOT_DEFAULT_RUNNER
+
+from tvm.testing.aot import compile_and_run, AOTTestModel, AOTTestRunner
+
+import tvm
+from tvm import relay
+from tvm.relay.backend.contrib.uma._template.backend import MyAiHwBackend
+from tvm.relay import transform
+from collections import OrderedDict
+
+import numpy as np
+import tarfile
+from pathlib import Path
+import onnx
+
+from tvm.testing.aot import (
+ AOTTestModel,
+ AOTTestRunner,
+ generate_ref_data,
+ compile_and_run,
+)
+
+
+def create_conv2d(groups=1, test_runner=AOT_DEFAULT_RUNNER, weight_shape=32):
+ dtype = "float32"
+ ishape = (1, 32, 14, 14)
+ wshape = (32, weight_shape, 3, 3)
+ pass_config = {"tir.usmp.enable": True}
+ test_runner = AOTTestRunner(
+ makefile=test_runner.makefile,
+ prologue=test_runner.prologue,
+ epilogue=test_runner.epilogue,
+ includes=test_runner.includes,
+ parameters=test_runner.parameters,
+ pass_config=pass_config,
+ )
+ data0 = relay.var("data", shape=ishape, dtype=dtype)
+ weight0 = relay.var("weight", shape=wshape, dtype=dtype)
+ out = relay.nn.conv2d(data0, weight0, kernel_size=(3, 3), padding=(1, 1), groups=groups)
+ main_f = relay.Function([data0, weight0], out)
+ mod = tvm.IRModule()
+ mod["main"] = main_f
+ mod = transform.InferType()(mod)
+ i_data = np.random.uniform(0, 1, ishape).astype(dtype)
+ w1_data = np.random.uniform(0, 1, wshape).astype(dtype)
+ inputs = OrderedDict([("data", i_data), ("weight", w1_data)])
+ output_list = generate_ref_data(mod, inputs)
+ return mod, inputs, output_list, test_runner
+
+
+def main():
+ mod, inputs, output_list, test_runner = create_conv2d()
+
+ uma_backend = MyAiHwBackend()
+ uma_backend.register()
+ mod = uma_backend.partition(mod)
+ target = tvm.target.Target("my_ai_hw", host=tvm.target.Target("c"))
+
+ export_directory = tvm.contrib.utils.tempdir(keep_for_debug=True).path
+ print(f"Generated files are in {export_directory}")
+ compile_and_run(
+ AOTTestModel(module=mod, inputs=inputs, outputs=output_list),
+ test_runner,
+ interface_api="c",
+ use_unpacked_api=True,
+ target=target,
+ test_dir=str(export_directory),
+ )
+
+
+if __name__ == "__main__":
+ main()
Review Comment:
we would tend to move anything TVMC and CLI related to the next PR
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
+
+ Tool to create code skeletons for an easy integration of
+ new AI hardware accelerators/libraries into TVM using UMA
+"""
+
+import argparse
+import os
+import shutil
+import sys
+from inflection import camelize, underscore
+
+
+def _parse_args():
+ parser = argparse.ArgumentParser(description="UMA Interface command line interface")
+ parser.add_argument(
+ "--add_hardware",
+ type=str,
+ required=True,
+ )
+ parser.add_argument(
+ "--tutorial",
+ type=str,
+ )
+ args = parser.parse_args()
+ return args
+
+
+def replace_template_name(
+ files: list, template_name: str, add_hw_name: str, template_source: str = "_template"
+) -> None:
+ """
+ Replace names in template skeleton code by new name
+ """
+ for f in files:
+ with open(f) as read_file:
+ data = read_file.read()
+ for case in [underscore, camelize]:
+ data = data.replace(case(template_name), case(add_hw_name))
+ data = data.replace(template_source, underscore(add_hw_name))
+ with open(f, "w") as write_file:
+ write_file.write(data)
+
+
+def main():
Review Comment:
we would tend to move anything TVMC and CLI related to the next PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] cbalint13 commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
cbalint13 commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1190026779
> thanks @MichaelJKlaiber I've given this a look over. I think it's generally in the right direction, suggest a couple more comments and a few minor changes here and there. overall though, the approach looks great. i'd love it if we could explore integrating this further with tvmc since we are registering a custom Target. happy to discuss further!
@areusch ,
IMHO, for consideration purpose:
- UMA with it's approach to bridge "out-of-tree" things/implementations into TVM might have "looser" tie & offerings for tvmc. I would only second the idea that tvmc's "exposing fixed CLI features" **must** relay when in reality it only **may** reach what (full/partially) will ever be implemented (by UMA consumers) on top of UMA bridge itself. And this **may** statement is valid even if we register a full tvm legit custom ```target``` via UMA.
- I can imagine someone's only goal via UMA is to generate lowered "c" code blobs and **period**, code that includes super-optimised (inc. auto-tensorized) __asm(".word " 0xc0fefe) calls for very special ISA extensions. I won't condition this PR to tvmc with **a must** however some really obvious things probably **can be** exposed via tvmc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] sunggg commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
sunggg commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925792693
##########
python/tvm/relay/backend/contrib/uma/api/partitioner.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Partitioner base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Dict, List, Tuple, Optional
+
+import tvm
+from tvm import relay
+from tvm.relay.build_module import bind_params_by_name
+from tvm.relay.op.contrib.register import register_pattern_table
+from .utils import PassPhase
+
+
+PatternTable = List[Tuple[str, tvm.relay.dataflow_pattern.DFPattern, Callable]]
+
+
+class UMAPartitioner:
Review Comment:
@cgerum Cool! Thanks for the clarification :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r926627576
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
Review Comment:
Just talked to @cgerum. We would tend to move it to apps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] cgerum commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
cgerum commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925423264
##########
python/tvm/relay/backend/contrib/uma/api/partitioner.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Partitioner base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Dict, List, Tuple, Optional
+
+import tvm
+from tvm import relay
+from tvm.relay.build_module import bind_params_by_name
+from tvm.relay.op.contrib.register import register_pattern_table
+from .utils import PassPhase
+
+
+PatternTable = List[Tuple[str, tvm.relay.dataflow_pattern.DFPattern, Callable]]
+
+
+class UMAPartitioner:
Review Comment:
It is intended to play nicely with Collage. But we do not have the full plumbing in place yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936302355
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939213435
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
+ self._tir_passes: List[Tuple[PassPhase, tvm.tir.transform.PrimFuncPass]] = []
+
+ def _lower_relay_to_tir(self, relay_prim_func: relay.Function) -> tvm.tir.PrimFunc:
+ """Lower a Relay primitive function to a S-TIR primitive function.
+
+ Parameters
+ ----------
+ prim_func : tvm.relay.Function
+ The Relay function to lower.
+
+ Returns
+ -------
+ out : tvm.tir.PrimFunc
+ The lowered schedulable TensorIR primitive function.
+
+ """
+
+ def _get_tensors(te_cached_func):
+ outputs = list(te_cached_func.outputs)
+ stack = []
+ visited = set()
+ for output_ in outputs:
+ if output_ not in visited:
+ visited.add(output_)
+ stack.append(output_)
+
+ args = []
+ while len(stack) != 0:
+ tensor = stack.pop()
+ if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+ args.append(tensor)
+ elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+ inputs = tensor.op.input_tensors
+ for input_ in inputs:
+ if input_ not in visited:
+ visited.add(input_)
+ stack.append(input_)
+
+ return args + outputs
+
+ f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
Review Comment:
ah, sorry, you are referring to the f. I thought you meant the relay.backend.LowerToTE.
Good that @cgerum understood what you meant and fixed it :smirk:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930041551
##########
python/tvm/relay/backend/contrib/uma/uma_cli.py:
##########
@@ -0,0 +1,92 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+ UMA Command Line Interface (CLI)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925590913
##########
include/tvm/relay/transform.h:
##########
@@ -509,6 +509,8 @@ TVM_DLL Pass SimplifyExpr();
*
* \param config All available targets.
*
+ * \param config All available targets.
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930006872
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930022435
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+ partitioner = UMAPartitioner("test_partition")
+ assert get_pattern_table("test_partition") is None
+
+ partitioner.register()
+
+ assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+ "workload,backend,merge,expected_partitions",
+ [
+ ("resnet", "dnnl", False, 17),
+ ("resnet", "dnnl", True, 17),
+ ("mlp", "dnnl", False, 1),
+ ("resnet", "cutlass", False, 2),
+ ("resnet", "cutlass", True, 2),
+ ("mlp", "cutlass", False, 4),
+ ("mlp", "cutlass", True, 2),
+ ],
+)
+def test_existing_pattern_tables(workload, backend, merge, expected_partitions):
+ partitioner = UMAPartitioner(backend + "_uma", merge)
+ pattern_table = get_pattern_table(backend)
+
+ for entry in pattern_table:
+ partitioner.add_pattern(*entry)
+
+ if workload == "resnet":
+ net = resnet.get_net(1, 10)
+ elif workload == "mlp":
+ net = mlp.get_net(1, 10)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930028351
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
@areusch, should we move this to a seperate PR then?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] kslavka commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
kslavka commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925268875
##########
python/tvm/relay/backend/contrib/uma/tutorial.md:
##########
@@ -0,0 +1,195 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+Making your hardware accelerator TVM-ready with UMA
+=============================================
+
+**Disclaimer**: *This is an early preliminary version of this tutorial. Feel free to aks questions or give feedback via the UMA thread in the TVM
+discussion forum [[link](https://discuss.tvm.apache.org/t/rfc-uma-universal-modular-accelerator-interface/12039)].*
+
+
+This tutorial will give you step-by-step guidance how to use UMA to
+make your hardware accelerator TVM-ready.
+While there is no one-fits-all solution for this problem, UMA targets to provide a stable and Python-only
+API to integrate a number of hardware accelerator classes into TVM.
+
+In this tutorial you will get to know the UMA API in three use cases of increasing complexity.
+In these use case the three mock-accelerators
+**Vanilla**, **Strawberry** and **Chocolate** are introduced and
+integrated into TVM using UMA.
+
+
+Vanilla
+===
+**Vanilla** is a simple accelerator consisting of a MAC array and has no internal memory.
+It is can ONLY process Conv2D layers, all other layers are executed on a CPU, that also orchestrates **Vanilla**.
+Both the CPU and Vanilla use a shared memory.
+
+For this purpose **Vanilla** has a C interface `vanilla_conv2dnchw`, that accepts pointers to input data *if_map*,
+*weights* and *result* data, as well as the parameters of `Conv2D`: `oc`, `iw`, `ih`, `ic`, `kh`, `kw`.
+```c
+int vanilla_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic, int kh, int kw);
+```
+
+The script `uma_cli` creates code skeletons with API-calls into the UMA-API for new accelerators.
+For **Vanilla** we use it like this:
+
+```
+cd tvm/python/tvm/relay/backend/contrib/uma
+python uma_cli.py --add-accelerator vanilla_accelerator --tutorial vanilla
Review Comment:
Should it be "--add_hardware" as it is coded in uma_cli.py?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929946409
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r928579415
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r938582720
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
Review Comment:
Would not this break if includes was not provided ?
Also I think this is a user facing error and ICHECK is not suitable for such.
Please change this to LOG(FATAL) or CHECK() and that needs to be tested to ensure the error is emitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r938581055
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Optional
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(self, includes: Optional[Callable[[], str]] = None) -> None:
+ """Registration of UMA helper functions, e.g. includes and replace_call_extern.
+
+ Parameters
+ ----------
+ includes : OptionalCallable[[], str]]
+ user-defined function that adds C-#include statement to UMA C-Code.
+ """
+ if includes is not None:
Review Comment:
It seems like "includes" is an important option.
It would better to declare that separately to kwargs ?
Also following the current code does not look like includes is optional. See the comment in the other place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930020506
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(
+ self,
+ includes: Callable[[], str] = None,
+ replace_call_extern: Callable[[tvm.ir.container.Array], str] = None,
+ ) -> None:
+ if includes is not None:
+ tvm._ffi.register_func(
+ "relay.ext.uma.codegen_c_includes_{}".format(self.target_name), includes
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r940045662
##########
apps/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,95 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+
+ /*!
+ * \brief Conv2D function for mock-accelerator examples. Limited to same-padded Conv2D with
+ * stride (1,1) and datatype float. \param ifmap Pointer to input feature map data of size
+ * iw*ih*ic*sizeof(float). \param weights Pointer to weight data of size
+ * kh*kw*ic**oc*sizeof(float). \param result Pointer to output feature map data of size
+ * iw*ih*oc*sizeof(float). \param oc Number of channels of output feature map. \param iw Width
+ * of input feature map, ifmap. \param ih Height of input feature map, ifmap. \param ic Number
+ * of channels of input feature map. \param kh Height of convolution kernels. \param kw Width of
+ * convolution kernels.
+ *
+ * \return error code
+ *
+ */
+ int
+ my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic,
Review Comment:
@leandron @areusch @driazati do you know any reason that this file will not be linted ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
areusch commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r937941027
##########
apps/uma/_template/__init__.py:
##########
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+
+Template files for UMA tutorial
+
+Do not import
Review Comment:
just noting that it's a bit weird to say do not import but provide an `__init__.py`. you could prevent importing by deleting this file
##########
apps/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,95 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+
Review Comment:
could you guys add a TODO and file an issue to leverage pragma `import_c` in the future?
##########
apps/uma/_template/passes.py:
##########
@@ -0,0 +1,142 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ _EXTERNAL_FUNCTION_NAME = "my_ai_hw_conv2dnchw"
+ _TVM_BLOCK_MATCH_NAME = "conv2d_nchw"
+
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @classmethod
+ def _my_ai_hw_conv2d_pass(cls, func, mod, ctx):
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+
+ def _has_block(name: str, func: tvm.tir.PrimFunc) -> bool:
+ """
+ Determine of a tir.block with `name` exists in `func`
+ """
+
+ def _hb(op):
+ if isinstance(op, tvm.tir.Block):
+ _found_blocks.append(op.name_hint)
+
+ _found_blocks = []
+ tvm.tir.stmt_functor.post_order_visit(func.body, _hb)
+ return name in _found_blocks
+
+ def _detect_and_replace_conv2d(
+ func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ def _replace_conv2d(op):
+ if op == _entry_node:
+ irb = tvm.tir.ir_builder.create()
+ # Collection of buffer address
+ buffers = [b[1].data for b in _handles]
+ # extraction of loop offsets
+ for k, v in _loops.items():
+ assert v.min.value == 0
+ offset_order = ["co", "w", "h", "ci", "kh", "kw"]
+ offsets = [_loops[i].extent.value for i in offset_order]
+ args = buffers + offsets
+ external_call = tvm.tir.Evaluate(
+ tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args)
+ )
+ ext_calls = tvm.tir.SeqStmt([external_call])
Review Comment:
just noting i think you can just say:
```
irb.emit(tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args))
```
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
Review Comment:
just noting i think this one isn't done yet
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
ok, i'm fine to defer this to a follow-on PR if @manupa-arm is ok with that.
##########
apps/uma/_template/passes.py:
##########
@@ -0,0 +1,142 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ _EXTERNAL_FUNCTION_NAME = "my_ai_hw_conv2dnchw"
+ _TVM_BLOCK_MATCH_NAME = "conv2d_nchw"
+
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @classmethod
+ def _my_ai_hw_conv2d_pass(cls, func, mod, ctx):
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+
+ def _has_block(name: str, func: tvm.tir.PrimFunc) -> bool:
+ """
+ Determine of a tir.block with `name` exists in `func`
+ """
+
+ def _hb(op):
+ if isinstance(op, tvm.tir.Block):
+ _found_blocks.append(op.name_hint)
+
+ _found_blocks = []
+ tvm.tir.stmt_functor.post_order_visit(func.body, _hb)
+ return name in _found_blocks
+
+ def _detect_and_replace_conv2d(
+ func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ def _replace_conv2d(op):
+ if op == _entry_node:
+ irb = tvm.tir.ir_builder.create()
+ # Collection of buffer address
+ buffers = [b[1].data for b in _handles]
+ # extraction of loop offsets
+ for k, v in _loops.items():
+ assert v.min.value == 0
+ offset_order = ["co", "w", "h", "ci", "kh", "kw"]
+ offsets = [_loops[i].extent.value for i in offset_order]
+ args = buffers + offsets
+ external_call = tvm.tir.Evaluate(
+ tir_call(irb, True, cls._EXTERNAL_FUNCTION_NAME, *args)
+ )
+ ext_calls = tvm.tir.SeqStmt([external_call])
+ irb.emit(ext_calls)
+ irb_result = irb.get()
+ return irb_result
+ elif isinstance(op, tvm.tir.SeqStmt):
+ # Remove that pad block of TOPI's conv2DNCHW by only returning the 2nd statement
+ return op.seq[1]
+ return op
+
+ sch = tir.Schedule(func)
+
+ if _has_block(cls._TVM_BLOCK_MATCH_NAME, func):
+ conv2d_block = sch.get_block(cls._TVM_BLOCK_MATCH_NAME)
+ rv_loops = sch.get_loops(conv2d_block)
+ assert len(rv_loops) == 7
+ loops = dict(
+ n=rv_loops[0],
+ co=rv_loops[1],
+ h=rv_loops[2],
+ w=rv_loops[3],
+ ci=rv_loops[4],
+ kh=rv_loops[5],
+ kw=rv_loops[6],
+ )
+ _entry_node = sch.get(rv_loops[1])
+ _loops = {k: sch.get(v) for k, v in loops.items()}
+ _handles = func.buffer_map.items()
+
+ x = tvm.tir.stmt_functor.ir_transform(
+ func.body, None, _replace_conv2d, ["tir.For", "tir.SeqStmt"]
+ )
+ return func.with_body(x)
+ else:
+ return func
+
+ r = _detect_and_replace_conv2d(func, mod, ctx)
+ return r
+
+
+def tir_call(ib: tvm.tir.ir_builder, extern: bool, name: str, *args):
+ """
+ ib: ir_builder
+ extern: bool
+ True --> tvm.tir.call_extern
+ False --> tvm.tir.call_packed
+ name: str
+ function name
+ *args:
+ arguments for function call
+ """
+
+ def buf_from_array(ib, arr, dtype):
+ # Allocate enough memory to store the whole array
+ var = ib.allocate("int32", (len(arr),), scope="global")
+ for i, v in enumerate(arr):
+ var[i] = v
+ # Declare a buffer, which is basically a view on the chunk of memory that we allocated
+ buf = tvm.tir.decl_buffer((len(arr),), dtype, data=var, scope="global")
+ return buf
+
+ if extern:
+ args = [i.data if isinstance(i, tvm.tir.Buffer) else i for i in args]
+ call = tvm.tir.call_extern("int32", name, *args)
+ else:
+ args = [
+ buf_from_array(ib, i, "int32")
+ if isinstance(i, (tuple, list, tvm.ir.container.Array))
+ else i
+ for i in args
+ ]
+ call = tvm.tir.call_packed(name, *args)
+
+ return call
Review Comment:
any reason not to just emit the call here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939153521
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
+ self._tir_passes: List[Tuple[PassPhase, tvm.tir.transform.PrimFuncPass]] = []
+
+ def _lower_relay_to_tir(self, relay_prim_func: relay.Function) -> tvm.tir.PrimFunc:
+ """Lower a Relay primitive function to a S-TIR primitive function.
+
+ Parameters
+ ----------
+ prim_func : tvm.relay.Function
+ The Relay function to lower.
+
+ Returns
+ -------
+ out : tvm.tir.PrimFunc
+ The lowered schedulable TensorIR primitive function.
+
+ """
+
+ def _get_tensors(te_cached_func):
+ outputs = list(te_cached_func.outputs)
+ stack = []
+ visited = set()
+ for output_ in outputs:
+ if output_ not in visited:
+ visited.add(output_)
+ stack.append(output_)
+
+ args = []
+ while len(stack) != 0:
+ tensor = stack.pop()
+ if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+ args.append(tensor)
+ elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+ inputs = tensor.op.input_tensors
+ for input_ in inputs:
+ if input_ not in visited:
+ visited.add(input_)
+ stack.append(input_)
+
+ return args + outputs
+
+ f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
Review Comment:
well, probably a matter of taste :smile: .
We followed the convention/naming used in ethossu/tir/compiler.py
@cgerum: any thoughts here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] cbalint13 commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
cbalint13 commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1186011020
Cc @driazati may help with missing ```clang``` [i386 test failure](https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm/branches/PR-12087/runs/7/nodes/360/steps/1197/log/?start=0) @ ```test_lower_with_uma[1-224-224-3-5-5-4]```
```
[2022-07-15T13:33:16.200Z] > raise RuntimeError("cannot find clang, candidates are: " + str(cc_list))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925592054
##########
python/tvm/relay/backend/contrib/uma/tutorial.md:
##########
@@ -0,0 +1,195 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+Making your hardware accelerator TVM-ready with UMA
Review Comment:
Generally yes. Would be great!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925689353
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review Comment:
Given the out-of-tree usecase of UMA, I personally feel this might be better suited as an apps/ example, where we showcase to a "user" how UMA could be used. WDYT ?
Even if its a template (in which case, we should hold the template here, maybe ?), however it would be still good demonstrate how to use that template is used in a "user" flow.
*user here means a out-of-tree dev that wants to attach accelerator codegen to TVM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] PaulPalomeroBernardo commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
PaulPalomeroBernardo commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925717162
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
this is in fact just a copy of the ethosu code. I think, it is a generally useful pass for accelerator flows, so +1 to make this a generic transform pass, if this is an option.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1200925097
CI pipeline still fails for the target i386, see https://ci.tlcpack.ai/blue/rest/organizations/jenkins/pipelines/tvm/branches/PR-12087/runs/12/nodes/360/steps/1207/log/?start=0
As this does not seem an issue introduced by UMA code, I will remove UMA for i386 build, i.e.
tests/scripts/task_config_build_i386.sh
`echo set(USE_UMA OFF) >> config.cmake`
What are your thoughts?
@cbalint13 @areusch @areusch @cgerum @manupa-arm @PaulPalomeroBernardo
Does anyone of you understand what is different from i386 to build_cpu?
Does it have to do with:
echo set\(USE_LLVM llvm-config-4.0\) >> config.cmake
VS
echo set\(USE_LLVM llvm-config-11\) >> config.cmake
?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1190262638
> @cbalint13, appreciate sharing your thoughts! I also think this is a great out-of-tree approach and look forward to its actual usage.
>
> > Could you add more documentation in general (e.g., explain what parameters are)?
> > > Current sample (_template) & doc are fine for a contrib module.
> > > It greatly enables "out-of-tree" maintenance towards custom HW accelarator design.
> > > Example parts Vanilla, Strawberry & Chocolate are nice levels of introductory & guidance.
>
> I agree that the overall docstring structure (e.g., Vanilla, Strawberry & Chocolate) would be a great introduction/tutorial (Truly appreciate the authors!). My issue was more like a code-level docstring since some of them are missing and I wasn't sure what each function or parameter exactly means. https://github.com/apache/tvm/blob/bc211fdc91932ef9b77f7edd2ff65151ee5faaff/python/tvm/relay/backend/contrib/uma/_template/codegen.py https://github.com/apache/tvm/blob/bc211fdc91932ef9b77f7edd2ff65151ee5faaff/python/tvm/relay/backend/contrib/uma/_template/codegen.py
>
> > It would be great to have a testing for end-to-end model-level testing. (e.g., run BERT with UMA on ARM EthosU)
> > > Sounds beyond of scope, same way we could "dream" a fully verilated BERT running verilator-contrib module (!) @tmoreau89 , @vegaluisjose
> >
> >
> > > As for ARM-Ethos ⟨™⟩, my understanding is a quite different (not so "out-of-tree" principled) way to connect things inside, but @manupa-arm can comment more on this.
> >
> >
> > > Would be possible in the future to decouple ARM-Ethos⟨™⟩ (or Vitis-AI) to be more "out-of-tree" by porting it over UMA, but this would be quite bit of another task.
>
> I see. Thank you for the clarification! Maybe my example was not relevant. I just thought it would be nice to have some model-level test cases since we seem to have all necessary components (UMA Partitioner, UMA Pipeline, UMA Lower, UMA Codegen) in this PR. I just wanted to see if current PR also covers a case where we lower some parts with UMA while handling other unsupported parts via the conventional pipeline.
We have an example testcase for running MobileNet (created in Relay) with uma and the Vanilla Accelerator in place. Not sure if it should be added to this PR or a later one, as this PR already is a lot to discuss. I'd be in favor of moving this to the next UMA PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r936299201
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+ partitioner = UMAPartitioner("test_partition")
+ assert get_pattern_table("test_partition") is None
+
+ partitioner.register()
+
+ assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+ "workload,backend,merge,expected_partitions",
+ [
+ ("resnet", "dnnl", False, 17),
+ ("resnet", "dnnl", True, 17),
Review Comment:
@areusch, we removed the hardcoded integers for the expected paritions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r938577024
##########
apps/uma/_template/strategies.py:
##########
@@ -0,0 +1,17 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Strategies for the my_ai_hw accelerator"""
Review Comment:
Maybe it would be better to write some explanation (in the absense of an example) around what should go here
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Optional
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(self, includes: Optional[Callable[[], str]] = None) -> None:
+ """Registration of UMA helper functions, e.g. includes and replace_call_extern.
+
+ Parameters
+ ----------
+ includes : OptionalCallable[[], str]]
+ user-defined function that adds C-#include statement to UMA C-Code.
+ """
+ if includes is not None:
Review Comment:
It seems like "includes" is an important option.
It would better to declare that separately to kwargs ?
Also following the code the current code does not look like includes is optional. See the comment in the other place.
##########
include/tvm/relay/transform.h:
##########
@@ -509,6 +509,7 @@ TVM_DLL Pass SimplifyExpr();
*
* \param config All available targets.
*
+ *
Review Comment:
nit : was this intentional ?
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
+ self._tir_passes: List[Tuple[PassPhase, tvm.tir.transform.PrimFuncPass]] = []
+
+ def _lower_relay_to_tir(self, relay_prim_func: relay.Function) -> tvm.tir.PrimFunc:
+ """Lower a Relay primitive function to a S-TIR primitive function.
+
+ Parameters
+ ----------
+ prim_func : tvm.relay.Function
+ The Relay function to lower.
+
+ Returns
+ -------
+ out : tvm.tir.PrimFunc
+ The lowered schedulable TensorIR primitive function.
+
+ """
+
+ def _get_tensors(te_cached_func):
+ outputs = list(te_cached_func.outputs)
+ stack = []
+ visited = set()
+ for output_ in outputs:
+ if output_ not in visited:
+ visited.add(output_)
+ stack.append(output_)
+
+ args = []
+ while len(stack) != 0:
+ tensor = stack.pop()
+ if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+ args.append(tensor)
+ elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+ inputs = tensor.op.input_tensors
+ for input_ in inputs:
+ if input_ not in visited:
+ visited.add(input_)
+ stack.append(input_)
+
+ return args + outputs
+
+ f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
+ te_cached_func = f(relay_prim_func)
+ x = _get_tensors(te_cached_func)
+ tir_prim_func = te.create_prim_func(x)
+ tir_prim_func = tir_prim_func.with_attr(
+ "global_symbol", relay_prim_func.attrs["global_symbol"]
+ )
+ # TODO: The target should probably come from somewhere else instead of being created here.
+ tir_prim_func = tir_prim_func.with_attr("target", tvm.target.Target(self.target_name))
Review Comment:
Does not the relay prim func has the kCompiler attr ?
Given that we have this duplicity betwen kCompiler string and target name, I suppose we can align to use the same "value" at both places. Thus to guarantee that, maybe we can get rid of target name in UMALower ?
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
Review Comment:
Would not this break if includes was not provided ?
Also I think this is a user facing error and ICHECK is not suitable for such.
Please change this LOG(FATAL) or CHECK() and that needs to be tested to ensure the error is emitted.
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
+ self._tir_passes: List[Tuple[PassPhase, tvm.tir.transform.PrimFuncPass]] = []
+
+ def _lower_relay_to_tir(self, relay_prim_func: relay.Function) -> tvm.tir.PrimFunc:
+ """Lower a Relay primitive function to a S-TIR primitive function.
+
+ Parameters
+ ----------
+ prim_func : tvm.relay.Function
+ The Relay function to lower.
+
+ Returns
+ -------
+ out : tvm.tir.PrimFunc
+ The lowered schedulable TensorIR primitive function.
+
+ """
+
+ def _get_tensors(te_cached_func):
+ outputs = list(te_cached_func.outputs)
+ stack = []
+ visited = set()
+ for output_ in outputs:
+ if output_ not in visited:
+ visited.add(output_)
+ stack.append(output_)
+
+ args = []
+ while len(stack) != 0:
+ tensor = stack.pop()
+ if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+ args.append(tensor)
+ elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+ inputs = tensor.op.input_tensors
+ for input_ in inputs:
+ if input_ not in visited:
+ visited.add(input_)
+ stack.append(input_)
+
+ return args + outputs
+
+ f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
Review Comment:
nit : it would be better use a descriptive name here. e.g. f_lower_to_te
##########
apps/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,95 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+
+ /*!
+ * \brief Conv2D function for mock-accelerator examples. Limited to same-padded Conv2D with
+ * stride (1,1) and datatype float. \param ifmap Pointer to input feature map data of size
+ * iw*ih*ic*sizeof(float). \param weights Pointer to weight data of size
+ * kh*kw*ic**oc*sizeof(float). \param result Pointer to output feature map data of size
+ * iw*ih*oc*sizeof(float). \param oc Number of channels of output feature map. \param iw Width
+ * of input feature map, ifmap. \param ih Height of input feature map, ifmap. \param ic Number
+ * of channels of input feature map. \param kh Height of convolution kernels. \param kw Width of
+ * convolution kernels.
+ *
+ * \return error code
+ *
+ */
+ int
+ my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic,
Review Comment:
This source looks like as it was not linted. Maybe we dont lint apps ..
Anyhow, would you be able to run clang-format on this file ?
##########
src/relay/backend/contrib/uma/targets.cc:
##########
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/targets.cc
+ *
+ * \brief this file contains the targets for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+
+namespace tvm {
+
+namespace relay {
+namespace contrib {
+namespace uma {
+tvm::transform::Pass RelayToTIR(String target_name);
+runtime::Module TIRToRuntime(IRModule mod, Target target);
+} // namespace uma
+} // namespace contrib
+} // namespace relay
+
+TVM_REGISTER_GLOBAL("relay.backend.contrib.uma.RegisterTarget")
+ .set_body_typed([](String target_name, Map<String, ObjectRef> attr_options) -> bool {
+ // @todo(cgerum): We probably should get rid of target.register rather sooner than later
+ // And use a proper registry for uma backends
+ for (const String registered_target_name : ::tvm::TargetKindRegEntry::ListTargetKinds()) {
+ if (registered_target_name == target_name) {
+ return false;
+ }
+ }
+
+ auto target_kind =
+ ::tvm::TargetKindRegEntry::RegisterOrGet(target_name)
+ .set_name()
+ .set_device_type(kDLCPU)
+ .add_attr_option<Array<String>>("keys")
+ .add_attr_option<String>("tag")
+ .add_attr_option<String>("device")
+ .add_attr_option<String>("model")
+ .add_attr_option<Array<String>>("libs")
+ .add_attr_option<Target>("host")
+ .add_attr_option<Integer>("from_device")
+ .set_attr<FTVMRelayToTIR>(tvm::attr::kRelayToTIR,
+ relay::contrib::uma::RelayToTIR(target_name))
+ .set_attr<FTVMTIRToRuntime>("TIRToRuntime", relay::contrib::uma::TIRToRuntime);
+
+ for (auto& attr_option : attr_options) {
+ auto option_name = attr_option.first;
+ auto default_value = attr_option.second;
+ if (default_value->IsInstance<StringObj>()) {
+ target_kind.add_attr_option<String>(option_name, Downcast<String>(default_value));
+ } else if (default_value->IsInstance<IntImmNode>()) {
+ target_kind.add_attr_option<Integer>(option_name, Downcast<Integer>(default_value));
+ } else {
+ LOG(FATAL) << "Attribute option of type " << attr_option.second->GetTypeKey()
Review Comment:
We also need to test whether this user facing error is emitted
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
Maybe with a TODO/issue then ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r928576011
##########
cmake/config.cmake:
##########
@@ -296,9 +296,6 @@ set(USE_VTA_FPGA OFF)
# Whether use Thrust
set(USE_THRUST OFF)
-# Whether use cuRAND
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r929993795
##########
python/tvm/relay/backend/contrib/uma/_template/conv2dnchw.cc:
##########
@@ -0,0 +1,76 @@
+/*
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+*/
+#include <stdlib.h>
+
+#ifdef __cplusplus
+extern "C"
+#endif
+ int
+ my_ai_hw_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic,
+ int kh, int kw) {
+
+ int kw_low = kw / 2;
+ int kh_low = kh / 2;
+ int kw_high = iw + kw / 2;
+ int kh_high = ih + kh / 2;
+
+ int padded_iw = iw + 2 * kw_low;
+ int padded_ih = ih + 2 * kh_low;
+
+ float* pad_temp = (float*)malloc(
Review Comment:
Not really, this should imitate a driver/low-level interface to an accelerator.
Therefore, C/C++-only is intended to demonstrate that this part in user-specific.
We added some additional comment to clarify
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930030731
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
+ public:
+ explicit OutlineCompilerFunctionsMutator(const IRModule& mod, const std::string& compiler_name)
+ : mod_(mod), compiler_name_(compiler_name) {}
+
+ Expr VisitExpr_(const LetNode* op) final {
+ auto pre_visit = [this](const LetNode* op) {
+ Expr var = this->VisitExpr(op->var);
+ Expr value = this->VisitExpr(op->value);
+
+ // Outlineable function no longer needs let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[var] = value;
+ }
+ };
+ auto post_visit = [this](const LetNode* op) {
+ // Rely on the Memoizer to cache pre-visit values
+ Expr value = this->VisitExpr(op->value);
+ Expr body = this->VisitExpr(op->body);
+ auto expr = GetRef<Expr>(op);
+
+ // Drop the let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[expr] = this->VisitExpr(op->body);
+ } else {
+ Var var = Downcast<Var>(this->VisitExpr(op->var));
+ if (var.same_as(op->var) && value.same_as(op->value) && body.same_as(op->body)) {
+ this->memo_[expr] = expr;
+ } else {
+ this->memo_[expr] = Let(var, value, body);
+ }
+ }
+ };
+ ExpandANormalForm(op, pre_visit, post_visit);
+ return memo_[GetRef<Expr>(op)];
+ }
+
+ Expr Rewrite_(const CallNode* pre, const Expr& post) override {
+ Call call = Downcast<Call>(post);
+ if (CanOutlineExpr(call->op)) {
+ Function func = Downcast<Function>(call->op);
+ auto gv_name = func->GetAttr<String>("global_symbol").value_or("");
+ ICHECK_NE(gv_name, "")
+ << "Function to be outlined must have global_symbol attribute, but didn't.";
+ GlobalVar gv(gv_name);
+ if (func->checked_type_.defined()) {
+ gv->checked_type_ = func->checked_type();
+ }
+ mod_->Update(gv, func);
+ return Call(gv, call->args, call->attrs, call->type_args);
+ }
+ return post;
+ }
+
+ private:
+ /*!
+ * \brief Check if the expr is a function and has the same
+ * compiler name as compiler_name_.
+ *
+ * \param expr The input expr.
+ * \return True if is outlineable else False.
+ */
+ bool CanOutlineExpr(const Expr& expr) {
+ if (!expr->IsInstance<FunctionNode>()) {
+ return false;
+ }
+ Function func = Downcast<Function>(expr);
+ auto compiler = func->GetAttr<String>(attr::kCompiler);
+ if (!compiler.defined()) {
+ return false;
+ }
+ if (compiler != compiler_name_) {
+ return false;
+ }
+ return true;
+ }
+
+ /*! \brief The module that the pass will run on. */
+ IRModule mod_;
+ /*! \brief The name of the compiler to enable outlining on external functions for. */
+ std::string compiler_name_;
+};
+
+/*!
+ * \brief A pass to outline compiler specific functions.
+ */
+tvm::transform::Pass OutlineCompilerFunctions(const std::string& compiler_name) {
+ runtime::TypedPackedFunc<IRModule(IRModule, transform::PassContext)> pass_func =
+ [=](IRModule mod, transform::PassContext ctx) {
+ GlobalVar gv = mod->GetGlobalVar("main");
+ Function main_func = Downcast<Function>(mod->Lookup("main"));
+ auto new_main_body =
+ OutlineCompilerFunctionsMutator(mod, compiler_name).VisitExpr(main_func->body);
+ if (!new_main_body.same_as(main_func->body)) {
+ Function new_main_func = WithFields(main_func, main_func->params, new_main_body);
+ mod->Update(gv, new_main_func);
+ }
+ return mod;
+ };
+ return tvm::transform::CreateModulePass(pass_func, 0,
+ "relay.backend.contrib.uma.OutlineCompilerFunctions", {});
+}
+
+TVM_REGISTER_GLOBAL("relay.ext.uma.OutlineCompilerFunctions")
+ .set_body_typed(OutlineCompilerFunctions);
+
+/*!
+ * \brief This pass will lower NPU functions in a Relay module to scheduled TIR prim functions.
Review Comment:
@areusch , can you elaborate what needs to be fixed here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] sunggg commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
sunggg commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r923907669
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
Review Comment:
`_template` directory seems for testing/example purpose? At first, it sounded like it is a directory for templates that users can invoke or inherit to customize their accelerators to me.
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_replace_call_extern_" + target_str_);
+ if (replace_call_extern_pf == nullptr) {
+ CodeGenCHost::VisitExpr_(op, os);
+ } else {
+ // - funtion type (void) still gets printed before CallNode if extern call is wrapped in
+ // EvaluateNode
+ // - VarNode arguments might have "wrong" name_hints. The correct variable name is determined
+ // in C++ through GetVarID
+ String api_string = (*replace_call_extern_pf)(op->args);
+ os << api_string;
+ }
+ return;
+ }
+};
+
+runtime::Module TIRToRuntime(IRModule mod, Target target) {
+ bool output_ssa = false;
+ bool emit_asserts = false;
+ UMACodegen codegen(target->kind->name);
+ Array<String> function_names;
+ codegen.Init(output_ssa, emit_asserts);
+ for (auto kv : mod->functions) {
+ auto prim_func = Downcast<PrimFunc>(kv.second);
+ auto global_symbol = prim_func->GetAttr<String>(tvm::attr::kGlobalSymbol);
+ function_names.push_back(global_symbol.value());
+ codegen.AddFunction(prim_func);
+ }
+ std::string code = codegen.Finish();
+ return codegen::CSourceModuleCreate(code, "c", function_names);
Review Comment:
In my current understanding, we may need to invoke driver or DMA for loosely-connected accelerators (LCAs).
How does UMA codegen handle this?
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
+ # Target configuration
+ #######################################################################
+ self._register_target_attr("dimension")
+
+ #######################################################################
+ # Relay to Relay function registration
Review Comment:
Not sure if this is a correct comment. Maybe "Relay Pattern registration"?
##########
python/tvm/relay/backend/contrib/uma/api/partitioner.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Partitioner base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Dict, List, Tuple, Optional
+
+import tvm
+from tvm import relay
+from tvm.relay.build_module import bind_params_by_name
+from tvm.relay.op.contrib.register import register_pattern_table
+from .utils import PassPhase
+
+
+PatternTable = List[Tuple[str, tvm.relay.dataflow_pattern.DFPattern, Callable]]
+
+
+class UMAPartitioner:
Review Comment:
How does this partitioner works with other partitioners? For example, can we use UMA with Collage?
##########
python/tvm/relay/backend/contrib/uma/api/utils.py:
##########
@@ -0,0 +1,52 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Utility methods for the Universal Modular Accelerator Interface (UMA)"""
+
+from enum import Enum, auto
+import uuid
+import tvm.tir
+from tvm.contrib import utils, clang
+
+
+class PassPhase(Enum):
+ """UMA pass phases."""
+
+ PRE_PARTITIONING = auto()
+ POST_PARTITIONING_0 = auto()
Review Comment:
Could you elaborate what each phase means?
##########
python/tvm/relay/backend/contrib/uma/api/codegen.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Codegen base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable
+import tvm
+
+
+class UMACodegen(object):
+ """
+ Codegen base class of the Universal Modular Accelerator Interface (UMA)
+ """
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ def _register_codegen(self, fmt: str = "c", **kwargs) -> None:
+ if fmt == "c":
+ self._register_c_codegen(**kwargs)
+ else:
+ raise RuntimeError(f'Unsupported codegen format "{fmt}"')
+
+ def _register_c_codegen(
+ self,
+ includes: Callable[[], str] = None,
Review Comment:
Could you add the docstring for parameters?
Also, can `includes` be `Optional[str]`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925598695
##########
python/tvm/relay/backend/contrib/uma/tutorial.md:
##########
@@ -0,0 +1,195 @@
+<!--- Licensed to the Apache Software Foundation (ASF) under one -->
+<!--- or more contributor license agreements. See the NOTICE file -->
+<!--- distributed with this work for additional information -->
+<!--- regarding copyright ownership. The ASF licenses this file -->
+<!--- to you under the Apache License, Version 2.0 (the -->
+<!--- "License"); you may not use this file except in compliance -->
+<!--- with the License. You may obtain a copy of the License at -->
+
+<!--- http://www.apache.org/licenses/LICENSE-2.0 -->
+
+<!--- Unless required by applicable law or agreed to in writing, -->
+<!--- software distributed under the License is distributed on an -->
+<!--- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -->
+<!--- KIND, either express or implied. See the License for the -->
+<!--- specific language governing permissions and limitations -->
+<!--- under the License. -->
+
+Making your hardware accelerator TVM-ready with UMA
+=============================================
+
+**Disclaimer**: *This is an early preliminary version of this tutorial. Feel free to aks questions or give feedback via the UMA thread in the TVM
+discussion forum [[link](https://discuss.tvm.apache.org/t/rfc-uma-universal-modular-accelerator-interface/12039)].*
+
+
+This tutorial will give you step-by-step guidance how to use UMA to
+make your hardware accelerator TVM-ready.
+While there is no one-fits-all solution for this problem, UMA targets to provide a stable and Python-only
+API to integrate a number of hardware accelerator classes into TVM.
+
+In this tutorial you will get to know the UMA API in three use cases of increasing complexity.
+In these use case the three mock-accelerators
+**Vanilla**, **Strawberry** and **Chocolate** are introduced and
+integrated into TVM using UMA.
+
+
+Vanilla
+===
+**Vanilla** is a simple accelerator consisting of a MAC array and has no internal memory.
+It is can ONLY process Conv2D layers, all other layers are executed on a CPU, that also orchestrates **Vanilla**.
+Both the CPU and Vanilla use a shared memory.
+
+For this purpose **Vanilla** has a C interface `vanilla_conv2dnchw`, that accepts pointers to input data *if_map*,
+*weights* and *result* data, as well as the parameters of `Conv2D`: `oc`, `iw`, `ih`, `ic`, `kh`, `kw`.
+```c
+int vanilla_conv2dnchw(float* ifmap, float* weights, float* result, int oc, int iw, int ih, int ic, int kh, int kw);
+```
+
+The script `uma_cli` creates code skeletons with API-calls into the UMA-API for new accelerators.
+For **Vanilla** we use it like this:
+
+```
+cd tvm/python/tvm/relay/backend/contrib/uma
+python uma_cli.py --add-accelerator vanilla_accelerator --tutorial vanilla
Review Comment:
@kslavka , nice catch! Thanks. Fixed it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
areusch commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r931325454
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
Review Comment:
`@classmethod` should fix this
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
+ public:
+ explicit OutlineCompilerFunctionsMutator(const IRModule& mod, const std::string& compiler_name)
+ : mod_(mod), compiler_name_(compiler_name) {}
+
+ Expr VisitExpr_(const LetNode* op) final {
+ auto pre_visit = [this](const LetNode* op) {
+ Expr var = this->VisitExpr(op->var);
+ Expr value = this->VisitExpr(op->value);
+
+ // Outlineable function no longer needs let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[var] = value;
+ }
+ };
+ auto post_visit = [this](const LetNode* op) {
+ // Rely on the Memoizer to cache pre-visit values
+ Expr value = this->VisitExpr(op->value);
+ Expr body = this->VisitExpr(op->body);
+ auto expr = GetRef<Expr>(op);
+
+ // Drop the let binding
+ if (this->CanOutlineExpr(value)) {
+ this->memo_[expr] = this->VisitExpr(op->body);
+ } else {
+ Var var = Downcast<Var>(this->VisitExpr(op->var));
+ if (var.same_as(op->var) && value.same_as(op->value) && body.same_as(op->body)) {
+ this->memo_[expr] = expr;
+ } else {
+ this->memo_[expr] = Let(var, value, body);
+ }
+ }
+ };
+ ExpandANormalForm(op, pre_visit, post_visit);
+ return memo_[GetRef<Expr>(op)];
+ }
+
+ Expr Rewrite_(const CallNode* pre, const Expr& post) override {
+ Call call = Downcast<Call>(post);
+ if (CanOutlineExpr(call->op)) {
+ Function func = Downcast<Function>(call->op);
+ auto gv_name = func->GetAttr<String>("global_symbol").value_or("");
+ ICHECK_NE(gv_name, "")
+ << "Function to be outlined must have global_symbol attribute, but didn't.";
+ GlobalVar gv(gv_name);
+ if (func->checked_type_.defined()) {
+ gv->checked_type_ = func->checked_type();
+ }
+ mod_->Update(gv, func);
+ return Call(gv, call->args, call->attrs, call->type_args);
+ }
+ return post;
+ }
+
+ private:
+ /*!
+ * \brief Check if the expr is a function and has the same
+ * compiler name as compiler_name_.
+ *
+ * \param expr The input expr.
+ * \return True if is outlineable else False.
+ */
+ bool CanOutlineExpr(const Expr& expr) {
+ if (!expr->IsInstance<FunctionNode>()) {
+ return false;
+ }
+ Function func = Downcast<Function>(expr);
+ auto compiler = func->GetAttr<String>(attr::kCompiler);
+ if (!compiler.defined()) {
+ return false;
+ }
+ if (compiler != compiler_name_) {
+ return false;
+ }
+ return true;
+ }
+
+ /*! \brief The module that the pass will run on. */
+ IRModule mod_;
+ /*! \brief The name of the compiler to enable outlining on external functions for. */
+ std::string compiler_name_;
+};
+
+/*!
+ * \brief A pass to outline compiler specific functions.
+ */
+tvm::transform::Pass OutlineCompilerFunctions(const std::string& compiler_name) {
+ runtime::TypedPackedFunc<IRModule(IRModule, transform::PassContext)> pass_func =
+ [=](IRModule mod, transform::PassContext ctx) {
+ GlobalVar gv = mod->GetGlobalVar("main");
+ Function main_func = Downcast<Function>(mod->Lookup("main"));
+ auto new_main_body =
+ OutlineCompilerFunctionsMutator(mod, compiler_name).VisitExpr(main_func->body);
+ if (!new_main_body.same_as(main_func->body)) {
+ Function new_main_func = WithFields(main_func, main_func->params, new_main_body);
+ mod->Update(gv, new_main_func);
+ }
+ return mod;
+ };
+ return tvm::transform::CreateModulePass(pass_func, 0,
+ "relay.backend.contrib.uma.OutlineCompilerFunctions", {});
+}
+
+TVM_REGISTER_GLOBAL("relay.ext.uma.OutlineCompilerFunctions")
+ .set_body_typed(OutlineCompilerFunctions);
+
+/*!
+ * \brief This pass will lower NPU functions in a Relay module to scheduled TIR prim functions.
Review Comment:
NPU
##########
src/relay/backend/contrib/uma/relay_to_tir.cc:
##########
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file relay/backend/contrib/uma/codegen.cc
+ *
+ * \brief this file contains the target hooks for the Universal Modular Accelerator Interface (UMA).
+ */
+
+#include <tvm/ir/error.h>
+#include <tvm/relay/analysis.h>
+#include <tvm/relay/attrs/annotation.h>
+#include <tvm/relay/expr.h>
+#include <tvm/relay/expr_functor.h>
+#include <tvm/relay/transform.h>
+#include <tvm/target/target.h>
+#include <tvm/tir/function.h>
+
+#include <unordered_map>
+#include <unordered_set>
+#include <utility>
+#include <vector>
+
+namespace tvm {
+namespace relay {
+namespace contrib {
+namespace uma {
+
+/*!
+ * \brief This mutator outlines functions that are marked with a named
+ * "Compiler" attribute. Functions that do not match this condition remain
+ * unaltered.
+ */
+class OutlineCompilerFunctionsMutator : public MixedModeMutator {
Review Comment:
since we're duplicating it here, i'd prefer if we could factor it out now
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
+import tvm
+
+from tvm.relay.backend.contrib.uma.api import UMAPartitioner
+from tvm.relay.op.contrib.register import get_pattern_table
+from tvm.relay.testing import resnet, mlp
+
+
+def test_partition_table():
+ partitioner = UMAPartitioner("test_partition")
+ assert get_pattern_table("test_partition") is None
+
+ partitioner.register()
+
+ assert get_pattern_table("test_partition") is not None
+
+
+@pytest.mark.parametrize(
+ "workload,backend,merge,expected_partitions",
+ [
+ ("resnet", "dnnl", False, 17),
+ ("resnet", "dnnl", True, 17),
Review Comment:
ah could we either document in a comment where the number 17 came from or derive it in code (can still assert against the number 17, just want to help future test-editors know how to keep this test updated).
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
+ # Target configuration
+ #######################################################################
+ self._register_target_attr("dimension")
+
+ #######################################################################
+ # Relay to Relay function registration
+ #######################################################################
+ self._register_pattern("conv2d", conv2d_pattern())
+
+ #######################################################################
+ # Relay to TIR function registration
+ #######################################################################
+ self._register_tir_pass(PassPhase.TIR_PHASE_0, MyAiHwConv2dPass())
Review Comment:
cc @mbaret one thing we've been discussing is that it might make sense for these enums to roughly correspond to "guarantees about the form of the IRModule" (e.g. "all the primitive Functions are lowered" or "Buffers are defined"). To that end I think it would be great to avoid referencing e.g. TIR_PHASE_n in the enum unless those are distinct guarantees.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r930006872
##########
python/tvm/relay/backend/contrib/uma/_template/passes.py:
##########
@@ -0,0 +1,137 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Transform passes for the my_ai_hw accelerator"""
+
+import tvm
+from tvm import relay, tir
+from tvm.relay.backend.contrib.uma.api.utils import add_llvm_to_block
+
+
+@tvm.tir.transform.prim_func_pass(opt_level=2)
+class MyAiHwConv2dPass:
+ def transform_function(
+ self, func: tvm.tir.PrimFunc, mod: tvm.ir.IRModule, ctx: tvm.ir.transform.PassContext
+ ) -> tvm.tir.PrimFunc:
+ return self._my_ai_hw_conv2d_pass(func, mod, ctx)
+
+ @staticmethod
+ def _my_ai_hw_conv2d_pass(func, mod, ctx):
+ _found_blocks = []
+ _loops = dict()
+ _handles = []
+ _entry_node = None
+ _external_function_name = "my_ai_hw_conv2dnchw"
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
areusch commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1191960637
we discussed this in the [Community Meeting](https://discuss.tvm.apache.org/t/next-tvm-community-meeting-july-20/13148/2) yesterday. here are notes on the discussion:
- when the design references "Accelerator A" and "Accelerator B," does this mean we're using both simultaneously?
- not in this v1, though the architecture supports it. at present they can simply coexist as options.
- should we integrate this with TVMC?
- @areusch: it should be fairly easy to integrate the UMA targets with the `tvmc run` command
- @manupa-arm : this should be pretty straightforward to add to tvmc. the bigger concern here was around `uma_cli.py`, which is supposed to generate a starter implementation for new accelerators in uma.
- @areusch : we should either have tvmc or some other developer-facing entry point to house tools like this. probably not bad to add dev tools to tvmc now--we can always migrate them out if we need to.
- @MichaelJKlaiber : intention of uma_cli is just to make the tutorial easier to replicate on your own, so there are two steps there--create the accelerator flow and then run inference.
- @manupa-arm : do we expect the CLI to work when we're in an environment where only the tvm wheel is present? e.g what about the C sources included with accelerator? should those go in the wheel?
- @MichaelJKlaiber: those sources are copied into the generated dir by uma_cli.
- @areusch : what's the include path folks are expected to set on their downstream C compiler? seems like the C files included with accelerator template should really make it into the Model Library Format. Could produce another CSourceModule which would create another e.g. `default_lib3.cc` in the MLF. Could also use the `import_c` pragma [similar](https://github.com/apache/tvm/blob/main/python/tvm/topi/arm_cpu/mprofile/dsp/micro_kernel/max_pool.py#L87). to how we do for microTVM.
- where should the template live?
- @areusch : could go either way or both. how do we expect people to package their accelerator flow? if merging into mainline, perhaps we want in the python import path. if keeping accelerator flow private, perhaps apps is similar to carrying that code alongside the tvm wheel.
- @manupa-arm : deciding intended location based on whether a flow will get upstreamed makes sense. `_template` is an example rather than a target, so maybe `apps` could make more sense for it.
- @manupa-arm : also suggest to break the CLI changes into another PR.
- @MichaelJKlaiber : only vanilla accelerator was impl'd; do folks have suggestions for chocolate and strawberry? feel free ot post in discuss thread or contact
- @areusch : would be cool to see something that leverages usmp to model physical accelerator memories. could also be cool to see an example where buffers were marked to live on-device.
- Slava: are the optimization provided in the default TVM pipeline also part of the UMA pipeline?
- @areusch : you can classify the optimizations in terms of relay passes, scheduling, and post-scheduling passes. TVM tries to operate on an IRModule-to-IRModule prinicple, where each optimization or step takes an IRModule and returns an IRModule. when you mark a subgraph as offloaded to an UMA pipeline, some optimizations aren't enabled--for example, Relay-level operator fusion. Others e.g. those which operate post-scheduling (usmp, for example) will run on UMA operators.
- Slava: if I have a conv2d followed by batch norm, and only the conv2d is offloaded, then the batch norm is not fused by default?
- @areusch: the right way to do that would be to mark both as offloaded and do the fusion yourself. there are also some efforts to enable post-scheduling fusion via Relax, but those haven't landed yet.
- Slava: what's the best way to leverage UMA if e.g. we have 2 different implementations of conv2d depending on kernel size?
- @areusch : you'd need to give your pattern matcher enough fidelity to differentiate those two workloads. you can also inspect the matched subgraph after using a looser pattern.
- slava: what's the rough timeline?
- not really a timeline, but see https://github.com/apache/tvm/issues/11260
- @MichaelJKlaiber : can also discuss more questions in high-bandwidth with folks.
- suggest folks post up on the discuss forum. we can also use this meeting for further discussion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] cbalint13 commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
cbalint13 commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1188904111
> Thank you for the great contribution for better accelerator supports! partying_face I enjoyed reading the code and learning the design. Overall, implementation looks good to me. Here are a few high-level comments:
>
> * Could you add more documentation in general (e.g., explain what parameters are)?
> * It would be great to have a testing for end-to-end model-level testing. (e.g., run BERT with UMA on ARM EthosU)
>
> Please see below for more comments. Thanks!
> Thank you for the great contribution for better accelerator supports! partying_face I enjoyed reading the code and learning the design. Overall, implementation looks good to me. Here are a few high-level comments:
>
Personal view over the first part (non code related) of @sunggg requests, of course just IMHO:
> * Could you add more documentation in general (e.g., explain what parameters are)?
- Current sample (_template) & doc are fine for a **contrib** module.
- It greatly enables **"out-of-tree"** maintenance towards custom HW accelarator design.
- Explanatory parts **Vanilla**, **Strawberry** & **Chocolate** are nice levels of introductory guide.
> * It would be great to have a testing for end-to-end model-level testing. (e.g., run BERT with UMA on ARM EthosU)
- Sounds beyond of scope, same way we could "dream" a fully verilated BERT running **verilator-contrib** module (!) @tmoreau89 , @vegaluisjose
- As for ARM-Ethos, my understanding is a quite different (not so **"out-of-tree"** principled) way to connect things inside, but @manupa-arm can comment more on this.
- Would be possible in the future to decouple ARM-Ethos (or Vitis-AI) to be more **"out-of-tree"** by porting it over UMA, but this would be quite bit of another task.
Some personal efforts envisioned similar **out-of-tree** approaches {for i.e. [OLIMP](https://github.com/cbalint13/OLIMP) / [e-verest](https://github.com/cbalint13/e-verest) }, and UMA just landed as super nice **"out-of-tree"** topping. As my last wish, I put forward hope that we community one day will be able to leverage jointly the complex hardware-design space within TVM's unique compiler-space.
Beyond code-related requests below, I halt my review here with a @lgtm claim, and a go for UMA v1.0 contrib !
> Please see below for more comments. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939153921
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
+ self._tir_passes: List[Tuple[PassPhase, tvm.tir.transform.PrimFuncPass]] = []
+
+ def _lower_relay_to_tir(self, relay_prim_func: relay.Function) -> tvm.tir.PrimFunc:
+ """Lower a Relay primitive function to a S-TIR primitive function.
+
+ Parameters
+ ----------
+ prim_func : tvm.relay.Function
+ The Relay function to lower.
+
+ Returns
+ -------
+ out : tvm.tir.PrimFunc
+ The lowered schedulable TensorIR primitive function.
+
+ """
+
+ def _get_tensors(te_cached_func):
+ outputs = list(te_cached_func.outputs)
+ stack = []
+ visited = set()
+ for output_ in outputs:
+ if output_ not in visited:
+ visited.add(output_)
+ stack.append(output_)
+
+ args = []
+ while len(stack) != 0:
+ tensor = stack.pop()
+ if isinstance(tensor.op, tvm.te.tensor.PlaceholderOp):
+ args.append(tensor)
+ elif isinstance(tensor.op, tvm.te.tensor.ComputeOp):
+ inputs = tensor.op.input_tensors
+ for input_ in inputs:
+ if input_ not in visited:
+ visited.add(input_)
+ stack.append(input_)
+
+ return args + outputs
+
+ f = tvm._ffi.get_global_func("relay.backend.LowerToTE")
+ te_cached_func = f(relay_prim_func)
+ x = _get_tensors(te_cached_func)
+ tir_prim_func = te.create_prim_func(x)
+ tir_prim_func = tir_prim_func.with_attr(
+ "global_symbol", relay_prim_func.attrs["global_symbol"]
+ )
+ # TODO: The target should probably come from somewhere else instead of being created here.
+ tir_prim_func = tir_prim_func.with_attr("target", tvm.target.Target(self.target_name))
Review Comment:
@cgerum can you comment?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r939146296
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
Review Comment:
I agree with your finding for "includes".
It is optional, therefore I removed the ICHECK from the cc code.
I also agree that it is important, therefore I added a separate parameter instead of kwargs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on PR #12087:
URL: https://github.com/apache/tvm/pull/12087#issuecomment-1206203534
> replace AOTTestRunner by project API @manupa-arm
I am happy AOTTestRunner as we envision that example/test to be useful for a embedded dev who wants to integrate few models to the their own embedded project.
In same time, it might be also valuable to add a Project API for users that quickly want spin up a project to test the inference.
So they both have value and their place.
Therefore, I would say not "replace" but maybe just extend/add ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r923152921
##########
CMakeLists.txt:
##########
@@ -494,6 +494,7 @@ include(cmake/modules/contrib/ArmComputeLib.cmake)
include(cmake/modules/contrib/TensorRT.cmake)
include(cmake/modules/contrib/VitisAI.cmake)
include(cmake/modules/contrib/Verilator.cmake)
+include(cmake/modules/contrib/UMA.cmake)
Review Comment:
CC @manupa-arm @areusch @sunggg @Mousius @lhutton1 @cgerum @PaulPalomeroBernardo
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] MichaelJKlaiber commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
MichaelJKlaiber commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925568107
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA backend for the my_ai_hw accelerator"""
+from .passes import MyAiHwConv2dPass
+from ..api.utils import PassPhase
+from ..backend import UMABackend
+from .codegen import gen_includes, gen_replace_call_extern
+from .patterns import conv2d_pattern
+
+
+class MyAiHwBackend(UMABackend):
+ """UMA backend for the MyAiHw accelerator."""
+
+ def __init__(self):
+ super().__init__()
+
+ #######################################################################
+ # Target configuration
+ #######################################################################
+ self._register_target_attr("dimension")
+
+ #######################################################################
+ # Relay to Relay function registration
Review Comment:
I agree. Comment will be adapted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925689353
##########
python/tvm/relay/backend/contrib/uma/_template/backend.py:
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review Comment:
Given the out-of-tree usecase of UMA, I personally feel this might be better suited as an apps/ example, where we showcase to a "user" how UMA could be used. WDYT ?
Even if its a template, then it would be still good demonstrate how to use that template is used in a "user" flow.
*user here means a out-of-tree dev that wants to attach accelerator codegen to TVM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] lhutton1 commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
lhutton1 commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r924943667
##########
python/tvm/relay/backend/contrib/uma/api/partitioner.py:
##########
@@ -0,0 +1,118 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Partitioner base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import Callable, Dict, List, Tuple, Optional
+
+import tvm
+from tvm import relay
+from tvm.relay.build_module import bind_params_by_name
+from tvm.relay.op.contrib.register import register_pattern_table
+from .utils import PassPhase
+
+
+PatternTable = List[Tuple[str, tvm.relay.dataflow_pattern.DFPattern, Callable]]
+
+
+class UMAPartitioner:
+ """Partitioner base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str, merge_compiler_regions: bool = True) -> None:
+ self.target_name = target_name
+ self.merge_compiler_regions = merge_compiler_regions
+
+ self._relay_passes: List[Tuple[PassPhase, tvm.transform.Pass]] = []
+ self._patterns: PatternTable = []
+
+ def add_pattern(
+ self,
+ name: str,
+ pattern: tvm.relay.dataflow_pattern.DFPattern,
+ predicate: Optional[Callable] = None,
+ ) -> None:
+ """Add pattern to UMA partitioner
+
+ Parameters
+ ----------
+ name : str
+ relay name of pattern
+
+ pattern: tvm.relay.dataflow_pattern.DFPattern
+ pattern description as DFPattern
+
+ predicate: Optional[Callable]
+ Optional predicate
+
+ """
+
+ name = self.target_name + "." + name
+ if predicate:
+ self._patterns.append((name, pattern, predicate))
+ else:
+ self._patterns.append((name, pattern))
+
+ def _pattern_table(self) -> PatternTable:
+ return self._patterns
+
+ def register(self) -> None:
+ """Register all relevant relay-to-relay functions."""
+ register_pattern_table(self.target_name, self._pattern_table)
+
+ def partition(
+ self, mod: tvm.IRModule, params: Optional[Dict[str, tvm.runtime.NDArray]] = None
+ ) -> tvm.IRModule:
+ """Partition the relay graph in by the NPU supported and unsupported parts.
+
+ Parameters
+ ----------
+ mod : tvm.IRModule
+ The relay module to be partitioned.
+
+ params: Optional[Dict[str, tvm.runtime.NDArray]]
+
+ Returns
+ -------
+ out : tvm.IRModule
+ The partitioned relay module.
+
+ """
+ if params:
+ mod["main"] = bind_params_by_name(mod["main"], params)
+
+ mod = relay.transform.InferType()(mod)
Review Comment:
I wonder if using a single `Sequential` block might make things a bit clearer here. e.g.
```
seq = []
seq.extend([p[1] for p in self._relay_passes if p[0] == PassPhase.PRE_PARTITIONING])
seq.append(relay.transform.MergeComposite(self._pattern_table()))
....
mod = seq(mod)
```
Further, it'll reduce the need to explicitly run `InferType` and manage other pass dependencies
##########
python/tvm/relay/backend/contrib/uma/api/lower.py:
##########
@@ -0,0 +1,159 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Lowering base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from typing import List, Tuple, Callable, Optional
+
+import tvm
+from tvm import relay, te
+from tvm.relay.op.op import register_strategy
+from . import _ffi_api
+from .utils import PassPhase
+
+
+class UMALower:
+ """Lowering base class of the Universal Modular Accelerator Interface (UMA)."""
+
+ def __init__(self, target_name: str) -> None:
+ self.target_name = target_name
+
+ self._operator_strategies: List[
+ Tuple[
+ str,
+ Callable[
+ [tvm.ir.Attrs, tvm.ir.Array, tvm.ir.TensorType, tvm.target.Target],
+ tvm.relay.op.op.OpStrategy,
+ ],
+ Optional[int],
+ ]
+ ] = []
Review Comment:
Would it make sense to break these type annotations into smaller named variables? e.g. like `PatternTable` in partitioner.py
##########
python/tvm/relay/backend/contrib/uma/backend.py:
##########
@@ -0,0 +1,299 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Backend base class of the Universal Modular Accelerator Interface (UMA)"""
+
+from abc import ABC, abstractmethod
+from typing import Union, Dict, Callable, Optional, Any
+
+import tvm
+from tvm.relay.backend.contrib.uma.api.codegen import UMACodegen
+from tvm.relay.backend.contrib.uma.api.lower import UMALower
+from tvm.relay.backend.contrib.uma.api.partitioner import UMAPartitioner
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+
+
+class UMABackend(ABC):
+ """Backend base class of the Universal Modular Accelerator Interface (UMA)"""
+
+ def __init__(self, merge_compiler_regions: bool = True) -> None:
+ self._target_attrs: Dict = {}
+ self._target_preprocessor: Callable[[str], Dict[str, Any]] = None
+ self._relay_to_relay = UMAPartitioner(self.target_name, merge_compiler_regions)
+ self._relay_to_tir = UMALower(self.target_name)
+ self._tir_to_runtime = UMACodegen(self.target_name)
+
+ @property
+ @abstractmethod
+ def target_name(self) -> str:
+ """Name of the hardware target.
+
+ Returns
+ -------
+ out : str
+ The hardware target name.
+ """
+ ...
+
+ ############################################################################
+ # Target configuration
+ ############################################################################
+ def _register_target_attr(
+ self,
+ name: str,
+ default: Optional[Union[str, int, bool]] = "",
+ ) -> None:
+ """Register a target attribute name that can be used during target instantiation.
+ Parameters
+ ----------
+ name: str
+ The name of the target attribute.
+
+ default: Optional[Union[str, int, bool]]
+ A default value for the attribute.
+ If none is provided, the attribute will be treated as a string.
+
+ Example
+ -------
+ Here is an example of how two attribute options are registered.
+
+ .. code-block:: python
+
+ self._register_target_attr("attrA", default=0)
+ self._register_target_attr("attrB", default=False)
+ """
+ self._target_attrs[name] = default
+
+ ############################################################################
+ # Relay to Relay function registration
+ ############################################################################
+ def _register_relay_pass(self, phase: PassPhase, relay_pass: tvm.transform.Pass) -> None:
+ """Registers a relay pass at the given phase in the lowering process.
+
+ Parameters
+ ----------
+ phase: PassPhase
+ The phase at which the pass is registered.
+
+ relay_pass: tvm.transform.Pass
+ The relay pass to be registered.
+
+ Example
+ -------
+ Here is an example of how two relay passes are registered.
+ Passes of the same phase are executed in the order they are registered.
+
+ .. code-block:: python
+
+ self._register_relay_pass(PassPhase.PRE_PARTITIONING, MyPassA)
+ self._register_relay_pass(PassPhase.POST_PARTITIONING, MyPassB)
+
+ Where a relay pass can look like this:
+
+ .. code-block:: python
+
+ @tvm.ir.transform.module_pass(opt_level=0)
+ class MyPassA:
+ def transform_module(self, mod, ctx):
+ # My pass functionality...
+ return mod
+ """
+ self._relay_to_relay._relay_passes.append((phase, relay_pass))
+
+ def _register_pattern(
+ self,
+ name: str,
+ pattern: tvm.relay.dataflow_pattern.DFPattern,
+ predicate: Optional[Callable] = None,
+ ) -> None:
+ """Registers a dataflow pattern that is used to partition the relay graph.
+
+ Parameters
+ ----------
+ name: str
+ The name of the pattern.
+
+ pattern: tvm.relay.dataflow_pattern.DFPattern
+ The dataflow pattern.
+
+ predicate: Callable Receiving the matched pattern and
Review Comment:
nit: missing part of sentence?
##########
tests/scripts/task_python_uma.sh:
##########
@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euxo pipefail
+
+source tests/scripts/setup-pytest-env.sh
+
+run_pytest ctypes test_uma tests/python/contrib/test_uma
+run_pytest cython3 test_uma tests/python/contrib/test_uma
Review Comment:
Curious if there is a need to run these tests separately? I believe they should already get run as part of `task_python_integration` (https://github.com/apache/tvm/blob/main/tests/scripts/task_python_integration.sh#L64)?
##########
tests/python/contrib/test_uma/test_uma_vanilla_accelerator.py:
##########
@@ -0,0 +1,61 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""UMA testcase for the vanilla_accelerator accelerator"""
+import pytest
+
+import tvm
+from tvm import tir
+from tvm.relay.dataflow_pattern import is_op, wildcard
+from tvm.relay.backend.contrib.uma.api.utils import PassPhase
+from tvm.relay.backend.contrib.uma.backend import UMABackend
+from tvm.relay.backend.contrib.uma._template.passes import (
+ MyAiHwConv2dPass as VanillaAcceleratorConv2dPass,
+)
+from tvm.relay.backend.contrib.uma._template.codegen import gen_includes
+
+from tvm.relay.backend.contrib.uma._template.patterns import conv2d_pattern
+
+# def conv2d_pattern():
+# pattern = is_op("nn.conv2d")(wildcard(), wildcard())
+# pattern = pattern.has_attr({"strides": [1, 1]})
+# return pattern
Review Comment:
nit: remove
##########
tests/python/contrib/test_uma/test_partition.py:
##########
@@ -0,0 +1,71 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+import pytest
+
Review Comment:
Should the tests be guarded in some way based on whether or not TVM was compiled with UMA support?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] sunggg commented on a diff in pull request #12087: [UMA] UMA v1.0
Posted by GitBox <gi...@apache.org>.
sunggg commented on code in PR #12087:
URL: https://github.com/apache/tvm/pull/12087#discussion_r925793720
##########
src/relay/backend/contrib/uma/tir_to_runtime.cc:
##########
@@ -0,0 +1,104 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+#include <cmath>
+#include <fstream>
+#include <map>
+#include <sstream>
+#include <string>
+#include <vector>
+
+#include "../../../../runtime/file_utils.h"
+#include "../../../../target/source/codegen_c.h"
+#include "../../../../target/source/codegen_c_host.h"
+
+namespace tvm {
+using namespace tir;
+namespace relay {
+namespace contrib {
+namespace uma {
+
+class UMACodegen : public codegen::CodeGenCHost {
+ public:
+ explicit UMACodegen(String target_str) : target_str_(target_str) {}
+
+ void Init(bool output_ssa, bool emit_asserts) {
+ auto includes_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_includes_" + target_str_);
+ ICHECK(includes_pf);
+ String includes = (*includes_pf)();
+ decl_stream << includes;
+ std::unordered_set<std::string> devices;
+ devices.insert(target_str_);
+ CodeGenCHost::Init(output_ssa, emit_asserts, target_str_, devices);
+ }
+
+ /*!
+ * \brief Emit code that offloads a subgraph to the UMA target
+ *
+ * \return string of code that offloads a subgraph to the UMA target
+ */
+ void AddFunction(const PrimFunc& prim_func) { CodeGenC::AddFunction(prim_func); }
+
+ private:
+ String target_str_;
+
+ using codegen::CodeGenCHost::VisitStmt_;
+
+ /*! * \brief Emits target specific APIs for every call_extern */
+ void VisitExpr_(const CallNode* op, std::ostream& os) final {
+ if (!op->op.same_as(builtin::call_extern())) {
+ CodeGenCHost::VisitExpr_(op, os);
+ return;
+ }
+ auto replace_call_extern_pf =
+ tvm::runtime::Registry::Get("relay.ext.uma.codegen_c_replace_call_extern_" + target_str_);
+ if (replace_call_extern_pf == nullptr) {
+ CodeGenCHost::VisitExpr_(op, os);
+ } else {
+ // - funtion type (void) still gets printed before CallNode if extern call is wrapped in
+ // EvaluateNode
+ // - VarNode arguments might have "wrong" name_hints. The correct variable name is determined
+ // in C++ through GetVarID
+ String api_string = (*replace_call_extern_pf)(op->args);
+ os << api_string;
+ }
+ return;
+ }
+};
+
+runtime::Module TIRToRuntime(IRModule mod, Target target) {
+ bool output_ssa = false;
+ bool emit_asserts = false;
+ UMACodegen codegen(target->kind->name);
+ Array<String> function_names;
+ codegen.Init(output_ssa, emit_asserts);
+ for (auto kv : mod->functions) {
+ auto prim_func = Downcast<PrimFunc>(kv.second);
+ auto global_symbol = prim_func->GetAttr<String>(tvm::attr::kGlobalSymbol);
+ function_names.push_back(global_symbol.value());
+ codegen.AddFunction(prim_func);
+ }
+ std::string code = codegen.Finish();
+ return codegen::CSourceModuleCreate(code, "c", function_names);
Review Comment:
@MichaelJKlaiber Awesome. I will look forward to seeing the Strawberry example then!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org