You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by tq...@apache.org on 2024/02/15 16:36:33 UTC
(tvm-site) branch asf-site updated: deploying docs (apache/tvm@1a01102f876adec6f66c4a210e688601728fcd5a)
This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/tvm-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 85aecd2450 deploying docs (apache/tvm@1a01102f876adec6f66c4a210e688601728fcd5a)
85aecd2450 is described below
commit 85aecd2450f8c5780e5ea962259f462086104518
Author: tvm-bot <95...@users.noreply.github.com>
AuthorDate: Thu Feb 15 16:36:25 2024 +0000
deploying docs (apache/tvm@1a01102f876adec6f66c4a210e688601728fcd5a)
---
.../tune_relay_cuda.py | 11 -
.../0e2f38fcb1a1fb3e636e5953aa600dee/from_mxnet.py | 153 ----
.../tune_alu_vta.ipynb | 126 ---
.../tune_network_arm.py | 23 +-
.../tune_relay_arm.py | 11 -
.../deploy_model_on_nano.py | 30 +-
.../deploy_model_on_rasp.py | 32 +-
.../from_mxnet.ipynb | 180 ----
.../tune_network_arm.ipynb | 4 +-
.../tune_network_mali.ipynb | 4 +-
.../tune_relay_mobile_gpu.py | 11 -
.../tune_network_mali.py | 19 +-
.../tune_relay_x86.py | 11 -
.../tune_relay_mobile_gpu.ipynb | 2 +-
.../tune_alu_vta.py | 345 --------
.../deploy_quantized.py | 172 ----
.../deploy_model_on_rasp.ipynb | 10 +-
.../tune_relay_x86.ipynb | 2 +-
.../deploy_classification.ipynb | 133 ---
.../deploy_classification.py | 313 -------
.../deploy_quantized.ipynb | 144 ---
.../tune_network_x86.ipynb | 4 +-
.../tune_network_cuda.ipynb | 4 +-
.../bring_your_own_datatypes.ipynb | 349 --------
.../tune_relay_vta.ipynb | 186 ----
.../tune_relay_arm.ipynb | 2 +-
.../deploy_model_on_nano.ipynb | 8 +-
.../tune_relay_cuda.ipynb | 2 +-
.../tune_relay_vta.py | 538 ------------
.../tune_network_x86.py | 31 +-
.../tune_network_cuda.py | 20 +-
.../bring_your_own_datatypes.py | 415 ---------
.../sphx_glr_bring_your_own_datatypes_thumb.png | Bin 26794 -> 0 bytes
.../_images/sphx_glr_deploy_classification_001.png | Bin 245200 -> 0 bytes
.../sphx_glr_deploy_classification_thumb.png | Bin 104799 -> 0 bytes
docs/_images/sphx_glr_deploy_quantized_thumb.png | Bin 26794 -> 0 bytes
docs/_images/sphx_glr_from_mxnet_001.png | Bin 245200 -> 0 bytes
docs/_images/sphx_glr_from_mxnet_thumb.png | Bin 104799 -> 0 bytes
docs/_images/sphx_glr_tune_alu_vta_thumb.png | Bin 26794 -> 0 bytes
docs/_images/sphx_glr_tune_relay_vta_thumb.png | Bin 26794 -> 0 bytes
.../how_to/compile_models/from_darknet.rst.txt | 2 +-
.../how_to/compile_models/from_mxnet.rst.txt | 285 ------
.../how_to/compile_models/from_oneflow.rst.txt | 2 +-
.../how_to/compile_models/from_paddle.rst.txt | 5 -
.../how_to/compile_models/from_pytorch.rst.txt | 2 +-
.../how_to/compile_models/from_tensorflow.rst.txt | 2 +-
docs/_sources/how_to/compile_models/index.rst.txt | 18 -
.../compile_models/sg_execution_times.rst.txt | 22 +-
.../deploy_models/deploy_model_on_adreno.rst.txt | 4 +-
.../deploy_model_on_adreno_tvmc.rst.txt | 2 +-
.../deploy_models/deploy_model_on_android.rst.txt | 2 +-
.../deploy_models/deploy_model_on_nano.rst.txt | 54 +-
.../deploy_models/deploy_model_on_rasp.rst.txt | 56 +-
.../deploy_object_detection_pytorch.rst.txt | 4 +-
.../deploy_models/deploy_prequantized.rst.txt | 6 +-
.../deploy_prequantized_tflite.rst.txt | 2 +-
.../how_to/deploy_models/deploy_quantized.rst.txt | 289 ------
docs/_sources/how_to/deploy_models/index.rst.txt | 18 -
.../deploy_models/sg_execution_times.rst.txt | 22 +-
.../extend_tvm/bring_your_own_datatypes.rst.txt | 735 ----------------
docs/_sources/how_to/extend_tvm/index.rst.txt | 18 -
.../how_to/extend_tvm/sg_execution_times.rst.txt | 18 +-
.../how_to/extend_tvm/use_pass_instrument.rst.txt | 16 +-
.../optimize_operators/opt_conv_cuda.rst.txt | 2 +-
.../optimize_operators/opt_conv_tensorcore.rst.txt | 2 +-
.../how_to/optimize_operators/opt_gemm.rst.txt | 16 +-
.../optimize_operators/sg_execution_times.rst.txt | 8 +-
.../sg_execution_times.rst.txt | 14 +-
.../tune_conv2d_layer_cuda.rst.txt | 2 +-
.../tune_network_arm.rst.txt | 47 +-
.../tune_network_cuda.rst.txt | 44 +-
.../tune_network_mali.rst.txt | 41 +-
.../tune_network_x86.rst.txt | 55 +-
.../tune_with_autotvm/sg_execution_times.rst.txt | 8 +-
.../tune_with_autotvm/tune_conv2d_cuda.rst.txt | 2 +-
.../tune_with_autotvm/tune_relay_arm.rst.txt | 35 +-
.../tune_with_autotvm/tune_relay_cuda.rst.txt | 37 +-
.../tune_relay_mobile_gpu.rst.txt | 37 +-
.../tune_with_autotvm/tune_relay_x86.rst.txt | 23 +-
.../work_with_microtvm/micro_autotune.rst.txt | 18 +-
.../work_with_microtvm/micro_pytorch.rst.txt | 4 +-
.../how_to/work_with_microtvm/micro_train.rst.txt | 16 +-
.../work_with_microtvm/sg_execution_times.rst.txt | 14 +-
.../work_with_relay/sg_execution_times.rst.txt | 10 +-
.../how_to/work_with_schedules/intrin_math.rst.txt | 2 +-
.../work_with_schedules/sg_execution_times.rst.txt | 18 +-
.../topic/vta/tutorials/autotvm/index.rst.txt | 59 --
.../tutorials/autotvm/sg_execution_times.rst.txt | 14 -
.../vta/tutorials/autotvm/tune_alu_vta.rst.txt | 443 ----------
.../vta/tutorials/autotvm/tune_relay_vta.rst.txt | 666 --------------
.../frontend/deploy_classification.rst.txt | 442 ----------
.../tutorials/frontend/deploy_detection.rst.txt | 7 +-
.../topic/vta/tutorials/frontend/index.rst.txt | 18 -
.../tutorials/frontend/sg_execution_times.rst.txt | 10 +-
docs/_sources/topic/vta/tutorials/index.rst.txt | 66 --
.../tutorials/optimize/sg_execution_times.rst.txt | 6 +-
.../topic/vta/tutorials/sg_execution_times.rst.txt | 6 +-
.../tutorial/auto_scheduler_matmul_x86.rst.txt | 4 +-
docs/_sources/tutorial/autotvm_matmul_x86.rst.txt | 20 +-
docs/_sources/tutorial/autotvm_relay_x86.rst.txt | 57 +-
.../tutorial/cross_compilation_and_rpc.rst.txt | 2 +-
docs/_sources/tutorial/intro_topi.rst.txt | 2 +-
docs/_sources/tutorial/sg_execution_times.rst.txt | 18 +-
.../tutorial/tensor_expr_get_started.rst.txt | 49 +-
docs/commit_hash | 2 +-
docs/how_to/compile_models/from_coreml.html | 1 -
docs/how_to/compile_models/from_darknet.html | 3 +-
docs/how_to/compile_models/from_keras.html | 1 -
docs/how_to/compile_models/from_mxnet.html | 623 -------------
docs/how_to/compile_models/from_oneflow.html | 20 +-
docs/how_to/compile_models/from_onnx.html | 5 +-
docs/how_to/compile_models/from_paddle.html | 2 -
docs/how_to/compile_models/from_pytorch.html | 18 +-
docs/how_to/compile_models/from_tensorflow.html | 7 +-
docs/how_to/compile_models/from_tflite.html | 1 -
docs/how_to/compile_models/index.html | 4 -
docs/how_to/compile_models/sg_execution_times.html | 32 +-
.../deploy_models/deploy_model_on_adreno.html | 4 +-
.../deploy_models/deploy_model_on_adreno_tvmc.html | 37 +-
.../deploy_models/deploy_model_on_android.html | 2 +-
.../how_to/deploy_models/deploy_model_on_nano.html | 44 +-
.../how_to/deploy_models/deploy_model_on_rasp.html | 46 +-
.../deploy_object_detection_pytorch.html | 61 +-
docs/how_to/deploy_models/deploy_prequantized.html | 10 +-
.../deploy_models/deploy_prequantized_tflite.html | 6 +-
docs/how_to/deploy_models/deploy_quantized.html | 634 --------------
docs/how_to/deploy_models/deploy_sparse.html | 4 +-
docs/how_to/deploy_models/index.html | 3 -
docs/how_to/deploy_models/sg_execution_times.html | 38 +-
.../extend_tvm/bring_your_own_datatypes.html | 886 -------------------
docs/how_to/extend_tvm/index.html | 4 -
docs/how_to/extend_tvm/low_level_custom_pass.html | 1 -
docs/how_to/extend_tvm/sg_execution_times.html | 20 +-
docs/how_to/extend_tvm/use_pass_infra.html | 1 -
docs/how_to/extend_tvm/use_pass_instrument.html | 21 +-
docs/how_to/optimize_operators/opt_conv_cuda.html | 2 +-
.../optimize_operators/opt_conv_tensorcore.html | 2 +-
docs/how_to/optimize_operators/opt_gemm.html | 16 +-
.../optimize_operators/sg_execution_times.html | 8 +-
docs/how_to/profile/index.html | 4 +-
.../sg_execution_times.html | 18 +-
.../tune_conv2d_layer_cuda.html | 2 +-
.../tune_with_autoscheduler/tune_network_arm.html | 23 +-
.../tune_with_autoscheduler/tune_network_cuda.html | 24 +-
.../tune_with_autoscheduler/tune_network_mali.html | 19 +-
.../tune_with_autoscheduler/tune_network_x86.html | 35 +-
.../tune_with_autotvm/sg_execution_times.html | 8 +-
.../how_to/tune_with_autotvm/tune_conv2d_cuda.html | 2 +-
docs/how_to/tune_with_autotvm/tune_relay_arm.html | 11 -
docs/how_to/tune_with_autotvm/tune_relay_cuda.html | 11 -
.../tune_with_autotvm/tune_relay_mobile_gpu.html | 11 -
docs/how_to/tune_with_autotvm/tune_relay_x86.html | 11 -
docs/how_to/work_with_microtvm/micro_autotune.html | 18 +-
docs/how_to/work_with_microtvm/micro_pytorch.html | 6 +-
docs/how_to/work_with_microtvm/micro_train.html | 16 +-
.../work_with_microtvm/sg_execution_times.html | 14 +-
.../how_to/work_with_relay/sg_execution_times.html | 10 +-
docs/how_to/work_with_schedules/intrin_math.html | 2 +-
.../work_with_schedules/sg_execution_times.html | 18 +-
docs/install/nnpack.html | 12 +-
docs/objects.inv | Bin 26195 -> 25652 bytes
docs/reference/api/python/auto_scheduler.html | 4 +-
.../api/typedoc/classes/ArtifactCache.html | 8 +-
docs/reference/api/typedoc/classes/DLDataType.html | 14 +-
docs/reference/api/typedoc/classes/DLDevice.html | 12 +-
docs/reference/api/typedoc/classes/Instance.html | 104 +--
docs/reference/api/typedoc/classes/Module.html | 12 +-
docs/reference/api/typedoc/classes/NDArray.html | 30 +-
docs/reference/api/typedoc/classes/RPCServer.html | 16 +-
docs/reference/api/typedoc/classes/Scalar.html | 8 +-
docs/reference/api/typedoc/classes/TVMArray.html | 16 +-
docs/reference/api/typedoc/classes/TVMObject.html | 12 +-
.../api/typedoc/classes/VirtualMachine.html | 10 +-
.../classes/_internal_.CachedCallStack.html | 36 +-
.../classes/_internal_.CanvasRenderManager.html | 10 +-
.../typedoc/classes/_internal_.Environment.html | 14 +-
.../api/typedoc/classes/_internal_.FFILibrary.html | 22 +-
.../api/typedoc/classes/_internal_.Memory.html | 36 +-
.../typedoc/classes/_internal_.PackedFuncCell.html | 8 +-
.../typedoc/classes/_internal_.RuntimeContext.html | 54 +-
.../api/typedoc/classes/_internal_.TVMString.html | 14 +-
.../typedoc/classes/_internal_.WebGPUContext.html | 28 +-
.../typedoc/enums/_internal_.RPCServerState.html | 14 +-
docs/reference/api/typedoc/functions/assert.html | 2 +-
.../api/typedoc/functions/createPolyfillWASI.html | 2 +-
.../api/typedoc/functions/detectGPUDevice.html | 2 +-
.../api/typedoc/functions/hasNDArrayInCache.html | 2 +-
.../api/typedoc/functions/instantiate.html | 2 +-
docs/reference/api/typedoc/functions/wasmPath.html | 2 +-
.../api/typedoc/interfaces/Disposable.html | 4 +-
.../typedoc/interfaces/GPUDeviceDetectOutput.html | 8 +-
.../api/typedoc/interfaces/InitProgressReport.html | 10 +-
.../api/typedoc/interfaces/LibraryProvider.html | 6 +-
.../interfaces/_internal_.FunctionInfo.html | 8 +-
.../interfaces/_internal_.NDArrayCacheEntry.html | 14 +-
.../interfaces/_internal_.NDArrayShardEntry.html | 10 +-
.../api/typedoc/types/InitProgressCallback.html | 2 +-
docs/reference/api/typedoc/types/PackedFunc.html | 2 +-
.../types/_internal_.FObjectConstructor.html | 2 +-
.../types/_internal_.FTVMWasmAllocSpace.html | 2 +-
.../types/_internal_.FTVMWasmFreeSpace.html | 2 +-
.../types/_internal_.FTVMWasmPackedCFunc.html | 2 +-
.../api/typedoc/types/_internal_.Pointer.html | 2 +-
.../typedoc/types/_internal_.TVMObjectBase.html | 2 +-
docs/reference/langref/index.html | 4 +-
docs/searchindex.js | 2 +-
docs/topic/vta/tutorials/autotvm/index.html | 467 ----------
.../vta/tutorials/autotvm/sg_execution_times.html | 444 ----------
docs/topic/vta/tutorials/autotvm/tune_alu_vta.html | 802 -----------------
.../vta/tutorials/autotvm/tune_relay_vta.html | 970 ---------------------
.../tutorials/frontend/deploy_classification.html | 790 -----------------
.../vta/tutorials/frontend/deploy_detection.html | 11 +-
docs/topic/vta/tutorials/frontend/index.html | 13 +-
.../vta/tutorials/frontend/sg_execution_times.html | 12 +-
docs/topic/vta/tutorials/index.html | 18 +-
docs/topic/vta/tutorials/matrix_multiply.html | 1 -
.../vta/tutorials/optimize/convolution_opt.html | 4 +-
docs/topic/vta/tutorials/optimize/index.html | 4 +-
.../tutorials/optimize/matrix_multiply_opt.html | 8 +-
.../vta/tutorials/optimize/sg_execution_times.html | 6 +-
docs/topic/vta/tutorials/sg_execution_times.html | 6 +-
docs/topic/vta/tutorials/vta_get_started.html | 1 -
docs/tutorial/auto_scheduler_matmul_x86.html | 4 +-
docs/tutorial/autotvm_matmul_x86.html | 20 +-
docs/tutorial/autotvm_relay_x86.html | 269 +++---
docs/tutorial/cross_compilation_and_rpc.html | 2 +-
docs/tutorial/intro_topi.html | 2 +-
docs/tutorial/sg_execution_times.html | 18 +-
docs/tutorial/tensor_expr_get_started.html | 45 +-
229 files changed, 1220 insertions(+), 13340 deletions(-)
diff --git a/docs/_downloads/0387f07dee851b2b8c6b73e3e88c3140/tune_relay_cuda.py b/docs/_downloads/0387f07dee851b2b8c6b73e3e88c3140/tune_relay_cuda.py
index 47ea99884d..53d56c709d 100644
--- a/docs/_downloads/0387f07dee851b2b8c6b73e3e88c3140/tune_relay_cuda.py
+++ b/docs/_downloads/0387f07dee851b2b8c6b73e3e88c3140/tune_relay_cuda.py
@@ -104,17 +104,6 @@ def get_network(name, batch_size):
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
diff --git a/docs/_downloads/0e2f38fcb1a1fb3e636e5953aa600dee/from_mxnet.py b/docs/_downloads/0e2f38fcb1a1fb3e636e5953aa600dee/from_mxnet.py
deleted file mode 100644
index 132f098d92..0000000000
--- a/docs/_downloads/0e2f38fcb1a1fb3e636e5953aa600dee/from_mxnet.py
+++ /dev/null
@@ -1,153 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-.. _tutorial-from-mxnet:
-
-Compile MXNet Models
-====================
-**Author**: `Joshua Z. Zhang <https://zhreshold.github.io/>`_, \
- `Kazutaka Morita <https://github.com/kazum>`_
-
-This article is an introductory tutorial to deploy mxnet models with Relay. To begin, we must install `mxnet`:
-
-.. code-block:: bash
-
- %%shell
- pip install mxnet
-
-or please refer to official installation guide.
-https://mxnet.apache.org/versions/master/install/index.html
-"""
-
-# some standard imports
-# sphinx_gallery_start_ignore
-# sphinx_gallery_requires_cuda = True
-# sphinx_gallery_end_ignore
-import sys
-import mxnet as mx
-import tvm
-import tvm.relay as relay
-import numpy as np
-
-######################################################################
-# Download Resnet18 model from Gluon Model Zoo
-# ---------------------------------------------
-# In this section, we download a pretrained imagenet model and classify an image.
-from tvm.contrib.download import download_testdata
-from mxnet.gluon.model_zoo.vision import get_model
-from PIL import Image
-from matplotlib import pyplot as plt
-
-try:
- block = get_model("resnet18_v1", pretrained=True)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
-img_url = "https://github.com/dmlc/mxnet.js/blob/main/data/cat.png?raw=true"
-img_name = "cat.png"
-synset_url = "".join(
- [
- "https://gist.githubusercontent.com/zhreshold/",
- "4d0b62f3d01426887599d4f7ede23ee5/raw/",
- "596b27d23537e5a1b5751d2b0481ef172f58b539/",
- "imagenet1000_clsid_to_human.txt",
- ]
-)
-synset_name = "imagenet1000_clsid_to_human.txt"
-img_path = download_testdata(img_url, "cat.png", module="data")
-synset_path = download_testdata(synset_url, synset_name, module="data")
-with open(synset_path) as f:
- synset = eval(f.read())
-image = Image.open(img_path).resize((224, 224))
-plt.imshow(image)
-plt.show()
-
-
-def transform_image(image):
- image = np.array(image) - np.array([123.0, 117.0, 104.0])
- image /= np.array([58.395, 57.12, 57.375])
- image = image.transpose((2, 0, 1))
- image = image[np.newaxis, :]
- return image
-
-
-x = transform_image(image)
-print("x", x.shape)
-
-######################################################################
-# Compile the Graph
-# -----------------
-# Now we would like to port the Gluon model to a portable computational graph.
-# It's as easy as several lines.
-# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
-shape_dict = {"data": x.shape}
-mod, params = relay.frontend.from_mxnet(block, shape_dict)
-## we want a probability so add a softmax operator
-func = mod["main"]
-func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
-
-######################################################################
-# now compile the graph
-target = "cuda"
-with tvm.transform.PassContext(opt_level=3):
- lib = relay.build(func, target, params=params)
-
-######################################################################
-# Execute the portable graph on TVM
-# ---------------------------------
-# Now, we would like to reproduce the same forward computation using TVM.
-from tvm.contrib import graph_executor
-
-dev = tvm.cuda(0)
-dtype = "float32"
-m = graph_executor.GraphModule(lib["default"](dev))
-# set inputs
-m.set_input("data", tvm.nd.array(x.astype(dtype)))
-# execute
-m.run()
-# get outputs
-tvm_output = m.get_output(0)
-top1 = np.argmax(tvm_output.numpy()[0])
-print("TVM prediction top-1:", top1, synset[top1])
-
-######################################################################
-# Use MXNet symbol with pretrained weights
-# ----------------------------------------
-# MXNet often use `arg_params` and `aux_params` to store network parameters
-# separately, here we show how to use these weights with existing API
-def block2symbol(block):
- data = mx.sym.Variable("data")
- sym = block(data)
- args = {}
- auxs = {}
- for k, v in block.collect_params().items():
- args[k] = mx.nd.array(v.data().asnumpy())
- return sym, args, auxs
-
-
-mx_sym, args, auxs = block2symbol(block)
-# usually we would save/load it as checkpoint
-mx.model.save_checkpoint("resnet18_v1", 0, mx_sym, args, auxs)
-# there are 'resnet18_v1-0000.params' and 'resnet18_v1-symbol.json' on disk
-
-######################################################################
-# for a normal mxnet model, we start from here
-mx_sym, args, auxs = mx.model.load_checkpoint("resnet18_v1", 0)
-# now we use the same API to get Relay computation graph
-mod, relay_params = relay.frontend.from_mxnet(mx_sym, shape_dict, arg_params=args, aux_params=auxs)
-# repeat the same steps to run this model using TVM
diff --git a/docs/_downloads/178b6f23dffc01ac92f2cf95f41a5679/tune_alu_vta.ipynb b/docs/_downloads/178b6f23dffc01ac92f2cf95f41a5679/tune_alu_vta.ipynb
deleted file mode 100644
index 6301a89dc4..0000000000
--- a/docs/_downloads/178b6f23dffc01ac92f2cf95f41a5679/tune_alu_vta.ipynb
+++ /dev/null
@@ -1,126 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# Auto-tuning a ALU fused op on VTA\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import os\nimport sys\nfrom mxnet.gluon.model_zoo import vision\nimport numpy as np\nfrom PIL import Image\n\nfrom tvm import topi\nimport tvm\nfrom tvm import te\nfrom tvm import rpc, autotvm, relay\nfrom tvm.contrib import download\nfrom tvm.autotvm.measure.measure_methods import request_remote\nfrom tvm.autotvm.tuner import XGBTuner, GATuner, RandomTuner, GridSearchTuner\nfrom tvm.autotvm import record\n\nimport vta\nfrom vta.testing import simulator\nfrom vta.top import grap [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Compile network\nPerform vta-specific compilation with Relay from a Gluon model\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def compile_network(env, target, model, start_pack, stop_pack):\n\n # Populate the shape and data type dictionary\n dtype_dict = {\"data\": \"float32\"}\n shape_dict = {\"data\": (env.BATCH, 3, 224, 224)}\n\n # Get off the shelf gluon model, and convert to relay\n gluon_model = vision.get_model(model, pretrained=True)\n mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)\n\n # Update shape and type dictionary\n shape_dict.update({k: v.shape f [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "# Set Tuning Options\nBefore tuning, we should apply some configurations.\nHere we use an Pynq-Z1 board as an example.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Tracker host and port can be set by your environment\ntracker_host = os.environ.get(\"TVM_TRACKER_HOST\", \"0.0.0.0\")\ntracker_port = int(os.environ.get(\"TVM_TRACKER_PORT\", 9190))\n\n# Load VTA parameters from the vta/config/vta_config.json file\nenv = vta.get_env()\n\n# This target is used for cross compilation. You can query it by :code:`gcc -v` on your device.\n# Set ``device=arm_cpu`` to run inference on the CPU\n# or ``device=vta`` to run inference on the FPGA.\ndevice [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Register VTA-specific tuning tasks\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def register_vta_tuning_tasks():\n from tvm.autotvm.task import TaskExtractEnv\n\n @tvm.te.tag_scope(tag=topi.tag.ELEMWISE)\n def my_clip(x, a_min, a_max):\n \"\"\"Unlike topi's current clip, put min and max into two stages.\"\"\"\n const_min = tvm.tir.const(a_min, x.dtype)\n const_max = tvm.tir.const(a_max, x.dtype)\n x = te.compute(x.shape, lambda *i: tvm.te.min(x(*i), const_max), name=\"clipA\")\n x = te.compute(x.shape, lambda *i: [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Finally, we launch tuning jobs and evaluate the end-to-end performance.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def tune_and_evaluate(tuning_opt):\n\n if env.TARGET != \"intelfocl\":\n print(\"ALU only op only available for intelfocl target\")\n return\n\n # Register VTA tuning tasks\n register_vta_tuning_tasks()\n\n # Perform task extraction on Relay program\n print(\"Extract tasks...\")\n relay_prog, params = compile_network(env, target, network, start_pack, stop_pack)\n mod = tvm.IRModule.from_expr(relay_prog)\n tasks = autotvm.task.extract_from_pr [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.18"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py b/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py
index 0b59038f19..d795c3aba2 100644
--- a/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py
+++ b/docs/_downloads/17b139d609f9480c7eeda2da1f90f28c/tune_network_arm.py
@@ -120,19 +120,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32", use_sparse=Fal
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet50_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
elif name == "mlp":
mod, params = relay.testing.mlp.get_workload(
batch_size=batch_size, dtype=dtype, image_shape=image_shape, num_classes=1000
@@ -265,13 +252,9 @@ log_file = "%s-%s-B%d-%s.json" % (network, layout, batch_size, target.kind.name)
# Extract tasks from the network
print("Get model...")
-try:
- mod, params, input_shape, output_shape = get_network(
- network, batch_size, layout, dtype=dtype, use_sparse=use_sparse
- )
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+mod, params, input_shape, output_shape = get_network(
+ network, batch_size, layout, dtype=dtype, use_sparse=use_sparse
+)
print("Extract tasks...")
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
diff --git a/docs/_downloads/35eacf8f75629e07aeda1329bdb7d53c/tune_relay_arm.py b/docs/_downloads/35eacf8f75629e07aeda1329bdb7d53c/tune_relay_arm.py
index 4f2e952ce3..13674f5cda 100644
--- a/docs/_downloads/35eacf8f75629e07aeda1329bdb7d53c/tune_relay_arm.py
+++ b/docs/_downloads/35eacf8f75629e07aeda1329bdb7d53c/tune_relay_arm.py
@@ -105,17 +105,6 @@ def get_network(name, batch_size):
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
diff --git a/docs/_downloads/3fde0fe8b31bf786dec2a01858372eae/deploy_model_on_nano.py b/docs/_downloads/3fde0fe8b31bf786dec2a01858372eae/deploy_model_on_nano.py
index 761187e2d7..a656159546 100644
--- a/docs/_downloads/3fde0fe8b31bf786dec2a01858372eae/deploy_model_on_nano.py
+++ b/docs/_downloads/3fde0fe8b31bf786dec2a01858372eae/deploy_model_on_nano.py
@@ -102,22 +102,22 @@ from tvm.contrib.download import download_testdata
# -----------------------------
# Back to the host machine, which should have a full TVM installed (with LLVM).
#
-# We will use pre-trained model from
-# `MXNet Gluon model zoo <https://mxnet.apache.org/api/python/gluon/model_zoo.html>`_.
-# You can found more details about this part at tutorial :ref:`tutorial-from-mxnet`.
+# We will use pre-trained model from torchvision
-import sys
-
-from mxnet.gluon.model_zoo.vision import get_model
+import torch
+import torchvision
from PIL import Image
import numpy as np
# one line to get the model
-try:
- block = get_model("resnet18_v1", pretrained=True)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+model_name = "resnet18"
+model = getattr(torchvision.models, model_name)(pretrained=True)
+model = model.eval()
+
+# We grab the TorchScripted model via tracing
+input_shape = [1, 3, 224, 224]
+input_data = torch.randn(input_shape)
+scripted_model = torch.jit.trace(model, input_data).eval()
######################################################################
# In order to test our model, here we download an image of cat and
@@ -158,9 +158,9 @@ with open(synset_path) as f:
# Now we would like to port the Gluon model to a portable computational graph.
# It's as easy as several lines.
-# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
-shape_dict = {"data": x.shape}
-mod, params = relay.frontend.from_mxnet(block, shape_dict)
+input_name = "input0"
+shape_list = [(input_name, x.shape)]
+mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
# we want a probability so add a softmax operator
func = mod["main"]
func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
@@ -241,7 +241,7 @@ else:
module = runtime.GraphModule(rlib["default"](dev))
# set input data
-module.set_input("data", tvm.nd.array(x.astype("float32")))
+module.set_input(input_name, tvm.nd.array(x.astype("float32")))
# run
module.run()
# get output
diff --git a/docs/_downloads/408ef96692a668b33d94eca33cac7e0a/deploy_model_on_rasp.py b/docs/_downloads/408ef96692a668b33d94eca33cac7e0a/deploy_model_on_rasp.py
index 5196ae9ce1..64f83dbbc0 100644
--- a/docs/_downloads/408ef96692a668b33d94eca33cac7e0a/deploy_model_on_rasp.py
+++ b/docs/_downloads/408ef96692a668b33d94eca33cac7e0a/deploy_model_on_rasp.py
@@ -95,22 +95,22 @@ from tvm.contrib.download import download_testdata
# -----------------------------
# Back to the host machine, which should have a full TVM installed (with LLVM).
#
-# We will use pre-trained model from
-# `MXNet Gluon model zoo <https://mxnet.apache.org/api/python/gluon/model_zoo.html>`_.
-# You can found more details about this part at tutorial :ref:`tutorial-from-mxnet`.
+# We will use pre-trained model from torchvision
-import sys
-
-from mxnet.gluon.model_zoo.vision import get_model
+import torch
+import torchvision
from PIL import Image
import numpy as np
# one line to get the model
-try:
- block = get_model("resnet18_v1", pretrained=True)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+model_name = "resnet18"
+model = getattr(torchvision.models, model_name)(pretrained=True)
+model = model.eval()
+
+# We grab the TorchScripted model via tracing
+input_shape = [1, 3, 224, 224]
+input_data = torch.randn(input_shape)
+scripted_model = torch.jit.trace(model, input_data).eval()
######################################################################
# In order to test our model, here we download an image of cat and
@@ -148,12 +148,12 @@ with open(synset_path) as f:
synset = eval(f.read())
######################################################################
-# Now we would like to port the Gluon model to a portable computational graph.
+# Now we would like to port the PyTorch model to a portable computational graph.
# It's as easy as several lines.
-# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
-shape_dict = {"data": x.shape}
-mod, params = relay.frontend.from_mxnet(block, shape_dict)
+input_name = "input0"
+shape_list = [(input_name, x.shape)]
+mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
# we want a probability so add a softmax operator
func = mod["main"]
func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
@@ -226,7 +226,7 @@ rlib = remote.load_module("net.tar")
dev = remote.cpu(0)
module = runtime.GraphModule(rlib["default"](dev))
# set input data
-module.set_input("data", tvm.nd.array(x.astype("float32")))
+module.set_input(input_name, tvm.nd.array(x.astype("float32")))
# run
module.run()
# get output
diff --git a/docs/_downloads/4bbcfcce3c35b0b795a42c998ceb3770/from_mxnet.ipynb b/docs/_downloads/4bbcfcce3c35b0b795a42c998ceb3770/from_mxnet.ipynb
deleted file mode 100644
index 0f3a95608d..0000000000
--- a/docs/_downloads/4bbcfcce3c35b0b795a42c998ceb3770/from_mxnet.ipynb
+++ /dev/null
@@ -1,180 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI, with CUDA enabled. To use this,\n# you must request a Google Colab instance with a GPU by going to Runtime ->\n# Change runtime type -> Hardware accelerator -> GPU. If you wish to build from\n# source, see https://tvm.apache.org/docs/install/from_source.html\npip install tlcpack-nightly-cu113 --pre -f https://tlcpack.ai/wheels"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n\n# Compile MXNet Models\n**Author**: [Joshua Z. Zhang](https://zhreshold.github.io/), [Kazutaka Morita](https://github.com/kazum)\n\nThis article is an introductory tutorial to deploy mxnet models with Relay. To begin, we must install `mxnet`:\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\npip install mxnet"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "or please refer to official installation guide.\nhttps://mxnet.apache.org/versions/master/install/index.html\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# some standard imports\nimport sys\nimport mxnet as mx\nimport tvm\nimport tvm.relay as relay\nimport numpy as np"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Download Resnet18 model from Gluon Model Zoo\nIn this section, we download a pretrained imagenet model and classify an image.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from tvm.contrib.download import download_testdata\nfrom mxnet.gluon.model_zoo.vision import get_model\nfrom PIL import Image\nfrom matplotlib import pyplot as plt\n\ntry:\n block = get_model(\"resnet18_v1\", pretrained=True)\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)\n\nimg_url = \"https://github.com/dmlc/mxnet.js/blob/main/data/cat.png?raw=true\"\nimg_name = \"cat.png\"\nsynset_url = \"\".join(\n [\n [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Compile the Graph\nNow we would like to port the Gluon model to a portable computational graph.\nIt's as easy as several lines.\nWe support MXNet static graph(symbol) and HybridBlock in mxnet.gluon\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "shape_dict = {\"data\": x.shape}\nmod, params = relay.frontend.from_mxnet(block, shape_dict)\n## we want a probability so add a softmax operator\nfunc = mod[\"main\"]\nfunc = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "now compile the graph\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "target = \"cuda\"\nwith tvm.transform.PassContext(opt_level=3):\n lib = relay.build(func, target, params=params)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Execute the portable graph on TVM\nNow, we would like to reproduce the same forward computation using TVM.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from tvm.contrib import graph_executor\n\ndev = tvm.cuda(0)\ndtype = \"float32\"\nm = graph_executor.GraphModule(lib[\"default\"](dev))\n# set inputs\nm.set_input(\"data\", tvm.nd.array(x.astype(dtype)))\n# execute\nm.run()\n# get outputs\ntvm_output = m.get_output(0)\ntop1 = np.argmax(tvm_output.numpy()[0])\nprint(\"TVM prediction top-1:\", top1, synset[top1])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Use MXNet symbol with pretrained weights\nMXNet often use `arg_params` and `aux_params` to store network parameters\nseparately, here we show how to use these weights with existing API\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def block2symbol(block):\n data = mx.sym.Variable(\"data\")\n sym = block(data)\n args = {}\n auxs = {}\n for k, v in block.collect_params().items():\n args[k] = mx.nd.array(v.data().asnumpy())\n return sym, args, auxs\n\n\nmx_sym, args, auxs = block2symbol(block)\n# usually we would save/load it as checkpoint\nmx.model.save_checkpoint(\"resnet18_v1\", 0, mx_sym, args, auxs)\n# there are 'resnet18_v1-0000.params' and 'resnet18_v1-symbol.json' on disk"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "for a normal mxnet model, we start from here\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "mx_sym, args, auxs = mx.model.load_checkpoint(\"resnet18_v1\", 0)\n# now we use the same API to get Relay computation graph\nmod, relay_params = relay.frontend.from_mxnet(mx_sym, shape_dict, arg_params=args, aux_params=auxs)\n# repeat the same steps to run this model using TVM"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.18"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb b/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb
index 215151a769..9f87586cd2 100644
--- a/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb
+++ b/docs/_downloads/4dc30a43f3a6aa3ed4bc3077ad35ff70/tune_network_arm.ipynb
@@ -44,7 +44,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\", use_sparse=False):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_ [...]
+ "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\", use_sparse=False):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_ [...]
]
},
{
@@ -94,7 +94,7 @@
},
"outputs": [],
"source": [
- "# Extract tasks from the network\nprint(\"Get model...\")\ntry:\n mod, params, input_shape, output_shape = get_network(\n network, batch_size, layout, dtype=dtype, use_sparse=use_sparse\n )\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)\nprint(\"Extract tasks...\")\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"==== [...]
+ "# Extract tasks from the network\nprint(\"Get model...\")\nmod, params, input_shape, output_shape = get_network(\n network, batch_size, layout, dtype=dtype, use_sparse=use_sparse\n)\nprint(\"Extract tasks...\")\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"========== Task %d (workload key: %s) ==========\" % (idx, task.workload_key))\n print(task.compute_dag)"
]
},
{
diff --git a/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb b/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb
index c404bff1c1..087932e1a4 100644
--- a/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb
+++ b/docs/_downloads/5e4e499c097b16a90c517e630502253a/tune_network_mali.ipynb
@@ -44,7 +44,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\"):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_size, 1000)\n\n [...]
+ "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\"):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_size, 1000)\n\n [...]
]
},
{
@@ -80,7 +80,7 @@
},
"outputs": [],
"source": [
- "# Extract tasks from the network\nprint(\"Extract tasks...\")\ntry:\n mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"========== Task %d (workload key: %s) ==========\" % (idx, task.wor [...]
+ "# Extract tasks from the network\nprint(\"Extract tasks...\")\nmod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"========== Task %d (workload key: %s) ==========\" % (idx, task.workload_key))\n print(task.compute_dag)"
]
},
{
diff --git a/docs/_downloads/644d28fc67dfb3099fb0d275ffcf1c7c/tune_relay_mobile_gpu.py b/docs/_downloads/644d28fc67dfb3099fb0d275ffcf1c7c/tune_relay_mobile_gpu.py
index 3c2f173c23..d5b4b217ab 100644
--- a/docs/_downloads/644d28fc67dfb3099fb0d275ffcf1c7c/tune_relay_mobile_gpu.py
+++ b/docs/_downloads/644d28fc67dfb3099fb0d275ffcf1c7c/tune_relay_mobile_gpu.py
@@ -104,17 +104,6 @@ def get_network(name, batch_size):
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
diff --git a/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py b/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py
index 1c531a5303..15f3379013 100644
--- a/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py
+++ b/docs/_downloads/67bf7dd99bcfb837cf3e8b461a5eeb48/tune_network_mali.py
@@ -117,19 +117,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet50_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
return mod, params, input_shape, output_shape
@@ -171,11 +158,7 @@ device_key = "rk3399"
# Extract tasks from the network
print("Extract tasks...")
-try:
- mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
for idx, task in enumerate(tasks):
diff --git a/docs/_downloads/6836ce26807b8d33b8f499287c1f3d04/tune_relay_x86.py b/docs/_downloads/6836ce26807b8d33b8f499287c1f3d04/tune_relay_x86.py
index a637f7222e..b56ec0ad0e 100644
--- a/docs/_downloads/6836ce26807b8d33b8f499287c1f3d04/tune_relay_x86.py
+++ b/docs/_downloads/6836ce26807b8d33b8f499287c1f3d04/tune_relay_x86.py
@@ -74,17 +74,6 @@ def get_network(name, batch_size):
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={input_name: input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
diff --git a/docs/_downloads/6b0f549107f73f2e48c894372be08bcb/tune_relay_mobile_gpu.ipynb b/docs/_downloads/6b0f549107f73f2e48c894372be08bcb/tune_relay_mobile_gpu.ipynb
index df3be409e5..5b391c720e 100644
--- a/docs/_downloads/6b0f549107f73f2e48c894372be08bcb/tune_relay_mobile_gpu.ipynb
+++ b/docs/_downloads/6b0f549107f73f2e48c894372be08bcb/tune_relay_mobile_gpu.ipynb
@@ -51,7 +51,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
+ "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
]
},
{
diff --git a/docs/_downloads/6bbcf342fb9192416b8e1a86a1d4e981/tune_alu_vta.py b/docs/_downloads/6bbcf342fb9192416b8e1a86a1d4e981/tune_alu_vta.py
deleted file mode 100644
index 8ee58fe990..0000000000
--- a/docs/_downloads/6bbcf342fb9192416b8e1a86a1d4e981/tune_alu_vta.py
+++ /dev/null
@@ -1,345 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-Auto-tuning a ALU fused op on VTA
----------------------------------
-"""
-
-import os
-import sys
-from mxnet.gluon.model_zoo import vision
-import numpy as np
-from PIL import Image
-
-from tvm import topi
-import tvm
-from tvm import te
-from tvm import rpc, autotvm, relay
-from tvm.contrib import download
-from tvm.autotvm.measure.measure_methods import request_remote
-from tvm.autotvm.tuner import XGBTuner, GATuner, RandomTuner, GridSearchTuner
-from tvm.autotvm import record
-
-import vta
-from vta.testing import simulator
-from vta.top import graph_pack
-import copy
-
-
-#################################################################
-# Compile network
-# ---------------
-# Perform vta-specific compilation with Relay from a Gluon model
-def compile_network(env, target, model, start_pack, stop_pack):
-
- # Populate the shape and data type dictionary
- dtype_dict = {"data": "float32"}
- shape_dict = {"data": (env.BATCH, 3, 224, 224)}
-
- # Get off the shelf gluon model, and convert to relay
- gluon_model = vision.get_model(model, pretrained=True)
- mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)
-
- # Update shape and type dictionary
- shape_dict.update({k: v.shape for k, v in params.items()})
- dtype_dict.update({k: str(v.dtype) for k, v in params.items()})
-
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with relay.build_config(opt_level=3):
- with relay.quantize.qconfig(global_scale=8.0, skip_conv_layers=[0]):
- mod = relay.quantize.quantize(mod, params=params)
-
- # Perform graph packing and constant folding for VTA target
- if target.device_name == "vta":
- assert env.BLOCK_IN == env.BLOCK_OUT
- relay_prog = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=start_pack,
- stop_name=stop_pack,
- )
-
- return relay_prog, params
-
-
-###########################################
-# Set Tuning Options
-# ------------------
-# Before tuning, we should apply some configurations.
-# Here we use an Pynq-Z1 board as an example.
-
-# Tracker host and port can be set by your environment
-tracker_host = os.environ.get("TVM_TRACKER_HOST", "0.0.0.0")
-tracker_port = int(os.environ.get("TVM_TRACKER_PORT", 9190))
-
-# Load VTA parameters from the vta/config/vta_config.json file
-env = vta.get_env()
-
-# This target is used for cross compilation. You can query it by :code:`gcc -v` on your device.
-# Set ``device=arm_cpu`` to run inference on the CPU
-# or ``device=vta`` to run inference on the FPGA.
-device = "vta"
-target = env.target if device == "vta" else env.target_vta_cpu
-
-# Name of Gluon model to compile
-# The ``start_pack`` and ``stop_pack`` labels indicate where
-# to start and end the graph packing relay pass: in other words
-# where to start and finish offloading to VTA.
-network = "resnet50_v2"
-start_pack = "nn.max_pool2d"
-stop_pack = "nn.global_avg_pool2d"
-
-# Tuning option
-log_file = "%s.alu.%s.log" % (device, network)
-tuning_option = {
- "log_filename": log_file,
- "tuner": "random",
- "n_trial": 1000,
- "early_stopping": None,
- "measure_option": autotvm.measure_option(
- builder=autotvm.LocalBuilder(n_parallel=1),
- runner=autotvm.RPCRunner(
- env.TARGET,
- host=tracker_host,
- port=tracker_port,
- number=5,
- timeout=60,
- # check_correctness=True, # TODO: re-enable when check_correctness works again.
- ),
- ),
-}
-
-
-def log_to_file(file_out, protocol="json"):
- """Log the tuning records into file.
- The rows of the log are stored in the format of autotvm.record.encode.
- for lhs == rhs, we add an extra rhs = [] record
-
- Parameters
- ----------
- file_out : str
- The file to log to.
- protocol: str, optional
- The log protocol. Can be 'json' or 'pickle'
-
- Returns
- -------
- callback : callable
- Callback function to do the logging.
- """
-
- def _callback(_, inputs, results):
- with open(file_out, "a") as f:
- for inp, result in zip(inputs, results):
- f.write(record.encode(inp, result, protocol) + "\n")
-
- # we only consider task with same lhs and rhs
- if inp.task.args[0] == inp.task.args[1]:
- args = list(inp.task.args)
- args[1] = (args[0][0], (), args[0][2])
- inp_copy = copy.deepcopy(inp)
- inp_copy.task.args = tuple(args)
- f.write(record.encode(inp_copy, result, protocol) + "\n")
-
- return _callback
-
-
-def tune_tasks(
- tasks,
- measure_option,
- tuner="xgb",
- n_trial=10,
- early_stopping=None,
- log_filename="tuning.log",
- use_transfer_learning=True,
-):
-
- # create tmp log file
- tmp_log_file = log_filename + ".tmp"
- if os.path.exists(tmp_log_file):
- os.remove(tmp_log_file)
-
- for i, tsk in enumerate(reversed(tasks)):
- prefix = "[Task %2d/%2d] " % (i + 1, len(tasks))
-
- # create tuner
- if tuner == "xgb":
- tuner_obj = XGBTuner(tsk, loss_type="reg")
- elif tuner == "xgb_knob":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="knob")
- elif tuner == "xgb_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="itervar")
- elif tuner == "xgb_curve":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="curve")
- elif tuner == "xgb_rank":
- tuner_obj = XGBTuner(tsk, loss_type="rank")
- elif tuner == "xgb_rank_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="knob")
- elif tuner == "xgb_rank_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="itervar")
- elif tuner == "xgb_rank_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="curve")
- elif tuner == "xgb_rank_binary":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary")
- elif tuner == "xgb_rank_binary_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="knob")
- elif tuner == "xgb_rank_binary_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="itervar")
- elif tuner == "xgb_rank_binary_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="curve")
- elif tuner == "ga":
- tuner_obj = GATuner(tsk, pop_size=50)
- elif tuner == "random":
- tuner_obj = RandomTuner(tsk)
- elif tuner == "gridsearch":
- tuner_obj = GridSearchTuner(tsk)
- else:
- raise ValueError("Invalid tuner: " + tuner)
-
- if use_transfer_learning:
- if os.path.isfile(tmp_log_file):
- tuner_obj.load_history(autotvm.record.load_from_file(tmp_log_file))
-
- # do tuning
- tsk_trial = min(n_trial, len(tsk.config_space))
- tuner_obj.tune(
- n_trial=tsk_trial,
- early_stopping=early_stopping,
- measure_option=measure_option,
- callbacks=[
- autotvm.callback.progress_bar(tsk_trial, prefix=prefix),
- log_to_file(tmp_log_file),
- ],
- )
-
- # pick best records to a cache file
- autotvm.record.pick_best(tmp_log_file, log_filename)
- os.remove(tmp_log_file)
-
-
-########################################################################
-# Register VTA-specific tuning tasks
-def register_vta_tuning_tasks():
- from tvm.autotvm.task import TaskExtractEnv
-
- @tvm.te.tag_scope(tag=topi.tag.ELEMWISE)
- def my_clip(x, a_min, a_max):
- """Unlike topi's current clip, put min and max into two stages."""
- const_min = tvm.tir.const(a_min, x.dtype)
- const_max = tvm.tir.const(a_max, x.dtype)
- x = te.compute(x.shape, lambda *i: tvm.te.min(x(*i), const_max), name="clipA")
- x = te.compute(x.shape, lambda *i: tvm.te.max(x(*i), const_min), name="clipB")
- return x
-
- # init autotvm env to register VTA operator
- TaskExtractEnv()
-
- @autotvm.template("add.vta")
- def _topi_add(*args, **kwargs):
- assert not kwargs, "Do not support kwargs in template function call"
- A, B = args[:2]
-
- with tvm.target.vta():
- res = vta.top.op.add_packed(*args, **kwargs)
- res = my_clip(res, 0, 127)
- res = topi.cast(res, "int8")
-
- if tvm.target.Target.current().device_name == "vta":
- s = vta.top.op.schedule_add_packed([res])
- else:
- s = te.create_schedule([res.op])
- return s, [A, B, res]
-
- @autotvm.template("multiply.vta")
- def _topi_multiply(*args, **kwargs):
- assert not kwargs, "Do not support kwargs in template function call"
- A, B = args[:2]
-
- with tvm.target.vta():
- res = vta.top.op.multiply_packed(*args, **kwargs)
- res = my_clip(res, 0, 127)
- res = topi.cast(res, "int8")
-
- if tvm.target.Target.current().device_name == "vta":
- s = vta.top.op.schedule_multiply_packed([res])
- else:
- s = te.create_schedule([res.op])
- return s, [A, B, res]
-
-
-########################################################################
-# Finally, we launch tuning jobs and evaluate the end-to-end performance.
-def tune_and_evaluate(tuning_opt):
-
- if env.TARGET != "intelfocl":
- print("ALU only op only available for intelfocl target")
- return
-
- # Register VTA tuning tasks
- register_vta_tuning_tasks()
-
- # Perform task extraction on Relay program
- print("Extract tasks...")
- relay_prog, params = compile_network(env, target, network, start_pack, stop_pack)
- mod = tvm.IRModule.from_expr(relay_prog)
- tasks = autotvm.task.extract_from_program(
- mod,
- params=params,
- ops=(
- relay.op.get("add"),
- relay.op.get("multiply"),
- ),
- target=tvm.target.Target(target, host=env.target_host),
- )
-
- # filter out non-packed alu task
- tasks = list(filter(lambda t: len(t.args[0][1]) > 4, tasks))
- # filter out float alu task
- tasks = list(filter(lambda t: t.args[0][2] != "float32", tasks))
-
- # We should have extracted 10 convolution tasks
- tasks_set = {}
- print("Extracted {} alu tasks:".format(len(tasks)))
- for tsk in tasks:
- print("tsk = ", tsk)
-
- if len(tsk.args[1][1]) == 0:
- args = list(tsk.args)
- args[1] = args[0]
- tsk.args = tuple(args)
-
- if (tsk.name, tsk.args) in tasks_set:
- print("task {} already exists".format(tsk))
- tasks_set[(tsk.name, tsk.args)] = tsk
-
- tasks = list(tasks_set.values())
- print("After merged, final #tasks={}, tasks = {}".format(len(tasks), tasks))
-
- # run tuning tasks
- print("Tuning...")
- tune_tasks(tasks, **tuning_opt)
-
-
-# Run the tuning and evaluate the results
-try:
- tune_and_evaluate(tuning_option)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
diff --git a/docs/_downloads/7810ecf51bfc05f7d5e8a400ac3e815d/deploy_quantized.py b/docs/_downloads/7810ecf51bfc05f7d5e8a400ac3e815d/deploy_quantized.py
deleted file mode 100644
index 2cdb7da5f8..0000000000
--- a/docs/_downloads/7810ecf51bfc05f7d5e8a400ac3e815d/deploy_quantized.py
+++ /dev/null
@@ -1,172 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-Deploy a Quantized Model on Cuda
-================================
-**Author**: `Wuwei Lin <https://github.com/vinx13>`_
-
-This article is an introductory tutorial of automatic quantization with TVM.
-Automatic quantization is one of the quantization modes in TVM. More details on
-the quantization story in TVM can be found
-`here <https://discuss.tvm.apache.org/t/quantization-story/3920>`_.
-In this tutorial, we will import a GluonCV pre-trained model on ImageNet to
-Relay, quantize the Relay model and then perform the inference.
-"""
-
-import logging
-import os
-import sys
-
-import tvm
-from tvm import te
-from tvm import relay
-import mxnet as mx
-from tvm.contrib.download import download_testdata
-from mxnet import gluon
-
-
-batch_size = 1
-model_name = "resnet18_v1"
-target = "cuda"
-dev = tvm.device(target)
-
-###############################################################################
-# Prepare the Dataset
-# -------------------
-# We will demonstrate how to prepare the calibration dataset for quantization.
-# We first download the validation set of ImageNet and pre-process the dataset.
-calibration_rec = download_testdata(
- "http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/data/val_256_q90.rec",
- "val_256_q90.rec",
-)
-
-
-def get_val_data(num_workers=4):
- mean_rgb = [123.68, 116.779, 103.939]
- std_rgb = [58.393, 57.12, 57.375]
-
- def batch_fn(batch):
- return batch.data[0].asnumpy(), batch.label[0].asnumpy()
-
- img_size = 299 if model_name == "inceptionv3" else 224
- val_data = mx.io.ImageRecordIter(
- path_imgrec=calibration_rec,
- preprocess_threads=num_workers,
- shuffle=False,
- batch_size=batch_size,
- resize=256,
- data_shape=(3, img_size, img_size),
- mean_r=mean_rgb[0],
- mean_g=mean_rgb[1],
- mean_b=mean_rgb[2],
- std_r=std_rgb[0],
- std_g=std_rgb[1],
- std_b=std_rgb[2],
- )
- return val_data, batch_fn
-
-
-###############################################################################
-# The calibration dataset should be an iterable object. We define the
-# calibration dataset as a generator object in Python. In this tutorial, we
-# only use a few samples for calibration.
-
-calibration_samples = 10
-
-
-def calibrate_dataset():
- val_data, batch_fn = get_val_data()
- val_data.reset()
- for i, batch in enumerate(val_data):
- if i * batch_size >= calibration_samples:
- break
- data, _ = batch_fn(batch)
- yield {"data": data}
-
-
-###############################################################################
-# Import the model
-# ----------------
-# We use the Relay MxNet frontend to import a model from the Gluon model zoo.
-def get_model():
- gluon_model = gluon.model_zoo.vision.get_model(model_name, pretrained=True)
- img_size = 299 if model_name == "inceptionv3" else 224
- data_shape = (batch_size, 3, img_size, img_size)
- mod, params = relay.frontend.from_mxnet(gluon_model, {"data": data_shape})
- return mod, params
-
-
-###############################################################################
-# Quantize the Model
-# ------------------
-# In quantization, we need to find the scale for each weight and intermediate
-# feature map tensor of each layer.
-#
-# For weights, the scales are directly calculated based on the value of the
-# weights. Two modes are supported: `power2` and `max`. Both modes find the
-# maximum value within the weight tensor first. In `power2` mode, the maximum
-# is rounded down to power of two. If the scales of both weights and
-# intermediate feature maps are power of two, we can leverage bit shifting for
-# multiplications. This make it computationally more efficient. In `max` mode,
-# the maximum is used as the scale. Without rounding, `max` mode might have
-# better accuracy in some cases. When the scales are not powers of two, fixed
-# point multiplications will be used.
-#
-# For intermediate feature maps, we can find the scales with data-aware
-# quantization. Data-aware quantization takes a calibration dataset as the
-# input argument. Scales are calculated by minimizing the KL divergence between
-# distribution of activation before and after quantization.
-# Alternatively, we can also use pre-defined global scales. This saves the time
-# for calibration. But the accuracy might be impacted.
-
-
-def quantize(mod, params, data_aware):
- if data_aware:
- with relay.quantize.qconfig(calibrate_mode="kl_divergence", weight_scale="max"):
- mod = relay.quantize.quantize(mod, params, dataset=calibrate_dataset())
- else:
- with relay.quantize.qconfig(calibrate_mode="global_scale", global_scale=8.0):
- mod = relay.quantize.quantize(mod, params)
- return mod
-
-
-###############################################################################
-# Run Inference
-# -------------
-# We create a Relay VM to build and execute the model.
-def run_inference(mod):
- model = relay.create_executor("vm", mod, dev, target).evaluate()
- val_data, batch_fn = get_val_data()
- for i, batch in enumerate(val_data):
- data, label = batch_fn(batch)
- prediction = model(data)
- if i > 10: # only run inference on a few samples in this tutorial
- break
-
-
-def main():
- try:
- mod, params = get_model()
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- return
- mod = quantize(mod, params, data_aware=True)
- run_inference(mod)
-
-
-if __name__ == "__main__":
- main()
diff --git a/docs/_downloads/7c392f39b90d93406ef30c6185c5686c/deploy_model_on_rasp.ipynb b/docs/_downloads/7c392f39b90d93406ef30c6185c5686c/deploy_model_on_rasp.ipynb
index 99be5240ce..f319c3a196 100644
--- a/docs/_downloads/7c392f39b90d93406ef30c6185c5686c/deploy_model_on_rasp.ipynb
+++ b/docs/_downloads/7c392f39b90d93406ef30c6185c5686c/deploy_model_on_rasp.ipynb
@@ -47,7 +47,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Prepare the Pre-trained Model\nBack to the host machine, which should have a full TVM installed (with LLVM).\n\nWe will use pre-trained model from\n[MXNet Gluon model zoo](https://mxnet.apache.org/api/python/gluon/model_zoo.html).\nYou can found more details about this part at tutorial `tutorial-from-mxnet`.\n\n"
+ "## Prepare the Pre-trained Model\nBack to the host machine, which should have a full TVM installed (with LLVM).\n\nWe will use pre-trained model from torchvision\n\n"
]
},
{
@@ -58,7 +58,7 @@
},
"outputs": [],
"source": [
- "import sys\n\nfrom mxnet.gluon.model_zoo.vision import get_model\nfrom PIL import Image\nimport numpy as np\n\n# one line to get the model\ntry:\n block = get_model(\"resnet18_v1\", pretrained=True)\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)"
+ "import torch\nimport torchvision\nfrom PIL import Image\nimport numpy as np\n\n# one line to get the model\nmodel_name = \"resnet18\"\nmodel = getattr(torchvision.models, model_name)(pretrained=True)\nmodel = model.eval()\n\n# We grab the TorchScripted model via tracing\ninput_shape = [1, 3, 224, 224]\ninput_data = torch.randn(input_shape)\nscripted_model = torch.jit.trace(model, input_data).eval()"
]
},
{
@@ -101,7 +101,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "Now we would like to port the Gluon model to a portable computational graph.\nIt's as easy as several lines.\n\n"
+ "Now we would like to port the PyTorch model to a portable computational graph.\nIt's as easy as several lines.\n\n"
]
},
{
@@ -112,7 +112,7 @@
},
"outputs": [],
"source": [
- "# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon\nshape_dict = {\"data\": x.shape}\nmod, params = relay.frontend.from_mxnet(block, shape_dict)\n# we want a probability so add a softmax operator\nfunc = mod[\"main\"]\nfunc = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)"
+ "input_name = \"input0\"\nshape_list = [(input_name, x.shape)]\nmod, params = relay.frontend.from_pytorch(scripted_model, shape_list)\n# we want a probability so add a softmax operator\nfunc = mod[\"main\"]\nfunc = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)"
]
},
{
@@ -173,7 +173,7 @@
},
"outputs": [],
"source": [
- "# obtain an RPC session from remote device.\nif local_demo:\n remote = rpc.LocalSession()\nelse:\n # The following is my environment, change this to the IP address of your target device\n host = \"10.77.1.162\"\n port = 9090\n remote = rpc.connect(host, port)\n\n# upload the library to remote device and load it\nremote.upload(lib_fname)\nrlib = remote.load_module(\"net.tar\")\n\n# create the remote runtime module\ndev = remote.cpu(0)\nmodule = runtime.GraphModule( [...]
+ "# obtain an RPC session from remote device.\nif local_demo:\n remote = rpc.LocalSession()\nelse:\n # The following is my environment, change this to the IP address of your target device\n host = \"10.77.1.162\"\n port = 9090\n remote = rpc.connect(host, port)\n\n# upload the library to remote device and load it\nremote.upload(lib_fname)\nrlib = remote.load_module(\"net.tar\")\n\n# create the remote runtime module\ndev = remote.cpu(0)\nmodule = runtime.GraphModule( [...]
]
}
],
diff --git a/docs/_downloads/910e6ecee4ecac8d8ca0baeb6d00689d/tune_relay_x86.ipynb b/docs/_downloads/910e6ecee4ecac8d8ca0baeb6d00689d/tune_relay_x86.ipynb
index 499e0d6c87..237b0e654c 100644
--- a/docs/_downloads/910e6ecee4ecac8d8ca0baeb6d00689d/tune_relay_x86.ipynb
+++ b/docs/_downloads/910e6ecee4ecac8d8ca0baeb6d00689d/tune_relay_x86.ipynb
@@ -44,7 +44,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
+ "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
]
},
{
diff --git a/docs/_downloads/95395e118195f25266654dd8fbf487d4/deploy_classification.ipynb b/docs/_downloads/95395e118195f25266654dd8fbf487d4/deploy_classification.ipynb
deleted file mode 100644
index 922db11bb3..0000000000
--- a/docs/_downloads/95395e118195f25266654dd8fbf487d4/deploy_classification.ipynb
+++ /dev/null
@@ -1,133 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# Deploy Pretrained Vision Model from MxNet on VTA\n**Author**: [Thierry Moreau](https://homes.cs.washington.edu/~moreau/)\n\nThis tutorial provides an end-to-end demo, on how to run ImageNet classification\ninference onto the VTA accelerator design to perform ImageNet classification tasks.\nIt showcases Relay as a front end compiler that can perform quantization (VTA\nonly supports int8/32 inference) as well as graph packing (in order to enable\ntensorization in the core) to [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install dependencies\nTo use the autotvm package in tvm, we need to install some extra dependencies.\n(change \"3\" to \"2\" if you use python2):\n\n```bash\npip3 install --user mxnet requests \"Pillow<7\"\n```\nNow return to the python code. Import packages.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from __future__ import absolute_import, print_function\n\nimport argparse, json, os, requests, sys, time\nfrom io import BytesIO\nfrom os.path import join, isfile\nimport sys\nfrom PIL import Image\n\nfrom mxnet.gluon.model_zoo import vision\nimport numpy as np\nfrom matplotlib import pyplot as plt\n\nimport tvm\nfrom tvm import te\nfrom tvm import rpc, autotvm, relay\nfrom tvm.contrib import graph_executor, utils, download\nfrom tvm.contrib.debugger import debug_executor\nfrom [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Define the platform and model targets\nExecute on CPU vs. VTA, and define the model.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file\nenv = vta.get_env()\n\n# Set ``device=arm_cpu`` to run inference on the CPU\n# or ``device=vta`` to run inference on the FPGA.\ndevice = \"vta\"\ntarget = env.target if device == \"vta\" else env.target_vta_cpu\n\n# Dictionary lookup for when to start/end bit packing\npack_dict = {\n \"resnet18_v1\": [\"nn.max_pool2d\", \"nn.global_avg_pool2d\"],\n \"resnet34_v1\": [\"nn.max_pool2d\", \"nn.global_a [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Obtain an execution remote\nWhen target is 'pynq', reconfigure FPGA and runtime.\nOtherwise, if target is 'sim', execute locally.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "if env.TARGET not in [\"sim\", \"tsim\", \"intelfocl\"]:\n\n # Get remote from tracker node if environment variable is set.\n # To set up the tracker, you'll need to follow the \"Auto-tuning\n # a convolutional network for VTA\" tutorial.\n tracker_host = os.environ.get(\"TVM_TRACKER_HOST\", None)\n tracker_port = os.environ.get(\"TVM_TRACKER_PORT\", None)\n # Otherwise if you have a device you want to program directly from\n # the host, make sure you've set [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Build the inference graph executor\nGrab vision model from Gluon model zoo and compile with Relay.\nThe compilation steps are:\n\n1. Front end translation from MxNet into Relay module.\n2. Apply 8-bit quantization: here we skip the first conv layer,\n and dense layer which will both be executed in fp32 on the CPU.\n3. Perform graph packing to alter the data layout for tensorization.\n4. Perform constant folding to reduce number of operators (e.g. eliminate batch norm multip [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Load pre-configured AutoTVM schedules\nwith autotvm.tophub.context(target):\n\n # Populate the shape and data type dictionary for ImageNet classifier input\n dtype_dict = {\"data\": \"float32\"}\n shape_dict = {\"data\": (env.BATCH, 3, 224, 224)}\n\n # Get off the shelf gluon model, and convert to relay\n try:\n gluon_model = vision.get_model(model, pretrained=True)\n except RuntimeError:\n print(\"Downloads from mxnet no longer supported\", fil [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Perform image classification inference\nWe run classification on an image sample from ImageNet\nWe just need to download the categories files, `synset.txt`\nand an input test image.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Download ImageNet categories\ncateg_url = \"https://github.com/uwsampl/web-data/raw/main/vta/models/\"\ncateg_fn = \"synset.txt\"\ndownload.download(join(categ_url, categ_fn), categ_fn)\nsynset = eval(open(categ_fn).read())\n\n# Download test image\nimage_url = \"https://homes.cs.washington.edu/~moreau/media/vta/cat.jpg\"\nimage_fn = \"cat.png\"\ndownload.download(image_url, image_fn)\n\n# Prepare test image for inference\nimage = Image.open(image_fn).resize((224, 224))\nplt.i [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.18"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/9e8de33a5822b31748bfd76861009f92/deploy_classification.py b/docs/_downloads/9e8de33a5822b31748bfd76861009f92/deploy_classification.py
deleted file mode 100644
index c741a1678f..0000000000
--- a/docs/_downloads/9e8de33a5822b31748bfd76861009f92/deploy_classification.py
+++ /dev/null
@@ -1,313 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-Deploy Pretrained Vision Model from MxNet on VTA
-================================================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This tutorial provides an end-to-end demo, on how to run ImageNet classification
-inference onto the VTA accelerator design to perform ImageNet classification tasks.
-It showcases Relay as a front end compiler that can perform quantization (VTA
-only supports int8/32 inference) as well as graph packing (in order to enable
-tensorization in the core) to massage the compute graph for the hardware target.
-"""
-
-######################################################################
-# Install dependencies
-# --------------------
-# To use the autotvm package in tvm, we need to install some extra dependencies.
-# (change "3" to "2" if you use python2):
-#
-# .. code-block:: bash
-#
-# pip3 install --user mxnet requests "Pillow<7"
-#
-# Now return to the python code. Import packages.
-
-from __future__ import absolute_import, print_function
-
-import argparse, json, os, requests, sys, time
-from io import BytesIO
-from os.path import join, isfile
-import sys
-from PIL import Image
-
-from mxnet.gluon.model_zoo import vision
-import numpy as np
-from matplotlib import pyplot as plt
-
-import tvm
-from tvm import te
-from tvm import rpc, autotvm, relay
-from tvm.contrib import graph_executor, utils, download
-from tvm.contrib.debugger import debug_executor
-from tvm.relay import transform
-
-import vta
-from vta.testing import simulator
-from vta.top import graph_pack
-
-
-# Make sure that TVM was compiled with RPC=1
-assert tvm.runtime.enabled("rpc")
-
-######################################################################
-# Define the platform and model targets
-# -------------------------------------
-# Execute on CPU vs. VTA, and define the model.
-
-# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
-env = vta.get_env()
-
-# Set ``device=arm_cpu`` to run inference on the CPU
-# or ``device=vta`` to run inference on the FPGA.
-device = "vta"
-target = env.target if device == "vta" else env.target_vta_cpu
-
-# Dictionary lookup for when to start/end bit packing
-pack_dict = {
- "resnet18_v1": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet34_v1": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet18_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet34_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet50_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet101_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
-}
-
-# Name of Gluon model to compile
-# The ``start_pack`` and ``stop_pack`` labels indicate where
-# to start and end the graph packing relay pass: in other words
-# where to start and finish offloading to VTA.
-model = "resnet18_v1"
-assert model in pack_dict
-
-######################################################################
-# Obtain an execution remote
-# --------------------------
-# When target is 'pynq', reconfigure FPGA and runtime.
-# Otherwise, if target is 'sim', execute locally.
-
-if env.TARGET not in ["sim", "tsim", "intelfocl"]:
-
- # Get remote from tracker node if environment variable is set.
- # To set up the tracker, you'll need to follow the "Auto-tuning
- # a convolutional network for VTA" tutorial.
- tracker_host = os.environ.get("TVM_TRACKER_HOST", None)
- tracker_port = os.environ.get("TVM_TRACKER_PORT", None)
- # Otherwise if you have a device you want to program directly from
- # the host, make sure you've set the variables below to the IP of
- # your board.
- device_host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- device_port = os.environ.get("VTA_RPC_PORT", "9091")
- if not tracker_host or not tracker_port:
- remote = rpc.connect(device_host, int(device_port))
- else:
- remote = autotvm.measure.request_remote(
- env.TARGET, tracker_host, int(tracker_port), timeout=10000
- )
-
- # Reconfigure the JIT runtime and FPGA.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- reconfig_start = time.time()
- vta.reconfig_runtime(remote)
- vta.program_fpga(remote, bitstream=None)
- reconfig_time = time.time() - reconfig_start
- print("Reconfigured FPGA and RPC runtime in {0:.2f}s!".format(reconfig_time))
-
-# In simulation mode, host the RPC server locally.
-else:
- remote = rpc.LocalSession()
-
- if env.TARGET in ["intelfocl"]:
- # program intelfocl aocx
- vta.program_fpga(remote, bitstream="vta.bitstream")
-
-# Get execution context from remote
-ctx = remote.ext_dev(0) if device == "vta" else remote.cpu(0)
-
-######################################################################
-# Build the inference graph executor
-# ----------------------------------
-# Grab vision model from Gluon model zoo and compile with Relay.
-# The compilation steps are:
-#
-# 1. Front end translation from MxNet into Relay module.
-# 2. Apply 8-bit quantization: here we skip the first conv layer,
-# and dense layer which will both be executed in fp32 on the CPU.
-# 3. Perform graph packing to alter the data layout for tensorization.
-# 4. Perform constant folding to reduce number of operators (e.g. eliminate batch norm multiply).
-# 5. Perform relay build to object file.
-# 6. Load the object file onto remote (FPGA device).
-# 7. Generate graph executor, `m`.
-#
-
-# Load pre-configured AutoTVM schedules
-with autotvm.tophub.context(target):
-
- # Populate the shape and data type dictionary for ImageNet classifier input
- dtype_dict = {"data": "float32"}
- shape_dict = {"data": (env.BATCH, 3, 224, 224)}
-
- # Get off the shelf gluon model, and convert to relay
- try:
- gluon_model = vision.get_model(model, pretrained=True)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
- # Measure build start time
- build_start = time.time()
-
- # Start front end compilation
- mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)
-
- # Update shape and type dictionary
- shape_dict.update({k: v.shape for k, v in params.items()})
- dtype_dict.update({k: str(v.dtype) for k, v in params.items()})
-
- if target.device_name == "vta":
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with tvm.transform.PassContext(opt_level=3):
- with relay.quantize.qconfig(global_scale=8.0, skip_conv_layers=[0]):
- mod = relay.quantize.quantize(mod, params=params)
- # Perform graph packing and constant folding for VTA target
- assert env.BLOCK_IN == env.BLOCK_OUT
- # do device annotation if target is intelfocl or sim
- relay_prog = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=pack_dict[model][0],
- stop_name=pack_dict[model][1],
- device_annot=(env.TARGET == "intelfocl"),
- )
- else:
- relay_prog = mod["main"]
-
- # Compile Relay program with AlterOpLayout disabled
- if target.device_name != "vta":
- with tvm.transform.PassContext(opt_level=3, disabled_pass={"AlterOpLayout"}):
- graph, lib, params = relay.build(
- relay_prog, target=tvm.target.Target(target, host=env.target_host), params=params
- )
- else:
- if env.TARGET == "intelfocl":
- # multiple targets to run both on cpu and vta
- target = {"cpu": env.target_vta_cpu, "ext_dev": target}
- with vta.build_config(
- opt_level=3, disabled_pass={"AlterOpLayout", "tir.CommonSubexprElimTIR"}
- ):
- graph, lib, params = relay.build(
- relay_prog, target=tvm.target.Target(target, host=env.target_host), params=params
- )
-
- # Measure Relay build time
- build_time = time.time() - build_start
- print(model + " inference graph built in {0:.2f}s!".format(build_time))
-
- # Send the inference library over to the remote RPC server
- temp = utils.tempdir()
- lib.export_library(temp.relpath("graphlib.tar"))
- remote.upload(temp.relpath("graphlib.tar"))
- lib = remote.load_module("graphlib.tar")
-
- if env.TARGET == "intelfocl":
- ctxes = [remote.ext_dev(0), remote.cpu(0)]
- m = graph_executor.create(graph, lib, ctxes)
- else:
- # Graph runtime
- m = graph_executor.create(graph, lib, ctx)
-
-######################################################################
-# Perform image classification inference
-# --------------------------------------
-# We run classification on an image sample from ImageNet
-# We just need to download the categories files, `synset.txt`
-# and an input test image.
-
-# Download ImageNet categories
-categ_url = "https://github.com/uwsampl/web-data/raw/main/vta/models/"
-categ_fn = "synset.txt"
-download.download(join(categ_url, categ_fn), categ_fn)
-synset = eval(open(categ_fn).read())
-
-# Download test image
-image_url = "https://homes.cs.washington.edu/~moreau/media/vta/cat.jpg"
-image_fn = "cat.png"
-download.download(image_url, image_fn)
-
-# Prepare test image for inference
-image = Image.open(image_fn).resize((224, 224))
-plt.imshow(image)
-plt.show()
-image = np.array(image) - np.array([123.0, 117.0, 104.0])
-image /= np.array([58.395, 57.12, 57.375])
-image = image.transpose((2, 0, 1))
-image = image[np.newaxis, :]
-image = np.repeat(image, env.BATCH, axis=0)
-
-# Set the network parameters and inputs
-m.set_input(**params)
-m.set_input("data", image)
-
-# Perform inference and gather execution statistics
-# More on: :py:method:`tvm.runtime.Module.time_evaluator`
-num = 4 # number of times we run module for a single measurement
-rep = 3 # number of measurements (we derive std dev from this)
-timer = m.module.time_evaluator("run", ctx, number=num, repeat=rep)
-
-if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
- timer()
- sim_stats = simulator.stats()
- print("\nExecution statistics:")
- for k, v in sim_stats.items():
- # Since we execute the workload many times, we need to normalize stats
- # Note that there is always one warm up run
- # Therefore we divide the overall stats by (num * rep + 1)
- print("\t{:<16}: {:>16}".format(k, v // (num * rep + 1)))
-else:
- tcost = timer()
- std = np.std(tcost.results) * 1000
- mean = tcost.mean * 1000
- print("\nPerformed inference in %.2fms (std = %.2f) for %d samples" % (mean, std, env.BATCH))
- print("Average per sample inference time: %.2fms" % (mean / env.BATCH))
-
-# Get classification results
-tvm_output = m.get_output(0, tvm.nd.empty((env.BATCH, 1000), "float32", remote.cpu(0)))
-for b in range(env.BATCH):
- top_categories = np.argsort(tvm_output.numpy()[b])
- # Report top-5 classification results
- print("\n{} prediction for sample {}".format(model, b))
- print("\t#1:", synset[top_categories[-1]])
- print("\t#2:", synset[top_categories[-2]])
- print("\t#3:", synset[top_categories[-3]])
- print("\t#4:", synset[top_categories[-4]])
- print("\t#5:", synset[top_categories[-5]])
- # This just checks that one of the 5 top categories
- # is one variety of cat; this is by no means an accurate
- # assessment of how quantization affects classification
- # accuracy but is meant to catch changes to the
- # quantization pass that would accuracy in the CI.
- cat_detected = False
- for k in top_categories[-5:]:
- if "cat" in synset[k]:
- cat_detected = True
- assert cat_detected
diff --git a/docs/_downloads/a269cb38341b190be980a0bd3ea8a625/deploy_quantized.ipynb b/docs/_downloads/a269cb38341b190be980a0bd3ea8a625/deploy_quantized.ipynb
deleted file mode 100644
index 8501889b26..0000000000
--- a/docs/_downloads/a269cb38341b190be980a0bd3ea8a625/deploy_quantized.ipynb
+++ /dev/null
@@ -1,144 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# Deploy a Quantized Model on Cuda\n**Author**: [Wuwei Lin](https://github.com/vinx13)\n\nThis article is an introductory tutorial of automatic quantization with TVM.\nAutomatic quantization is one of the quantization modes in TVM. More details on\nthe quantization story in TVM can be found\n[here](https://discuss.tvm.apache.org/t/quantization-story/3920).\nIn this tutorial, we will import a GluonCV pre-trained model on ImageNet to\nRelay, quantize the Relay model and then per [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import logging\nimport os\nimport sys\n\nimport tvm\nfrom tvm import te\nfrom tvm import relay\nimport mxnet as mx\nfrom tvm.contrib.download import download_testdata\nfrom mxnet import gluon\n\n\nbatch_size = 1\nmodel_name = \"resnet18_v1\"\ntarget = \"cuda\"\ndev = tvm.device(target)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Prepare the Dataset\nWe will demonstrate how to prepare the calibration dataset for quantization.\nWe first download the validation set of ImageNet and pre-process the dataset.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "calibration_rec = download_testdata(\n \"http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/data/val_256_q90.rec\",\n \"val_256_q90.rec\",\n)\n\n\ndef get_val_data(num_workers=4):\n mean_rgb = [123.68, 116.779, 103.939]\n std_rgb = [58.393, 57.12, 57.375]\n\n def batch_fn(batch):\n return batch.data[0].asnumpy(), batch.label[0].asnumpy()\n\n img_size = 299 if model_name == \"inceptionv3\" else 224\n val_data = mx.io.ImageRecordIter(\n path [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The calibration dataset should be an iterable object. We define the\ncalibration dataset as a generator object in Python. In this tutorial, we\nonly use a few samples for calibration.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "calibration_samples = 10\n\n\ndef calibrate_dataset():\n val_data, batch_fn = get_val_data()\n val_data.reset()\n for i, batch in enumerate(val_data):\n if i * batch_size >= calibration_samples:\n break\n data, _ = batch_fn(batch)\n yield {\"data\": data}"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Import the model\nWe use the Relay MxNet frontend to import a model from the Gluon model zoo.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def get_model():\n gluon_model = gluon.model_zoo.vision.get_model(model_name, pretrained=True)\n img_size = 299 if model_name == \"inceptionv3\" else 224\n data_shape = (batch_size, 3, img_size, img_size)\n mod, params = relay.frontend.from_mxnet(gluon_model, {\"data\": data_shape})\n return mod, params"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Quantize the Model\nIn quantization, we need to find the scale for each weight and intermediate\nfeature map tensor of each layer.\n\nFor weights, the scales are directly calculated based on the value of the\nweights. Two modes are supported: `power2` and `max`. Both modes find the\nmaximum value within the weight tensor first. In `power2` mode, the maximum\nis rounded down to power of two. If the scales of both weights and\nintermediate feature maps are power of two, we can [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def quantize(mod, params, data_aware):\n if data_aware:\n with relay.quantize.qconfig(calibrate_mode=\"kl_divergence\", weight_scale=\"max\"):\n mod = relay.quantize.quantize(mod, params, dataset=calibrate_dataset())\n else:\n with relay.quantize.qconfig(calibrate_mode=\"global_scale\", global_scale=8.0):\n mod = relay.quantize.quantize(mod, params)\n return mod"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Run Inference\nWe create a Relay VM to build and execute the model.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def run_inference(mod):\n model = relay.create_executor(\"vm\", mod, dev, target).evaluate()\n val_data, batch_fn = get_val_data()\n for i, batch in enumerate(val_data):\n data, label = batch_fn(batch)\n prediction = model(data)\n if i > 10: # only run inference on a few samples in this tutorial\n break\n\n\ndef main():\n try:\n mod, params = get_model()\n except RuntimeError:\n print(\"Downloads from mxnet no longer [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.18"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb b/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb
index 112255b631..f9abe3e87c 100644
--- a/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb
+++ b/docs/_downloads/ad2a7f55d615d188ad664d56696815a6/tune_network_x86.ipynb
@@ -44,7 +44,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\", use_sparse=False):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_ [...]
+ "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\", use_sparse=False):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_ [...]
]
},
{
@@ -62,7 +62,7 @@
},
"outputs": [],
"source": [
- "# Extract tasks from the network\nprint(\"Get model...\")\ntry:\n mod, params, input_shape, output_shape = get_network(\n network,\n batch_size,\n layout,\n dtype=dtype,\n use_sparse=use_sparse,\n )\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)\n\nprint(\"Extract tasks...\")\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task [...]
+ "# Extract tasks from the network\nprint(\"Get model...\")\nmod, params, input_shape, output_shape = get_network(\n network,\n batch_size,\n layout,\n dtype=dtype,\n use_sparse=use_sparse,\n)\n\nprint(\"Extract tasks...\")\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"========== Task %d (workload key: %s) ==========\" % (idx, task.workload_key))\n print(task.compute_dag)"
]
},
{
diff --git a/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb b/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb
index 8d829ffc0d..0ee19f53a1 100644
--- a/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb
+++ b/docs/_downloads/af264436d049e3cd84803b67b6620b63/tune_network_cuda.ipynb
@@ -44,7 +44,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\"):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_size, 1000)\n\n [...]
+ "def get_network(name, batch_size, layout=\"NHWC\", dtype=\"float32\"):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n\n # auto-scheduler prefers NHWC layout\n if layout == \"NHWC\":\n image_shape = (224, 224, 3)\n elif layout == \"NCHW\":\n image_shape = (3, 224, 224)\n else:\n raise ValueError(\"Invalid layout: \" + layout)\n\n input_shape = (batch_size,) + image_shape\n output_shape = (batch_size, 1000)\n\n [...]
]
},
{
@@ -62,7 +62,7 @@
},
"outputs": [],
"source": [
- "# Extract tasks from the network\nprint(\"Extract tasks...\")\ntry:\n mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"========== Task %d (workload key: %s) ==========\" % (idx, task.wor [...]
+ "# Extract tasks from the network\nprint(\"Extract tasks...\")\nmod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)\ntasks, task_weights = auto_scheduler.extract_tasks(mod[\"main\"], params, target)\n\nfor idx, task in enumerate(tasks):\n print(\"========== Task %d (workload key: %s) ==========\" % (idx, task.workload_key))\n print(task.compute_dag)"
]
},
{
diff --git a/docs/_downloads/b11795df0596a55e4982bf895d0c8c38/bring_your_own_datatypes.ipynb b/docs/_downloads/b11795df0596a55e4982bf895d0c8c38/bring_your_own_datatypes.ipynb
deleted file mode 100644
index cd92662c0e..0000000000
--- a/docs/_downloads/b11795df0596a55e4982bf895d0c8c38/bring_your_own_datatypes.ipynb
+++ /dev/null
@@ -1,349 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# Bring Your Own Datatypes to TVM\n**Authors**: [Gus Smith](https://github.com/gussmith23), [Andrew Liu](https://github.com/hypercubestart)\n\nIn this tutorial, we will show you how to utilize the Bring Your Own Datatypes framework to use your own custom datatypes in TVM.\nNote that the Bring Your Own Datatypes framework currently only handles **software emulated versions of datatypes**.\nThe framework does not support compiling for custom accelerator datatypes out-of-the-box. [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## A Simple TVM Program\n\nWe'll begin by writing a simple program in TVM; afterwards, we will re-write it to use custom datatypes.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import sys\n\nimport tvm\nfrom tvm import relay\n\n# Our basic program: Z = X + Y\nx = relay.var(\"x\", shape=(3,), dtype=\"float32\")\ny = relay.var(\"y\", shape=(3,), dtype=\"float32\")\nz = x + y\nprogram = relay.Function([x, y], z)\nmodule = tvm.IRModule.from_expr(program)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now, we create random inputs to feed into this program using numpy:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import numpy as np\n\nnp.random.seed(23) # for reproducibility\n\nx_input = np.random.rand(3).astype(\"float32\")\ny_input = np.random.rand(3).astype(\"float32\")\nprint(\"x: {}\".format(x_input))\nprint(\"y: {}\".format(y_input))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Finally, we're ready to run the program:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "z_output = relay.create_executor(mod=module).evaluate()(x_input, y_input)\nprint(\"z: {}\".format(z_output))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Adding Custom Datatypes\nNow, we will do the same, but we will use a custom datatype for our intermediate computation.\n\nWe use the same input variables ``x`` and ``y`` as above, but before adding ``x + y``, we first cast both ``x`` and ``y`` to a custom datatype via the ``relay.cast(...)`` call.\n\nNote how we specify the custom datatype: we indicate it using the special ``custom[...]`` syntax.\nAdditionally, note the \"32\" after the datatype: this is the bitwidth of the c [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "try:\n with tvm.transform.PassContext(config={\"tir.disable_vectorize\": True}):\n x_myfloat = relay.cast(x, dtype=\"custom[myfloat]32\")\n y_myfloat = relay.cast(y, dtype=\"custom[myfloat]32\")\n z_myfloat = x_myfloat + y_myfloat\n z = relay.cast(z_myfloat, dtype=\"float32\")\nexcept tvm.TVMError as e:\n # Print last line of error\n print(str(e).split(\"\\n\")[-1])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Trying to generate this program throws an error from TVM.\nTVM does not know how to handle any custom datatype out of the box!\nWe first have to register the custom type with TVM, giving it a name and a type code:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "tvm.target.datatype.register(\"myfloat\", 150)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Note that the type code, 150, is currently chosen manually by the user.\nSee ``TVMTypeCode::kCustomBegin`` in [include/tvm/runtime/c_runtime_api.h](https://github.com/apache/tvm/blob/main/include/tvm/runtime/data_type.h).\nNow we can generate our program again:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "x_myfloat = relay.cast(x, dtype=\"custom[myfloat]32\")\ny_myfloat = relay.cast(y, dtype=\"custom[myfloat]32\")\nz_myfloat = x_myfloat + y_myfloat\nz = relay.cast(z_myfloat, dtype=\"float32\")\nprogram = relay.Function([x, y], z)\nmodule = tvm.IRModule.from_expr(program)\nmodule = relay.transform.InferType()(module)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now we have a Relay program that uses myfloat!\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "print(program)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now that we can express our program without errors, let's try running it!\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "try:\n with tvm.transform.PassContext(config={\"tir.disable_vectorize\": True}):\n z_output_myfloat = relay.create_executor(\"graph\", mod=module).evaluate()(x_input, y_input)\n print(\"z: {}\".format(y_myfloat))\nexcept tvm.TVMError as e:\n # Print last line of error\n print(str(e).split(\"\\n\")[-1])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now, trying to compile this program throws an error.\nLet's dissect this error.\n\nThe error is occurring during the process of lowering the custom datatype code to code that TVM can compile and run.\nTVM is telling us that it cannot find a *lowering function* for the ``Cast`` operation, when casting from source type 2 (``float``, in TVM), to destination type 150 (our custom datatype).\nWhen lowering custom datatypes, if TVM encounters an operation over a custom datatype, it loo [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "tvm.target.datatype.register_op(\n tvm.target.datatype.create_lower_func(\n {\n (32, 32): \"FloatToCustom32\", # cast from float32 to myfloat32\n }\n ),\n \"Cast\",\n \"llvm\",\n \"float\",\n \"myfloat\",\n)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "The ``register_op(...)`` call takes a lowering function, and a number of parameters which specify exactly the operation which should be lowered with the provided lowering function.\nIn this case, the arguments we pass specify that this lowering function is for lowering a ``Cast`` from ``float`` to ``myfloat`` for target ``\"llvm\"``.\n\nThe lowering function passed into this call is very general: it should take an operation of the specified type (in this case, `Cast`) and return [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# We can now re-try running the program:\ntry:\n with tvm.transform.PassContext(config={\"tir.disable_vectorize\": True}):\n z_output_myfloat = relay.create_executor(\"graph\", mod=module).evaluate()(x_input, y_input)\n print(\"z: {}\".format(z_output_myfloat))\nexcept tvm.TVMError as e:\n # Print last line of error\n print(str(e).split(\"\\n\")[-1])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This new error tells us that the ``Add`` lowering function is not found, which is good news, as it's no longer complaining about the ``Cast``!\nWe know what to do from here: we just need to register the lowering functions for the other operations in our program.\n\nNote that for ``Add``, ``create_lower_func`` takes in a dict where the key is an integer.\nFor ``Cast`` operations, we require a 2-tuple to specify the ``src_bit_length`` and the ``dest_bit_length``,\nwhile for all ot [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "tvm.target.datatype.register_op(\n tvm.target.datatype.create_lower_func({32: \"Custom32Add\"}),\n \"Add\",\n \"llvm\",\n \"myfloat\",\n)\ntvm.target.datatype.register_op(\n tvm.target.datatype.create_lower_func({(32, 32): \"Custom32ToFloat\"}),\n \"Cast\",\n \"llvm\",\n \"myfloat\",\n \"float\",\n)\n\n# Now, we can run our program without errors.\nwith tvm.transform.PassContext(config={\"tir.disable_vectorize\": True}):\n z_output_myfloat = relay.c [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Running Models With Custom Datatypes\n\nWe will first choose the model which we would like to run with myfloat.\nIn this case we use [Mobilenet](https://arxiv.org/abs/1704.04861).\nWe choose Mobilenet due to its small size.\nIn this alpha state of the Bring Your Own Datatypes framework, we have not implemented any software optimizations for running software emulations of custom datatypes; the result is poor performance due to many calls into our datatype emulation library.\n\ [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def get_mobilenet():\n dshape = (1, 3, 224, 224)\n from mxnet.gluon.model_zoo.vision import get_model\n\n block = get_model(\"mobilenet0.25\", pretrained=True)\n shape_dict = {\"data\": dshape}\n return relay.frontend.from_mxnet(block, shape_dict)\n\n\ndef get_cat_image():\n from tvm.contrib.download import download_testdata\n from PIL import Image\n\n url = \"https://gist.githubusercontent.com/zhreshold/bcda4716699ac97ea44f791c24310193/raw/fa7ef0e9c9a5da [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "It's easy to execute MobileNet with native TVM:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "ex = tvm.relay.create_executor(\"graph\", mod=module, params=params)\ninput = get_cat_image()\nresult = ex.evaluate()(input).numpy()\n# print first 10 elements\nprint(result.flatten()[:10])"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now, we would like to change the model to use myfloat internally. To do so, we need to convert the network. To do this, we first define a function which will help us convert tensors:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def convert_ndarray(dst_dtype, array):\n \"\"\"Converts an NDArray into the specified datatype\"\"\"\n x = relay.var(\"x\", shape=array.shape, dtype=str(array.dtype))\n cast = relay.Function([x], x.astype(dst_dtype))\n with tvm.transform.PassContext(config={\"tir.disable_vectorize\": True}):\n return relay.create_executor(\"graph\").evaluate(cast)(array)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Now, to actually convert the entire network, we have written [a pass in Relay](https://github.com/gussmith23/tvm/blob/ea174c01c54a2529e19ca71e125f5884e728da6e/python/tvm/relay/frontend/change_datatype.py#L21) which simply converts all nodes within the model to use the new datatype.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from tvm.relay.frontend.change_datatype import ChangeDatatype\n\nsrc_dtype = \"float32\"\ndst_dtype = \"custom[myfloat]32\"\n\nmodule = relay.transform.InferType()(module)\n\n# Currently, custom datatypes only work if you run simplify_inference beforehand\nmodule = tvm.relay.transform.SimplifyInference()(module)\n\n# Run type inference before changing datatype\nmodule = tvm.relay.transform.InferType()(module)\n\n# Change datatype from float to myfloat and re-infer types\ncdtype [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "When we attempt to run the model, we get a familiar error telling us that more functions need to be registered for myfloat.\n\nBecause this is a neural network, many more operations are required.\nHere, we register all the needed functions:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "tvm.target.datatype.register_op(\n tvm.target.datatype.create_lower_func({32: \"FloatToCustom32\"}),\n \"FloatImm\",\n \"llvm\",\n \"myfloat\",\n)\n\ntvm.target.datatype.register_op(\n tvm.target.datatype.lower_ite, \"Call\", \"llvm\", \"myfloat\", intrinsic_name=\"tir.if_then_else\"\n)\n\ntvm.target.datatype.register_op(\n tvm.target.datatype.lower_call_pure_extern,\n \"Call\",\n \"llvm\",\n \"myfloat\",\n intrinsic_name=\"tir.call_pure_extern\",\n [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Note we are making use of two new functions: ``register_min_func`` and ``create_min_lower_func``.\n\n``register_min_func`` takes in an integer ``num_bits`` for the bit length, and should return an operation\nrepresenting the minimum finite representable value for the custom data type with the specified bit length.\n\nSimilar to ``register_op`` and ``create_lower_func``, the ``create_min_lower_func`` handles the general case\nwhere the minimum representable custom datatype value [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Vectorization is not implemented with custom datatypes.\nwith tvm.transform.PassContext(config={\"tir.disable_vectorize\": True}):\n result_myfloat = relay.create_executor(mod=module).evaluate(expr)(input, **params)\n result_myfloat = convert_ndarray(src_dtype, result_myfloat).numpy()\n # print first 10 elements\n print(result_myfloat.flatten()[:10])\n\n# Again, note that the output using 32-bit myfloat exactly the same as 32-bit floats,\n# because myfloat is exact [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.18"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/b1b0cbd807166348a0eabbad6bfbbdaf/tune_relay_vta.ipynb b/docs/_downloads/b1b0cbd807166348a0eabbad6bfbbdaf/tune_relay_vta.ipynb
deleted file mode 100644
index f5aa4ecbab..0000000000
--- a/docs/_downloads/b1b0cbd807166348a0eabbad6bfbbdaf/tune_relay_vta.ipynb
+++ /dev/null
@@ -1,186 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# Auto-tuning a convolutional network on VTA\n**Author**: [Lianmin Zheng](https://github.com/merrymercy), [Thierry Moreau](https://homes.cs.washington.edu/~moreau/)\n\nAuto-tuning for a specific accelerator design is critical for getting the best\nperformance for any given operator. This is a tutorial showcases how to tune a\nwhole convolutional network on VTA.\n\nThe operator implementation for VTA in TVM is written in template form.\nThe template has many tunable knobs (tile [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install dependencies\nTo use the autotvm package in tvm, we need to install some extra dependencies.\n(change \"3\" to \"2\" if you use python2):\n\n```bash\npip3 install --user psutil xgboost tornado mxnet requests \"Pillow<7\" cloudpickle\n```\nTo make TVM run faster during tuning, it is recommended to use cython\nas FFI of TVM. In the root directory of TVM, execute\n(change \"3\" to \"2\" if you use python2):\n\n```bash\npip3 install --user cython\nsudo make cython3\n```\n [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "import os\nimport sys\n\nfrom mxnet.gluon.model_zoo import vision\nimport numpy as np\nfrom PIL import Image\n\nfrom tvm import topi\nimport tvm\nfrom tvm import te\nfrom tvm import rpc, autotvm, relay\nfrom tvm.contrib import graph_executor, utils, download\nfrom tvm.autotvm.measure.measure_methods import request_remote\nfrom tvm.autotvm.tuner import XGBTuner, GATuner, RandomTuner, GridSearchTuner\n\nimport vta\nfrom vta.testing import simulator\nfrom vta.top import graph_pack"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Compile network\nPerform vta-specific compilation with Relay from a Gluon model\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def compile_network(env, target, model, start_pack, stop_pack):\n\n # Populate the shape and data type dictionary\n dtype_dict = {\"data\": \"float32\"}\n shape_dict = {\"data\": (env.BATCH, 3, 224, 224)}\n\n # Get off the shelf gluon model, and convert to relay\n gluon_model = vision.get_model(model, pretrained=True)\n mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)\n\n # Update shape and type dictionary\n shape_dict.update({k: v.shape f [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Start RPC Tracker\nTVM uses an RPC session to communicate with Pynq boards.\nDuring tuning, the tuner will send the generated code to the board and\nmeasure the speed of code on the board.\n\nTo scale up tuning, TVM uses an RPC Tracker to manage multiple devices.\nThe RPC Tracker is a centralized controller node. We can register all devices to\nthe tracker. For example, if we have 10 Pynq boards, we can register all of them\nto the tracker, and run 10 measurements in parallel [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Register devices to RPC Tracker\nNow we can register our devices to the tracker. The first step is to\nbuild the TVM runtime for the Pynq devices.\n\nFollow `vta-index`\nto build the TVM runtime on the device. Then register the device to the tracker with:\n\n```bash\npython -m tvm.exec.rpc_server --tracker=[HOST_IP]:9190 --key=pynq\n```\n(replace :code:`[HOST_IP]` with the IP address of your host machine)\n\nAfter registering devices, we can confirm it by querying the rpc_tra [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Set Tuning Options\nBefore tuning, we should apply some configurations.\nHere we use an Pynq-Z1 board as an example.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Tracker host and port can be set by your environment\ntracker_host = os.environ.get(\"TVM_TRACKER_HOST\", \"127.0.0.1\")\ntracker_port = int(os.environ.get(\"TVM_TRACKER_PORT\", 9190))\n\n# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file\nenv = vta.get_env()\n\n# This target is used for cross compilation. You can query it by :code:`gcc -v` on your device.\n# Set ``device=arm_cpu`` to run inference on the CPU\n# or ``device=vta`` to run inference on the [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>How to set tuning options\n\n In general, the default values provided here work well.\n If you have enough time budget, you can set :code:`n_trial`, :code:`early_stopping`\n to larger values, makes the tuning run for longer.\n If your device is under-powered or your conv2d operators are large, consider\n setting a longer timeout.</p></div>\n\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Begin Tuning\nNow we can extract tuning tasks from the network and begin tuning.\nHere, we provide a simple utility function to tune a list of tasks.\nThis function is just an initial implementation which tunes them in sequential order.\nWe will introduce a more sophisticated tuning scheduler in the future.\n\nGiven that the tuning will be done on Pynq FPGA boards, make sure that\nthe ```TARGET`` entry in the ``vta_config.json`` file is set to ``pynq``.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# You can skip the implementation of this function for this tutorial.\ndef tune_tasks(\n tasks,\n measure_option,\n tuner=\"xgb\",\n n_trial=1000,\n early_stopping=None,\n log_filename=\"tuning.log\",\n use_transfer_learning=True,\n):\n\n # create tmp log file\n tmp_log_file = log_filename + \".tmp\"\n if os.path.exists(tmp_log_file):\n os.remove(tmp_log_file)\n\n for i, tsk in enumerate(reversed(tasks)):\n prefix = \"[Task %2d/%2d] [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Register VTA-specific tuning tasks\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def register_vta_tuning_tasks():\n from tvm.autotvm.task import TaskExtractEnv\n\n @tvm.te.tag_scope(tag=topi.tag.ELEMWISE)\n def my_clip(x, a_min, a_max):\n \"\"\"Unlike topi's current clip, put min and max into two stages.\"\"\"\n const_min = tvm.tir.const(a_min, x.dtype)\n const_max = tvm.tir.const(a_max, x.dtype)\n x = te.compute(x.shape, lambda *i: tvm.te.min(x(*i), const_max), name=\"clipA\")\n x = te.compute(x.shape, lambda *i: [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Finally, we launch tuning jobs and evaluate the end-to-end performance.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "def tune_and_evaluate(tuning_opt):\n\n # Register VTA tuning tasks\n register_vta_tuning_tasks()\n\n # Perform task extraction on Relay program\n print(\"Extract tasks...\")\n relay_prog, params = compile_network(env, target, network, start_pack, stop_pack)\n mod = tvm.IRModule.from_expr(relay_prog)\n tasks = autotvm.task.extract_from_program(\n mod,\n params=params,\n ops=(relay.op.get(\"nn.conv2d\"),),\n target=target,\n [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Sample Output\nThe tuning needs to compile many programs and extract feature from them.\nSo a high performance CPU is recommended.\nOne sample output is listed below.\nIt takes about 2 hours on a 16T CPU, and 6 Pynq boards.\n\n```bash\nExtract tasks...\n[Warning] Invalid shape during AutoTVM task creation\nExtracted 10 conv2d tasks:\n Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (32, 16, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>**Experiencing Difficulties?**\n\n The auto tuning module is error-prone. If you always see \" 0.00/ 0.00 GFLOPS\",\n then there must be something wrong.\n\n First, make sure you set the correct configuration of your device.\n Then, you can print debug information by adding these lines in the beginning\n of the script. It will print every measurement result, where you can find useful\n error messages.\n\n```python\nimport log [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.18"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/bc33c0d33026b287306b6ead1a50b04a/tune_relay_arm.ipynb b/docs/_downloads/bc33c0d33026b287306b6ead1a50b04a/tune_relay_arm.ipynb
index 74632cf823..a52439b387 100644
--- a/docs/_downloads/bc33c0d33026b287306b6ead1a50b04a/tune_relay_arm.ipynb
+++ b/docs/_downloads/bc33c0d33026b287306b6ead1a50b04a/tune_relay_arm.ipynb
@@ -51,7 +51,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
+ "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
]
},
{
diff --git a/docs/_downloads/cafefaac0e14b00fd7644da616cab35a/deploy_model_on_nano.ipynb b/docs/_downloads/cafefaac0e14b00fd7644da616cab35a/deploy_model_on_nano.ipynb
index 21080ea8ab..7bbbb7f87b 100644
--- a/docs/_downloads/cafefaac0e14b00fd7644da616cab35a/deploy_model_on_nano.ipynb
+++ b/docs/_downloads/cafefaac0e14b00fd7644da616cab35a/deploy_model_on_nano.ipynb
@@ -47,7 +47,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Prepare the Pre-trained Model\nBack to the host machine, which should have a full TVM installed (with LLVM).\n\nWe will use pre-trained model from\n[MXNet Gluon model zoo](https://mxnet.apache.org/api/python/gluon/model_zoo.html).\nYou can found more details about this part at tutorial `tutorial-from-mxnet`.\n\n"
+ "## Prepare the Pre-trained Model\nBack to the host machine, which should have a full TVM installed (with LLVM).\n\nWe will use pre-trained model from torchvision\n\n"
]
},
{
@@ -58,7 +58,7 @@
},
"outputs": [],
"source": [
- "import sys\n\nfrom mxnet.gluon.model_zoo.vision import get_model\nfrom PIL import Image\nimport numpy as np\n\n# one line to get the model\ntry:\n block = get_model(\"resnet18_v1\", pretrained=True)\nexcept RuntimeError:\n print(\"Downloads from mxnet no longer supported\", file=sys.stderr)\n sys.exit(0)"
+ "import torch\nimport torchvision\nfrom PIL import Image\nimport numpy as np\n\n# one line to get the model\nmodel_name = \"resnet18\"\nmodel = getattr(torchvision.models, model_name)(pretrained=True)\nmodel = model.eval()\n\n# We grab the TorchScripted model via tracing\ninput_shape = [1, 3, 224, 224]\ninput_data = torch.randn(input_shape)\nscripted_model = torch.jit.trace(model, input_data).eval()"
]
},
{
@@ -112,7 +112,7 @@
},
"outputs": [],
"source": [
- "# We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon\nshape_dict = {\"data\": x.shape}\nmod, params = relay.frontend.from_mxnet(block, shape_dict)\n# we want a probability so add a softmax operator\nfunc = mod[\"main\"]\nfunc = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)"
+ "input_name = \"input0\"\nshape_list = [(input_name, x.shape)]\nmod, params = relay.frontend.from_pytorch(scripted_model, shape_list)\n# we want a probability so add a softmax operator\nfunc = mod[\"main\"]\nfunc = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)"
]
},
{
@@ -173,7 +173,7 @@
},
"outputs": [],
"source": [
- "# obtain an RPC session from remote device.\nif local_demo:\n remote = rpc.LocalSession()\nelse:\n # The following is my environment, change this to the IP address of your target device\n host = \"192.168.1.11\"\n port = 9091\n remote = rpc.connect(host, port)\n\n# upload the library to remote device and load it\nremote.upload(lib_fname)\nrlib = remote.load_module(\"net.tar\")\n\n# create the remote runtime module\nif local_demo:\n dev = remote.cpu(0)\nelse:\n [...]
+ "# obtain an RPC session from remote device.\nif local_demo:\n remote = rpc.LocalSession()\nelse:\n # The following is my environment, change this to the IP address of your target device\n host = \"192.168.1.11\"\n port = 9091\n remote = rpc.connect(host, port)\n\n# upload the library to remote device and load it\nremote.upload(lib_fname)\nrlib = remote.load_module(\"net.tar\")\n\n# create the remote runtime module\nif local_demo:\n dev = remote.cpu(0)\nelse:\n [...]
]
}
],
diff --git a/docs/_downloads/d1434e80dd27eef6b1c9cbaa13f1197b/tune_relay_cuda.ipynb b/docs/_downloads/d1434e80dd27eef6b1c9cbaa13f1197b/tune_relay_cuda.ipynb
index aed8b26b7e..aa956df2df 100644
--- a/docs/_downloads/d1434e80dd27eef6b1c9cbaa13f1197b/tune_relay_cuda.ipynb
+++ b/docs/_downloads/d1434e80dd27eef6b1c9cbaa13f1197b/tune_relay_cuda.ipynb
@@ -51,7 +51,7 @@
},
"outputs": [],
"source": [
- "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
+ "def get_network(name, batch_size):\n \"\"\"Get the symbol definition and random weight of a network\"\"\"\n input_shape = (batch_size, 3, 224, 224)\n output_shape = (batch_size, 1000)\n\n if \"resnet\" in name:\n n_layer = int(name.split(\"-\")[1])\n mod, params = relay.testing.resnet.get_workload(\n num_layers=n_layer, batch_size=batch_size, dtype=dtype\n )\n elif \"vgg\" in name:\n n_layer = int(name.split(\"-\")[1])\n [...]
]
},
{
diff --git a/docs/_downloads/d7b7e50e9f5b4ff04d55a56e52314c71/tune_relay_vta.py b/docs/_downloads/d7b7e50e9f5b4ff04d55a56e52314c71/tune_relay_vta.py
deleted file mode 100644
index b5de247883..0000000000
--- a/docs/_downloads/d7b7e50e9f5b4ff04d55a56e52314c71/tune_relay_vta.py
+++ /dev/null
@@ -1,538 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-Auto-tuning a convolutional network on VTA
-==========================================
-**Author**: `Lianmin Zheng <https://github.com/merrymercy>`_, `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-Auto-tuning for a specific accelerator design is critical for getting the best
-performance for any given operator. This is a tutorial showcases how to tune a
-whole convolutional network on VTA.
-
-The operator implementation for VTA in TVM is written in template form.
-The template has many tunable knobs (tile factor, virtual threads, etc).
-We will tune all convolution operators in the neural network. After tuning,
-we produce a log file which stores the best schedule parameters for all tuned
-operators. When the TVM compiler compiles these operators, it will query this
-log file to get the best knob parameters.
-
-"""
-
-######################################################################
-# Install dependencies
-# --------------------
-# To use the autotvm package in tvm, we need to install some extra dependencies.
-# (change "3" to "2" if you use python2):
-#
-# .. code-block:: bash
-#
-# pip3 install --user psutil xgboost tornado mxnet requests "Pillow<7" cloudpickle
-#
-# To make TVM run faster during tuning, it is recommended to use cython
-# as FFI of TVM. In the root directory of TVM, execute
-# (change "3" to "2" if you use python2):
-#
-# .. code-block:: bash
-#
-# pip3 install --user cython
-# sudo make cython3
-#
-# Now return to python code. Import packages.
-
-import os
-import sys
-
-from mxnet.gluon.model_zoo import vision
-import numpy as np
-from PIL import Image
-
-from tvm import topi
-import tvm
-from tvm import te
-from tvm import rpc, autotvm, relay
-from tvm.contrib import graph_executor, utils, download
-from tvm.autotvm.measure.measure_methods import request_remote
-from tvm.autotvm.tuner import XGBTuner, GATuner, RandomTuner, GridSearchTuner
-
-import vta
-from vta.testing import simulator
-from vta.top import graph_pack
-
-#################################################################
-# Compile network
-# ---------------
-# Perform vta-specific compilation with Relay from a Gluon model
-
-
-def compile_network(env, target, model, start_pack, stop_pack):
-
- # Populate the shape and data type dictionary
- dtype_dict = {"data": "float32"}
- shape_dict = {"data": (env.BATCH, 3, 224, 224)}
-
- # Get off the shelf gluon model, and convert to relay
- gluon_model = vision.get_model(model, pretrained=True)
- mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)
-
- # Update shape and type dictionary
- shape_dict.update({k: v.shape for k, v in params.items()})
- dtype_dict.update({k: str(v.dtype) for k, v in params.items()})
-
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with tvm.transform.PassContext(opt_level=3):
- with relay.quantize.qconfig(global_scale=8.0, skip_conv_layers=[0]):
- mod = relay.quantize.quantize(mod, params=params)
-
- # Perform graph packing and constant folding for VTA target
- if target.device_name == "vta":
- assert env.BLOCK_IN == env.BLOCK_OUT
- relay_prog = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=start_pack,
- stop_name=stop_pack,
- )
-
- return relay_prog, params
-
-
-#################################################################
-# Start RPC Tracker
-# -----------------
-# TVM uses an RPC session to communicate with Pynq boards.
-# During tuning, the tuner will send the generated code to the board and
-# measure the speed of code on the board.
-#
-# To scale up tuning, TVM uses an RPC Tracker to manage multiple devices.
-# The RPC Tracker is a centralized controller node. We can register all devices to
-# the tracker. For example, if we have 10 Pynq boards, we can register all of them
-# to the tracker, and run 10 measurements in parallel, accelerating the tuning process.
-#
-# To start an RPC tracker, run this command on the host machine. The tracker is
-# required during the whole tuning process, so we need to open a new terminal for
-# this command:
-#
-# .. code-block:: bash
-#
-# python -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9190
-#
-# The expected output is:
-#
-# .. code-block:: bash
-#
-# INFO:RPCTracker:bind to 0.0.0.0:9190
-
-#################################################################
-# Register devices to RPC Tracker
-# -----------------------------------
-# Now we can register our devices to the tracker. The first step is to
-# build the TVM runtime for the Pynq devices.
-#
-# Follow :ref:`vta-index`
-# to build the TVM runtime on the device. Then register the device to the tracker with:
-#
-# .. code-block:: bash
-#
-# python -m tvm.exec.rpc_server --tracker=[HOST_IP]:9190 --key=pynq
-#
-# (replace :code:`[HOST_IP]` with the IP address of your host machine)
-#
-# After registering devices, we can confirm it by querying the rpc_tracker:
-#
-# .. code-block:: bash
-#
-# python -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190
-#
-# For example, if we have 6 Pynq boards and 11 Raspberry Pi 3B,
-# the output can be
-#
-# .. code-block:: bash
-#
-# Queue Status
-# ----------------------------------
-# key total free pending
-# ----------------------------------
-# pynq 6 6 0
-# rpi3b 11 11 0
-# ----------------------------------
-#
-# You can register multiple devices to the tracker to accelerate tuning.
-
-###########################################
-# Set Tuning Options
-# ------------------
-# Before tuning, we should apply some configurations.
-# Here we use an Pynq-Z1 board as an example.
-
-# Tracker host and port can be set by your environment
-tracker_host = os.environ.get("TVM_TRACKER_HOST", "127.0.0.1")
-tracker_port = int(os.environ.get("TVM_TRACKER_PORT", 9190))
-
-# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
-env = vta.get_env()
-
-# This target is used for cross compilation. You can query it by :code:`gcc -v` on your device.
-# Set ``device=arm_cpu`` to run inference on the CPU
-# or ``device=vta`` to run inference on the FPGA.
-device = "vta"
-target = env.target if device == "vta" else env.target_vta_cpu
-
-# Name of Gluon model to compile
-# The ``start_pack`` and ``stop_pack`` labels indicate where
-# to start and end the graph packing relay pass: in other words
-# where to start and finish offloading to VTA.
-network = "resnet18_v1"
-start_pack = "nn.max_pool2d"
-stop_pack = "nn.global_avg_pool2d"
-
-# Tuning option
-log_file = "%s.%s.log" % (device, network)
-tuning_option = {
- "log_filename": log_file,
- "tuner": "random",
- "n_trial": 1000,
- "early_stopping": None,
- "measure_option": autotvm.measure_option(
- builder=autotvm.LocalBuilder(),
- runner=autotvm.RPCRunner(
- env.TARGET,
- host=tracker_host,
- port=tracker_port,
- number=5,
- timeout=60,
- module_loader=vta.module_loader(),
- # check_correctness=True, # TODO: re-enable when check_correctness works again.
- ),
- ),
-}
-
-####################################################################
-#
-# .. note:: How to set tuning options
-#
-# In general, the default values provided here work well.
-# If you have enough time budget, you can set :code:`n_trial`, :code:`early_stopping`
-# to larger values, makes the tuning run for longer.
-# If your device is under-powered or your conv2d operators are large, consider
-# setting a longer timeout.
-#
-
-###################################################################
-# Begin Tuning
-# ------------
-# Now we can extract tuning tasks from the network and begin tuning.
-# Here, we provide a simple utility function to tune a list of tasks.
-# This function is just an initial implementation which tunes them in sequential order.
-# We will introduce a more sophisticated tuning scheduler in the future.
-#
-# Given that the tuning will be done on Pynq FPGA boards, make sure that
-# the ```TARGET`` entry in the ``vta_config.json`` file is set to ``pynq``.
-
-
-# You can skip the implementation of this function for this tutorial.
-def tune_tasks(
- tasks,
- measure_option,
- tuner="xgb",
- n_trial=1000,
- early_stopping=None,
- log_filename="tuning.log",
- use_transfer_learning=True,
-):
-
- # create tmp log file
- tmp_log_file = log_filename + ".tmp"
- if os.path.exists(tmp_log_file):
- os.remove(tmp_log_file)
-
- for i, tsk in enumerate(reversed(tasks)):
- prefix = "[Task %2d/%2d] " % (i + 1, len(tasks))
-
- # create tuner
- if tuner == "xgb":
- tuner_obj = XGBTuner(tsk, loss_type="reg")
- elif tuner == "xgb_knob":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="knob")
- elif tuner == "xgb_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="itervar")
- elif tuner == "xgb_curve":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="curve")
- elif tuner == "xgb_rank":
- tuner_obj = XGBTuner(tsk, loss_type="rank")
- elif tuner == "xgb_rank_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="knob")
- elif tuner == "xgb_rank_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="itervar")
- elif tuner == "xgb_rank_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="curve")
- elif tuner == "xgb_rank_binary":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary")
- elif tuner == "xgb_rank_binary_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="knob")
- elif tuner == "xgb_rank_binary_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="itervar")
- elif tuner == "xgb_rank_binary_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="curve")
- elif tuner == "ga":
- tuner_obj = GATuner(tsk, pop_size=50)
- elif tuner == "random":
- tuner_obj = RandomTuner(tsk)
- elif tuner == "gridsearch":
- tuner_obj = GridSearchTuner(tsk)
- else:
- raise ValueError("Invalid tuner: " + tuner)
-
- if use_transfer_learning:
- if os.path.isfile(tmp_log_file):
- tuner_obj.load_history(autotvm.record.load_from_file(tmp_log_file))
-
- # do tuning
- tsk_trial = min(n_trial, len(tsk.config_space))
- tuner_obj.tune(
- n_trial=tsk_trial,
- early_stopping=early_stopping,
- measure_option=measure_option,
- callbacks=[
- autotvm.callback.progress_bar(tsk_trial, prefix=prefix),
- autotvm.callback.log_to_file(tmp_log_file),
- ],
- )
-
- # pick best records to a cache file
- autotvm.record.pick_best(tmp_log_file, log_filename)
- os.remove(tmp_log_file)
-
-
-########################################################################
-# Register VTA-specific tuning tasks
-
-
-def register_vta_tuning_tasks():
- from tvm.autotvm.task import TaskExtractEnv
-
- @tvm.te.tag_scope(tag=topi.tag.ELEMWISE)
- def my_clip(x, a_min, a_max):
- """Unlike topi's current clip, put min and max into two stages."""
- const_min = tvm.tir.const(a_min, x.dtype)
- const_max = tvm.tir.const(a_max, x.dtype)
- x = te.compute(x.shape, lambda *i: tvm.te.min(x(*i), const_max), name="clipA")
- x = te.compute(x.shape, lambda *i: tvm.te.max(x(*i), const_min), name="clipB")
- return x
-
- # init autotvm env to register VTA operator
- TaskExtractEnv()
-
- @autotvm.template("conv2d_packed.vta")
- def _topi_nn_conv2d(*args, **kwargs):
- assert not kwargs, "Do not support kwargs in template function call"
- A, W = args[:2]
-
- with tvm.target.vta():
- res = vta.top.conv2d_packed(*args, **kwargs)
- res = topi.right_shift(res, 8)
- res = my_clip(res, 0, 127)
- res = topi.cast(res, "int8")
-
- if tvm.target.Target.current().device_name == "vta":
- s = vta.top.schedule_conv2d_packed([res])
- else:
- s = te.create_schedule([res.op])
- return s, [A, W, res]
-
-
-########################################################################
-# Finally, we launch tuning jobs and evaluate the end-to-end performance.
-
-
-def tune_and_evaluate(tuning_opt):
-
- # Register VTA tuning tasks
- register_vta_tuning_tasks()
-
- # Perform task extraction on Relay program
- print("Extract tasks...")
- relay_prog, params = compile_network(env, target, network, start_pack, stop_pack)
- mod = tvm.IRModule.from_expr(relay_prog)
- tasks = autotvm.task.extract_from_program(
- mod,
- params=params,
- ops=(relay.op.get("nn.conv2d"),),
- target=target,
- target_host=env.target_host,
- )
-
- # filter out non-packed conv2d task
- tasks = list(filter(lambda t: len(t.args[0][1]) > 4 and "conv" in t.name, tasks))
-
- # We should have extracted 10 convolution tasks
- assert len(tasks) == 10
- print("Extracted {} conv2d tasks:".format(len(tasks)))
- for tsk in tasks:
- inp = tsk.args[0][1]
- wgt = tsk.args[1][1]
- batch = inp[0] * inp[4]
- in_filter = inp[1] * inp[5]
- out_filter = wgt[0] * wgt[4]
- height, width = inp[2], inp[3]
- hkernel, wkernel = wgt[2], wgt[3]
- hstride, wstride = tsk.args[2][0], tsk.args[2][1]
- hpad, wpad = tsk.args[3][0], tsk.args[3][1]
- print(
- "({}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {})".format(
- batch,
- height,
- width,
- in_filter,
- out_filter,
- hkernel,
- wkernel,
- hpad,
- wpad,
- hstride,
- wstride,
- )
- )
-
- # We do not run the tuning in our webpage server since it takes too long.
- # Comment the following line to run it by yourself.
- return
-
- # run tuning tasks
- print("Tuning...")
- tune_tasks(tasks, **tuning_opt)
-
- # evaluate with tuning history
- if env.TARGET != "sim":
- # Get remote from fleet node
- remote = autotvm.measure.request_remote(
- env.TARGET, tracker_host, tracker_port, timeout=10000
- )
- # Reconfigure the JIT runtime and FPGA.
- vta.reconfig_runtime(remote)
- vta.program_fpga(remote, bitstream=None)
- else:
- # In simulation mode, host the RPC server locally.
- remote = rpc.LocalSession()
-
- # compile kernels with history best records
- with autotvm.tophub.context(target, extra_files=[log_file]):
- # Compile network
- print("Compile...")
- if target.device_name != "vta":
- with tvm.transform.PassContext(opt_level=3, disabled_pass={"AlterOpLayout"}):
- lib = relay.build(
- relay_prog, target=target, params=params, target_host=env.target_host
- )
- else:
- with vta.build_config(opt_level=3, disabled_pass={"AlterOpLayout"}):
- lib = relay.build(
- relay_prog, target=target, params=params, target_host=env.target_host
- )
-
- # Export library
- print("Upload...")
- temp = utils.tempdir()
- lib.export_library(temp.relpath("graphlib.tar"))
- remote.upload(temp.relpath("graphlib.tar"))
- lib = remote.load_module("graphlib.tar")
-
- # Generate the graph executor
- ctx = remote.ext_dev(0) if device == "vta" else remote.cpu(0)
- m = graph_executor.GraphModule(lib["default"](ctx))
-
- # upload parameters to device
- image = tvm.nd.array((np.random.uniform(size=(1, 3, 224, 224))).astype("float32"))
- m.set_input("data", image)
-
- # evaluate
- print("Evaluate inference time cost...")
- timer = m.module.time_evaluator("run", ctx, number=1, repeat=10)
- tcost = timer()
- prof_res = np.array(tcost.results) * 1000 # convert to millisecond
- print(
- "Mean inference time (std dev): %.2f ms (%.2f ms)"
- % (np.mean(prof_res), np.std(prof_res))
- )
-
-
-# Run the tuning and evaluate the results
-try:
- tune_and_evaluate(tuning_option)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
-######################################################################
-# Sample Output
-# -------------
-# The tuning needs to compile many programs and extract feature from them.
-# So a high performance CPU is recommended.
-# One sample output is listed below.
-# It takes about 2 hours on a 16T CPU, and 6 Pynq boards.
-#
-# .. code-block:: bash
-#
-# Extract tasks...
-# [Warning] Invalid shape during AutoTVM task creation
-# Extracted 10 conv2d tasks:
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (32, 16, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 16, 14, 14, 1, 16, 'int8'), (32, 16, 1, 1, 16, 16, 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 8, 28, 28, 1, 16), 'int8'), ('TENSOR', (16, 8, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 8, 28, 28, 1, 16, 'int8'), (16, 8, 1, 1, 16, 16, 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 4, 56, 56, 1, 16), 'int8'), ('TENSOR', (8, 4, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 4, 56, 56, 1, 16, 'int8'), (8, 4, 1, 1, 16, 16, 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 4, 56, 56, 1, 16), 'int8'), ('TENSOR', (4, 4, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 4, 56, 56, 1, 16, 'int8'), (4, 4, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 8, 28, 28, 1, 16), 'int8'), ('TENSOR', (8, 8, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 8, 28, 28, 1, 16, 'int8'), (8, 8, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 4, 56, 56, 1, 16), 'int8'), ('TENSOR', (8, 4, 3, 3, 16, 16), 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 4, 56, 56, 1, 16, 'int8'), (8, 4, 3, 3, 16, 16, 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (16, 16, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 16, 14, 14, 1, 16, 'int8'), (16, 16, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 8, 28, 28, 1, 16), 'int8'), ('TENSOR', (16, 8, 3, 3, 16, 16), 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 8, 28, 28, 1, 16, 'int8'), (16, 8, 3, 3, 16, 16, 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 32, 7, 7, 1, 16), 'int8'), ('TENSOR', (32, 32, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 32, 7, 7, 1, 16, 'int8'), (32, 32, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (32, 16, 3, 3, 16, 16), 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 16, 14, 14, 1, 16, 'int8'), (32, 16, 3, 3, 16, 16, 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
-# Tuning...
-# [Task 1/10] Current/Best: 0.72/ 23.24 GFLOPS | Progress: (480/1000) | 640.31 s Done.
-# [Task 2/10] Current/Best: 0.00/ 27.69 GFLOPS | Progress: (576/1000) | 810.09 s Done.
-# [Task 3/10] Current/Best: 0.00/ 22.97 GFLOPS | Progress: (1000/1000) | 1125.37 s Done.
-# [Task 4/10] Current/Best: 0.00/ 31.26 GFLOPS | Progress: (1000/1000) | 1025.52 s Done.
-# [Task 5/10] Current/Best: 0.00/ 15.15 GFLOPS | Progress: (1000/1000) | 1236.58 s Done.
-# [Task 6/10] Current/Best: 0.00/ 22.74 GFLOPS | Progress: (1000/1000) | 906.60 s Done.
-# [Task 7/10] Current/Best: 0.00/ 15.27 GFLOPS | Progress: (1000/1000) | 1056.25 s Done.
-# [Task 8/10] Current/Best: 0.00/ 2.18 GFLOPS | Progress: (1000/1000) | 2275.29 s Done.
-# [Task 9/10] Current/Best: 2.23/ 3.99 GFLOPS | Progress: (1000/1000) | 2527.25 s Done.
-# [Task 10/10] Current/Best: 1.56/ 6.32 GFLOPS | Progress: (480/1000) | 1304.84 s Done.
-# Compile...
-# Upload...
-# Evaluate inference time cost...
-# Mean inference time (std dev): 621.79 ms (0.14 ms)
-
-######################################################################
-#
-# .. note:: **Experiencing Difficulties?**
-#
-# The auto tuning module is error-prone. If you always see " 0.00/ 0.00 GFLOPS",
-# then there must be something wrong.
-#
-# First, make sure you set the correct configuration of your device.
-# Then, you can print debug information by adding these lines in the beginning
-# of the script. It will print every measurement result, where you can find useful
-# error messages.
-#
-# .. code-block:: python
-#
-# import logging
-# logging.getLogger('autotvm').setLevel(logging.DEBUG)
-#
-# Finally, always feel free to ask our community for help on https://discuss.tvm.apache.org
diff --git a/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py b/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py
index 96df3942ab..169567122f 100644
--- a/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py
+++ b/docs/_downloads/e416b94ca1090b0897c0f6e0df95b911/tune_network_x86.py
@@ -117,19 +117,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32", use_sparse=Fal
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet50_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
elif name == "mlp":
mod, params = relay.testing.mlp.get_workload(
batch_size=batch_size, dtype=dtype, image_shape=image_shape, num_classes=1000
@@ -169,17 +156,13 @@ log_file = "%s-%s-B%d-%s.json" % (network, layout, batch_size, target.kind.name)
# Extract tasks from the network
print("Get model...")
-try:
- mod, params, input_shape, output_shape = get_network(
- network,
- batch_size,
- layout,
- dtype=dtype,
- use_sparse=use_sparse,
- )
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+mod, params, input_shape, output_shape = get_network(
+ network,
+ batch_size,
+ layout,
+ dtype=dtype,
+ use_sparse=use_sparse,
+)
print("Extract tasks...")
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
diff --git a/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py b/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py
index 41e7e8fb41..1f8c0cc13a 100644
--- a/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py
+++ b/docs/_downloads/eafe360d52540634c9eea0fa89e804bd/tune_network_cuda.py
@@ -114,20 +114,6 @@ def get_network(name, batch_size, layout="NHWC", dtype="float32"):
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
-
return mod, params, input_shape, output_shape
@@ -152,11 +138,7 @@ log_file = "%s-%s-B%d-%s.json" % (network, layout, batch_size, target.kind.name)
# Extract tasks from the network
print("Extract tasks...")
-try:
- mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
for idx, task in enumerate(tasks):
diff --git a/docs/_downloads/ee99205e9f2e4f54c0fb7925008a5354/bring_your_own_datatypes.py b/docs/_downloads/ee99205e9f2e4f54c0fb7925008a5354/bring_your_own_datatypes.py
deleted file mode 100644
index e502aff3e0..0000000000
--- a/docs/_downloads/ee99205e9f2e4f54c0fb7925008a5354/bring_your_own_datatypes.py
+++ /dev/null
@@ -1,415 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-Bring Your Own Datatypes to TVM
-===============================
-**Authors**: `Gus Smith <https://github.com/gussmith23>`_, `Andrew Liu <https://github.com/hypercubestart>`_
-
-In this tutorial, we will show you how to utilize the Bring Your Own Datatypes framework to use your own custom datatypes in TVM.
-Note that the Bring Your Own Datatypes framework currently only handles **software emulated versions of datatypes**.
-The framework does not support compiling for custom accelerator datatypes out-of-the-box.
-
-Datatype Libraries
-------------------
-
-The Bring Your Own Datatypes allows users to register their own datatype implementations alongside TVM's native datatypes (such as ``float``).
-In the wild, these datatype implementations often appear as libraries.
-For example:
-
-- `libposit <https://github.com/cjdelisle/libposit>`_, a posit library
-- `Stillwater Universal <https://github.com/stillwater-sc/universal>`_, a library with posits, fixed-point numbers, and other types
-- `SoftFloat <https://github.com/ucb-bar/berkeley-softfloat-3>`_, Berkeley's software implementation of IEEE 754 floating-point
-
-The Bring Your Own Datatypes enables users to plug these datatype implementations into TVM!
-
-In this section, we will use an example library we have already implemented, located at ``3rdparty/byodt/myfloat.cc``.
-This datatype, which we dubbed "myfloat", is really just a IEE-754 float under-the-hood, but it serves a useful example
-to show that any datatype can be used in the BYODT framework.
-
-Setup
------
-
-Since we do not use any 3rdparty library, there is no setup needed.
-
-If you would like to try this with your own datatype library, first bring the library's functions into the process space with ``CDLL``:
-
-.. code-block:: python
-
- ctypes.CDLL('my-datatype-lib.so', ctypes.RTLD_GLOBAL)
-"""
-
-
-######################
-# A Simple TVM Program
-# --------------------
-#
-# We'll begin by writing a simple program in TVM; afterwards, we will re-write it to use custom datatypes.
-import sys
-
-import tvm
-from tvm import relay
-
-# Our basic program: Z = X + Y
-x = relay.var("x", shape=(3,), dtype="float32")
-y = relay.var("y", shape=(3,), dtype="float32")
-z = x + y
-program = relay.Function([x, y], z)
-module = tvm.IRModule.from_expr(program)
-
-######################################################################
-# Now, we create random inputs to feed into this program using numpy:
-
-import numpy as np
-
-np.random.seed(23) # for reproducibility
-
-x_input = np.random.rand(3).astype("float32")
-y_input = np.random.rand(3).astype("float32")
-print("x: {}".format(x_input))
-print("y: {}".format(y_input))
-
-######################################################################
-# Finally, we're ready to run the program:
-
-z_output = relay.create_executor(mod=module).evaluate()(x_input, y_input)
-print("z: {}".format(z_output))
-
-######################################################################
-# Adding Custom Datatypes
-# -----------------------
-# Now, we will do the same, but we will use a custom datatype for our intermediate computation.
-#
-# We use the same input variables ``x`` and ``y`` as above, but before adding ``x + y``, we first cast both ``x`` and ``y`` to a custom datatype via the ``relay.cast(...)`` call.
-#
-# Note how we specify the custom datatype: we indicate it using the special ``custom[...]`` syntax.
-# Additionally, note the "32" after the datatype: this is the bitwidth of the custom datatype. This tells TVM that each instance of ``myfloat`` is 32 bits wide.
-
-try:
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- x_myfloat = relay.cast(x, dtype="custom[myfloat]32")
- y_myfloat = relay.cast(y, dtype="custom[myfloat]32")
- z_myfloat = x_myfloat + y_myfloat
- z = relay.cast(z_myfloat, dtype="float32")
-except tvm.TVMError as e:
- # Print last line of error
- print(str(e).split("\n")[-1])
-
-######################################################################
-# Trying to generate this program throws an error from TVM.
-# TVM does not know how to handle any custom datatype out of the box!
-# We first have to register the custom type with TVM, giving it a name and a type code:
-
-tvm.target.datatype.register("myfloat", 150)
-
-######################################################################
-# Note that the type code, 150, is currently chosen manually by the user.
-# See ``TVMTypeCode::kCustomBegin`` in `include/tvm/runtime/c_runtime_api.h <https://github.com/apache/tvm/blob/main/include/tvm/runtime/data_type.h>`_.
-# Now we can generate our program again:
-
-x_myfloat = relay.cast(x, dtype="custom[myfloat]32")
-y_myfloat = relay.cast(y, dtype="custom[myfloat]32")
-z_myfloat = x_myfloat + y_myfloat
-z = relay.cast(z_myfloat, dtype="float32")
-program = relay.Function([x, y], z)
-module = tvm.IRModule.from_expr(program)
-module = relay.transform.InferType()(module)
-
-######################################################################
-# Now we have a Relay program that uses myfloat!
-print(program)
-
-######################################################################
-# Now that we can express our program without errors, let's try running it!
-try:
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- z_output_myfloat = relay.create_executor("graph", mod=module).evaluate()(x_input, y_input)
- print("z: {}".format(y_myfloat))
-except tvm.TVMError as e:
- # Print last line of error
- print(str(e).split("\n")[-1])
-
-######################################################################
-# Now, trying to compile this program throws an error.
-# Let's dissect this error.
-#
-# The error is occurring during the process of lowering the custom datatype code to code that TVM can compile and run.
-# TVM is telling us that it cannot find a *lowering function* for the ``Cast`` operation, when casting from source type 2 (``float``, in TVM), to destination type 150 (our custom datatype).
-# When lowering custom datatypes, if TVM encounters an operation over a custom datatype, it looks for a user-registered *lowering function*, which tells it how to lower the operation to an operation over datatypes it understands.
-# We have not told TVM how to lower ``Cast`` operations for our custom datatypes; thus, the source of this error.
-#
-# To fix this error, we simply need to specify a lowering function:
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func(
- {
- (32, 32): "FloatToCustom32", # cast from float32 to myfloat32
- }
- ),
- "Cast",
- "llvm",
- "float",
- "myfloat",
-)
-
-######################################################################
-# The ``register_op(...)`` call takes a lowering function, and a number of parameters which specify exactly the operation which should be lowered with the provided lowering function.
-# In this case, the arguments we pass specify that this lowering function is for lowering a ``Cast`` from ``float`` to ``myfloat`` for target ``"llvm"``.
-#
-# The lowering function passed into this call is very general: it should take an operation of the specified type (in this case, `Cast`) and return another operation which only uses datatypes which TVM understands.
-#
-# In the general case, we expect users to implement operations over their custom datatypes using calls to an external library.
-# In our example, our ``myfloat`` library implements a ``Cast`` from ``float`` to 32-bit ``myfloat`` in the function ``FloatToCustom32``.
-# To provide for the general case, we have made a helper function, ``create_lower_func(...)``,
-# which does just this: given a dictionary, it replaces the given operation with a ``Call`` to the appropriate function name provided based on the op and the bit widths.
-# It additionally removes usages of the custom datatype by storing the custom datatype in an opaque ``uint`` of the appropriate width; in our case, a ``uint32_t``.
-# For more information, see `the source code <https://github.com/apache/tvm/blob/main/python/tvm/target/datatype.py>`_.
-
-# We can now re-try running the program:
-try:
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- z_output_myfloat = relay.create_executor("graph", mod=module).evaluate()(x_input, y_input)
- print("z: {}".format(z_output_myfloat))
-except tvm.TVMError as e:
- # Print last line of error
- print(str(e).split("\n")[-1])
-
-######################################################################
-# This new error tells us that the ``Add`` lowering function is not found, which is good news, as it's no longer complaining about the ``Cast``!
-# We know what to do from here: we just need to register the lowering functions for the other operations in our program.
-#
-# Note that for ``Add``, ``create_lower_func`` takes in a dict where the key is an integer.
-# For ``Cast`` operations, we require a 2-tuple to specify the ``src_bit_length`` and the ``dest_bit_length``,
-# while for all other operations, the bit length is the same between the operands so we only require one integer to specify ``bit_length``.
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Add"}),
- "Add",
- "llvm",
- "myfloat",
-)
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({(32, 32): "Custom32ToFloat"}),
- "Cast",
- "llvm",
- "myfloat",
- "float",
-)
-
-# Now, we can run our program without errors.
-with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- z_output_myfloat = relay.create_executor(mod=module).evaluate()(x_input, y_input)
-print("z: {}".format(z_output_myfloat))
-
-print("x:\t\t{}".format(x_input))
-print("y:\t\t{}".format(y_input))
-print("z (float32):\t{}".format(z_output))
-print("z (myfloat32):\t{}".format(z_output_myfloat))
-
-# Perhaps as expected, the ``myfloat32`` results and ``float32`` are exactly the same!
-
-######################################################################
-# Running Models With Custom Datatypes
-# ------------------------------------
-#
-# We will first choose the model which we would like to run with myfloat.
-# In this case we use `Mobilenet <https://arxiv.org/abs/1704.04861>`_.
-# We choose Mobilenet due to its small size.
-# In this alpha state of the Bring Your Own Datatypes framework, we have not implemented any software optimizations for running software emulations of custom datatypes; the result is poor performance due to many calls into our datatype emulation library.
-#
-# First let us define two helper functions to get the mobilenet model and a cat image.
-
-
-def get_mobilenet():
- dshape = (1, 3, 224, 224)
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("mobilenet0.25", pretrained=True)
- shape_dict = {"data": dshape}
- return relay.frontend.from_mxnet(block, shape_dict)
-
-
-def get_cat_image():
- from tvm.contrib.download import download_testdata
- from PIL import Image
-
- url = "https://gist.githubusercontent.com/zhreshold/bcda4716699ac97ea44f791c24310193/raw/fa7ef0e9c9a5daea686d6473a62aacd1a5885849/cat.png"
- dst = "cat.png"
- real_dst = download_testdata(url, dst, module="data")
- img = Image.open(real_dst).resize((224, 224))
- # CoreML's standard model image format is BGR
- img_bgr = np.array(img)[:, :, ::-1]
- img = np.transpose(img_bgr, (2, 0, 1))[np.newaxis, :]
- return np.asarray(img, dtype="float32")
-
-
-try:
- module, params = get_mobilenet()
-except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
-######################################################################
-# It's easy to execute MobileNet with native TVM:
-
-ex = tvm.relay.create_executor("graph", mod=module, params=params)
-input = get_cat_image()
-result = ex.evaluate()(input).numpy()
-# print first 10 elements
-print(result.flatten()[:10])
-
-######################################################################
-# Now, we would like to change the model to use myfloat internally. To do so, we need to convert the network. To do this, we first define a function which will help us convert tensors:
-
-
-def convert_ndarray(dst_dtype, array):
- """Converts an NDArray into the specified datatype"""
- x = relay.var("x", shape=array.shape, dtype=str(array.dtype))
- cast = relay.Function([x], x.astype(dst_dtype))
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- return relay.create_executor("graph").evaluate(cast)(array)
-
-
-######################################################################
-# Now, to actually convert the entire network, we have written `a pass in Relay <https://github.com/gussmith23/tvm/blob/ea174c01c54a2529e19ca71e125f5884e728da6e/python/tvm/relay/frontend/change_datatype.py#L21>`_ which simply converts all nodes within the model to use the new datatype.
-
-from tvm.relay.frontend.change_datatype import ChangeDatatype
-
-src_dtype = "float32"
-dst_dtype = "custom[myfloat]32"
-
-module = relay.transform.InferType()(module)
-
-# Currently, custom datatypes only work if you run simplify_inference beforehand
-module = tvm.relay.transform.SimplifyInference()(module)
-
-# Run type inference before changing datatype
-module = tvm.relay.transform.InferType()(module)
-
-# Change datatype from float to myfloat and re-infer types
-cdtype = ChangeDatatype(src_dtype, dst_dtype)
-expr = cdtype.visit(module["main"])
-module = tvm.relay.transform.InferType()(module)
-
-# We also convert the parameters:
-params = {k: convert_ndarray(dst_dtype, v) for k, v in params.items()}
-
-# We also need to convert our input:
-input = convert_ndarray(dst_dtype, input)
-
-# Finally, we can try to run the converted model:
-try:
- # Vectorization is not implemented with custom datatypes.
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- result_myfloat = tvm.relay.create_executor("graph", mod=module).evaluate(expr)(
- input, **params
- )
-except tvm.TVMError as e:
- print(str(e).split("\n")[-1])
-
-######################################################################
-# When we attempt to run the model, we get a familiar error telling us that more functions need to be registered for myfloat.
-#
-# Because this is a neural network, many more operations are required.
-# Here, we register all the needed functions:
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "FloatToCustom32"}),
- "FloatImm",
- "llvm",
- "myfloat",
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.lower_ite, "Call", "llvm", "myfloat", intrinsic_name="tir.if_then_else"
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.lower_call_pure_extern,
- "Call",
- "llvm",
- "myfloat",
- intrinsic_name="tir.call_pure_extern",
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Mul"}),
- "Mul",
- "llvm",
- "myfloat",
-)
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Div"}),
- "Div",
- "llvm",
- "myfloat",
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Sqrt"}),
- "Call",
- "llvm",
- "myfloat",
- intrinsic_name="tir.sqrt",
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Sub"}),
- "Sub",
- "llvm",
- "myfloat",
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Exp"}),
- "Call",
- "llvm",
- "myfloat",
- intrinsic_name="tir.exp",
-)
-
-tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Max"}),
- "Max",
- "llvm",
- "myfloat",
-)
-
-tvm.target.datatype.register_min_func(
- tvm.target.datatype.create_min_lower_func({32: "MinCustom32"}, "myfloat"),
- "myfloat",
-)
-
-######################################################################
-# Note we are making use of two new functions: ``register_min_func`` and ``create_min_lower_func``.
-#
-# ``register_min_func`` takes in an integer ``num_bits`` for the bit length, and should return an operation
-# representing the minimum finite representable value for the custom data type with the specified bit length.
-#
-# Similar to ``register_op`` and ``create_lower_func``, the ``create_min_lower_func`` handles the general case
-# where the minimum representable custom datatype value is implemented using calls to an external library.
-#
-# Now we can finally run the model:
-
-# Vectorization is not implemented with custom datatypes.
-with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- result_myfloat = relay.create_executor(mod=module).evaluate(expr)(input, **params)
- result_myfloat = convert_ndarray(src_dtype, result_myfloat).numpy()
- # print first 10 elements
- print(result_myfloat.flatten()[:10])
-
-# Again, note that the output using 32-bit myfloat exactly the same as 32-bit floats,
-# because myfloat is exactly a float!
-np.testing.assert_array_equal(result, result_myfloat)
diff --git a/docs/_images/sphx_glr_bring_your_own_datatypes_thumb.png b/docs/_images/sphx_glr_bring_your_own_datatypes_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_bring_your_own_datatypes_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_deploy_classification_001.png b/docs/_images/sphx_glr_deploy_classification_001.png
deleted file mode 100644
index 635b87a9eb..0000000000
Binary files a/docs/_images/sphx_glr_deploy_classification_001.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_deploy_classification_thumb.png b/docs/_images/sphx_glr_deploy_classification_thumb.png
deleted file mode 100644
index 2db1354f86..0000000000
Binary files a/docs/_images/sphx_glr_deploy_classification_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_deploy_quantized_thumb.png b/docs/_images/sphx_glr_deploy_quantized_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_deploy_quantized_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_from_mxnet_001.png b/docs/_images/sphx_glr_from_mxnet_001.png
deleted file mode 100644
index 635b87a9eb..0000000000
Binary files a/docs/_images/sphx_glr_from_mxnet_001.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_from_mxnet_thumb.png b/docs/_images/sphx_glr_from_mxnet_thumb.png
deleted file mode 100644
index 2db1354f86..0000000000
Binary files a/docs/_images/sphx_glr_from_mxnet_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_tune_alu_vta_thumb.png b/docs/_images/sphx_glr_tune_alu_vta_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_tune_alu_vta_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_tune_relay_vta_thumb.png b/docs/_images/sphx_glr_tune_relay_vta_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_tune_relay_vta_thumb.png and /dev/null differ
diff --git a/docs/_sources/how_to/compile_models/from_darknet.rst.txt b/docs/_sources/how_to/compile_models/from_darknet.rst.txt
index 2a9ef36c66..a866052fd9 100644
--- a/docs/_sources/how_to/compile_models/from_darknet.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_darknet.rst.txt
@@ -318,7 +318,7 @@ The process is no different from other examples.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 37.614 seconds)
+ **Total running time of the script:** ( 1 minutes 31.637 seconds)
.. _sphx_glr_download_how_to_compile_models_from_darknet.py:
diff --git a/docs/_sources/how_to/compile_models/from_mxnet.rst.txt b/docs/_sources/how_to/compile_models/from_mxnet.rst.txt
deleted file mode 100644
index e5a953d9b5..0000000000
--- a/docs/_sources/how_to/compile_models/from_mxnet.rst.txt
+++ /dev/null
@@ -1,285 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "how_to/compile_models/from_mxnet.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_how_to_compile_models_from_mxnet.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/4bbcfcce3c35b0b795a42c998ceb3770/from_mxnet.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_how_to_compile_models_from_mxnet.py:
-
-
-.. _tutorial-from-mxnet:
-
-Compile MXNet Models
-====================
-**Author**: `Joshua Z. Zhang <https://zhreshold.github.io/>`_, `Kazutaka Morita <https://github.com/kazum>`_
-
-This article is an introductory tutorial to deploy mxnet models with Relay. To begin, we must install `mxnet`:
-
-.. code-block:: bash
-
- %%shell
- pip install mxnet
-
-or please refer to official installation guide.
-https://mxnet.apache.org/versions/master/install/index.html
-
-.. GENERATED FROM PYTHON SOURCE LINES 35-43
-
-.. code-block:: default
-
-
- # some standard imports
- import sys
- import mxnet as mx
- import tvm
- import tvm.relay as relay
- import numpy as np
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 47-50
-
-Download Resnet18 model from Gluon Model Zoo
----------------------------------------------
-In this section, we download a pretrained imagenet model and classify an image.
-
-.. GENERATED FROM PYTHON SOURCE LINES 50-92
-
-.. code-block:: default
-
- from tvm.contrib.download import download_testdata
- from mxnet.gluon.model_zoo.vision import get_model
- from PIL import Image
- from matplotlib import pyplot as plt
-
- try:
- block = get_model("resnet18_v1", pretrained=True)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
- img_url = "https://github.com/dmlc/mxnet.js/blob/main/data/cat.png?raw=true"
- img_name = "cat.png"
- synset_url = "".join(
- [
- "https://gist.githubusercontent.com/zhreshold/",
- "4d0b62f3d01426887599d4f7ede23ee5/raw/",
- "596b27d23537e5a1b5751d2b0481ef172f58b539/",
- "imagenet1000_clsid_to_human.txt",
- ]
- )
- synset_name = "imagenet1000_clsid_to_human.txt"
- img_path = download_testdata(img_url, "cat.png", module="data")
- synset_path = download_testdata(synset_url, synset_name, module="data")
- with open(synset_path) as f:
- synset = eval(f.read())
- image = Image.open(img_path).resize((224, 224))
- plt.imshow(image)
- plt.show()
-
-
- def transform_image(image):
- image = np.array(image) - np.array([123.0, 117.0, 104.0])
- image /= np.array([58.395, 57.12, 57.375])
- image = image.transpose((2, 0, 1))
- image = image[np.newaxis, :]
- return image
-
-
- x = transform_image(image)
- print("x", x.shape)
-
-
-
-
-.. image-sg:: /how_to/compile_models/images/sphx_glr_from_mxnet_001.png
- :alt: from mxnet
- :srcset: /how_to/compile_models/images/sphx_glr_from_mxnet_001.png
- :class: sphx-glr-single-img
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Downloading /workspace/.mxnet/models/resnet18_v1-a0666292.zip4df00681-7f99-4d44-a973-8c68eb4b2843 from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/models/resnet18_v1-a0666292.zip...
- x (1, 3, 224, 224)
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 93-98
-
-Compile the Graph
------------------
-Now we would like to port the Gluon model to a portable computational graph.
-It's as easy as several lines.
-We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
-
-.. GENERATED FROM PYTHON SOURCE LINES 98-104
-
-.. code-block:: default
-
- shape_dict = {"data": x.shape}
- mod, params = relay.frontend.from_mxnet(block, shape_dict)
- ## we want a probability so add a softmax operator
- func = mod["main"]
- func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 105-106
-
-now compile the graph
-
-.. GENERATED FROM PYTHON SOURCE LINES 106-110
-
-.. code-block:: default
-
- target = "cuda"
- with tvm.transform.PassContext(opt_level=3):
- lib = relay.build(func, target, params=params)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 111-114
-
-Execute the portable graph on TVM
----------------------------------
-Now, we would like to reproduce the same forward computation using TVM.
-
-.. GENERATED FROM PYTHON SOURCE LINES 114-128
-
-.. code-block:: default
-
- from tvm.contrib import graph_executor
-
- dev = tvm.cuda(0)
- dtype = "float32"
- m = graph_executor.GraphModule(lib["default"](dev))
- # set inputs
- m.set_input("data", tvm.nd.array(x.astype(dtype)))
- # execute
- m.run()
- # get outputs
- tvm_output = m.get_output(0)
- top1 = np.argmax(tvm_output.numpy()[0])
- print("TVM prediction top-1:", top1, synset[top1])
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- TVM prediction top-1: 282 tiger cat
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 129-133
-
-Use MXNet symbol with pretrained weights
-----------------------------------------
-MXNet often use `arg_params` and `aux_params` to store network parameters
-separately, here we show how to use these weights with existing API
-
-.. GENERATED FROM PYTHON SOURCE LINES 133-148
-
-.. code-block:: default
-
- def block2symbol(block):
- data = mx.sym.Variable("data")
- sym = block(data)
- args = {}
- auxs = {}
- for k, v in block.collect_params().items():
- args[k] = mx.nd.array(v.data().asnumpy())
- return sym, args, auxs
-
-
- mx_sym, args, auxs = block2symbol(block)
- # usually we would save/load it as checkpoint
- mx.model.save_checkpoint("resnet18_v1", 0, mx_sym, args, auxs)
- # there are 'resnet18_v1-0000.params' and 'resnet18_v1-symbol.json' on disk
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 149-150
-
-for a normal mxnet model, we start from here
-
-.. GENERATED FROM PYTHON SOURCE LINES 150-154
-
-.. code-block:: default
-
- mx_sym, args, auxs = mx.model.load_checkpoint("resnet18_v1", 0)
- # now we use the same API to get Relay computation graph
- mod, relay_params = relay.frontend.from_mxnet(mx_sym, shape_dict, arg_params=args, aux_params=auxs)
- # repeat the same steps to run this model using TVM
-
-
-
-
-
-
-
-
-.. _sphx_glr_download_how_to_compile_models_from_mxnet.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: from_mxnet.py <from_mxnet.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: from_mxnet.ipynb <from_mxnet.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/how_to/compile_models/from_oneflow.rst.txt b/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
index 6b03a95cf5..979064a99a 100644
--- a/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
@@ -121,7 +121,7 @@ Load a pretrained OneFlow model and save model
.. code-block:: none
Downloading: "https://oneflow-public.oss-cn-beijing.aliyuncs.com/model_zoo/flowvision/classification/ResNet/resnet18.zip" to /workspace/.oneflow/flowvision_cache/resnet18.zip
- 0%| | 0.00/41.5M [00:00<?, ?B/s] 15%|#5 | 6.33M/41.5M [00:00<00:01, 18.5MB/s] 20%|#9 | 8.09M/41.5M [00:00<00:02, 14.0MB/s] 35%|###4 | 14.3M/41.5M [00:00<00:01, 19.7MB/s] 39%|###9 | 16.2M/41.5M [00:00<00:01, 19.0MB/s] 58%|#####7 | 24.0M/41.5M [00:01<00:00, 26.7MB/s] 77%|#######7 | 32.0M/41.5M [00:01<00:00, 34.0MB/s] 85%|########5 | 35.3M/41.5M [00:01<00:00, 33.8MB/s] 93%|#########2| 38.6M/41.5M [00:01<00:00, 26.6MB/s] 99%|#########9| 41.3M/41.5M [00:01<00:00, 26.8MB/s] 100%|##########| 41.5M/41.5M [00:01<00:00, 25.2MB/s]
+ 0%| | 0.00/41.5M [00:00<?, ?B/s] 15%|#5 | 6.33M/41.5M [00:00<00:00, 60.2MB/s] 29%|##9 | 12.1M/41.5M [00:00<00:00, 44.4MB/s] 40%|###9 | 16.5M/41.5M [00:00<00:00, 42.7MB/s] 54%|#####3 | 22.3M/41.5M [00:00<00:00, 45.5MB/s] 64%|######4 | 26.7M/41.5M [00:00<00:00, 44.8MB/s] 77%|#######7 | 32.0M/41.5M [00:00<00:00, 42.6MB/s] 89%|########9 | 37.0M/41.5M [00:00<00:00, 45.3MB/s] 100%|#########9| 41.4M/41.5M [00:01<00:00, 40.2MB/s] 100%|##########| 41.5M/41.5M [00:01<00:00, 43.2MB/s]
diff --git a/docs/_sources/how_to/compile_models/from_paddle.rst.txt b/docs/_sources/how_to/compile_models/from_paddle.rst.txt
index e08fa02add..8df75c9e33 100644
--- a/docs/_sources/how_to/compile_models/from_paddle.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_paddle.rst.txt
@@ -207,11 +207,6 @@ Look up prediction top 1 index in 1000 class synset.
-.. rst-class:: sphx-glr-timing
-
- **Total running time of the script:** ( 1 minutes 3.683 seconds)
-
-
.. _sphx_glr_download_how_to_compile_models_from_paddle.py:
.. only:: html
diff --git a/docs/_sources/how_to/compile_models/from_pytorch.rst.txt b/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
index d3223467a6..424eb23e5e 100644
--- a/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
@@ -101,7 +101,7 @@ Load a pretrained PyTorch model
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /workspace/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
- 0%| | 0.00/44.7M [00:00<?, ?B/s] 14%|#4 | 6.30M/44.7M [00:00<00:00, 60.0MB/s] 27%|##6 | 12.0M/44.7M [00:00<00:00, 45.0MB/s] 37%|###6 | 16.5M/44.7M [00:00<00:00, 34.6MB/s] 54%|#####3 | 24.0M/44.7M [00:00<00:00, 41.7MB/s] 68%|######7 | 30.3M/44.7M [00:00<00:00, 40.8MB/s] 77%|#######6 | 34.3M/44.7M [00:00<00:00, 34.0MB/s] 86%|########5 | 38.3M/44.7M [00:01<00:00, 35.0MB/s] 94%|#########3| 41.8M/44.7M [00:01<00:00, 31.0MB/s] 100%|##########| 44.7M/44.7M [00:01<00:00, 37.8MB/s]
+ 0%| | 0.00/44.7M [00:00<?, ?B/s] 14%|#4 | 6.30M/44.7M [00:00<00:00, 55.4MB/s] 26%|##5 | 11.6M/44.7M [00:00<00:00, 46.1MB/s] 36%|###5 | 16.0M/44.7M [00:00<00:01, 30.0MB/s] 54%|#####3 | 24.0M/44.7M [00:00<00:00, 40.6MB/s] 71%|#######1 | 31.9M/44.7M [00:00<00:00, 51.6MB/s] 84%|########4 | 37.5M/44.7M [00:00<00:00, 46.4MB/s] 95%|#########5| 42.5M/44.7M [00:01<00:00, 41.4MB/s] 100%|##########| 44.7M/44.7M [00:01<00:00, 40.1MB/s]
diff --git a/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt b/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
index 732a1784b1..0a9b891a71 100644
--- a/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
@@ -430,7 +430,7 @@ Run the corresponding model on tensorflow
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 32.793 seconds)
+ **Total running time of the script:** ( 1 minutes 27.530 seconds)
.. _sphx_glr_download_how_to_compile_models_from_tensorflow.py:
diff --git a/docs/_sources/how_to/compile_models/index.rst.txt b/docs/_sources/how_to/compile_models/index.rst.txt
index a6a74f929a..995dde51ec 100644
--- a/docs/_sources/how_to/compile_models/index.rst.txt
+++ b/docs/_sources/how_to/compile_models/index.rst.txt
@@ -49,23 +49,6 @@ formats. These how-tos demostrate how to import models using the Python API.
</div>
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This article is an introductory tutorial to deploy mxnet models with Relay. To begin, we must i...">
-
-.. only:: html
-
- .. image:: /how_to/compile_models/images/thumb/sphx_glr_from_mxnet_thumb.png
- :alt: Compile MXNet Models
-
- :ref:`sphx_glr_how_to_compile_models_from_mxnet.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Compile MXNet Models</div>
- </div>
-
-
.. raw:: html
<div class="sphx-glr-thumbcontainer" tooltip="This article is an introductory tutorial to deploy ONNX models with Relay.">
@@ -195,7 +178,6 @@ formats. These how-tos demostrate how to import models using the Python API.
/how_to/compile_models/from_pytorch
/how_to/compile_models/from_tensorflow
- /how_to/compile_models/from_mxnet
/how_to/compile_models/from_onnx
/how_to/compile_models/from_keras
/how_to/compile_models/from_tflite
diff --git a/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt b/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
index 00a70c63f4..e6e93f71f0 100644
--- a/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
@@ -5,26 +5,24 @@
Computation times
=================
-**07:24.760** total execution time for **how_to_compile_models** files:
+**06:21.908** total execution time for **how_to_compile_models** files:
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_darknet.py` (``from_darknet.py``) | 01:37.614 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_darknet.py` (``from_darknet.py``) | 01:31.637 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_tensorflow.py` (``from_tensorflow.py``) | 01:32.793 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_tensorflow.py` (``from_tensorflow.py``) | 01:27.530 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_paddle.py` (``from_paddle.py``) | 01:03.683 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_paddle.py` (``from_paddle.py``) | 00:58.592 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_oneflow.py` (``from_oneflow.py``) | 00:43.886 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_oneflow.py` (``from_oneflow.py``) | 00:40.650 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_coreml.py` (``from_coreml.py``) | 00:39.151 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_coreml.py` (``from_coreml.py``) | 00:36.355 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_mxnet.py` (``from_mxnet.py``) | 00:35.009 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_pytorch.py` (``from_pytorch.py``) | 00:27.236 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_pytorch.py` (``from_pytorch.py``) | 00:29.143 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_keras.py` (``from_keras.py``) | 00:26.295 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_keras.py` (``from_keras.py``) | 00:28.215 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_tflite.py` (``from_tflite.py``) | 00:10.822 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_tflite.py` (``from_tflite.py``) | 00:12.347 | 0.0 MB |
-+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_onnx.py` (``from_onnx.py``) | 00:02.919 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_onnx.py` (``from_onnx.py``) | 00:02.791 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
index 82cf289ce3..ec5e7d1bc9 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
@@ -673,7 +673,7 @@ well as provides information about the model's performance
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 4121.6299 4121.0362 4128.0218 4118.5082 2.5144
+ 4005.7703 4006.1128 4013.0085 3995.7190 5.6147
@@ -681,7 +681,7 @@ well as provides information about the model's performance
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 20.092 seconds)
+ **Total running time of the script:** ( 1 minutes 16.927 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_model_on_adreno.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
index ef00b4d806..df25ace705 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
@@ -127,7 +127,7 @@ Make a Keras Resnet50 Model
.. code-block:: none
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/resnet/resnet50_weights_tf_dim_ordering_tf_kernels.h5
- 8192/102967424 [..............................] - ETA: 0s 2785280/102967424 [..............................] - ETA: 2s 8519680/102967424 [=>............................] - ETA: 1s 16769024/102967424 [===>..........................] - ETA: 0s 25157632/102967424 [======>.......................] - ETA: 0s 33546240/102967424 [========>.....................] - ETA: 0s 42778624/102967424 [===========>..................] - ETA: 0s 50323456/102967424 [=============>................] - ETA: 0s
58531840/102967424 [================>.............] - ETA: 0s 64798720/102967424 [=================>............] - ETA: 0s 67100672/102967424 [==================>...........] - ETA: 0s 74801152/102967424 [====================>.........] - ETA: 0s 77643776/102967424 [=====================>........] - ETA: 0s 83877888/102967424 [=======================>......] - ETA: 0s 92110848/102967424 [=========================>....] - ETA: 0s 96231424/102967424 [===========================>..] -
ETA: 0s 100646912/102967424 [============================>.] - ETA: 0s 102850560/102967424 [============================>.] - ETA: 0s 102967424/102967424 [==============================] - 1s 0us/step
+ 8192/102967424 [..............................] - ETA: 0s 6635520/102967424 [>.............................] - ETA: 1s 8159232/102967424 [=>............................] - ETA: 2s 8380416/102967424 [=>............................] - ETA: 3s 15024128/102967424 [===>..........................] - ETA: 2s 16769024/102967424 [===>..........................] - ETA: 2s 23412736/102967424 [=====>........................] - ETA: 2s 25157632/102967424 [======>.......................] - ETA: 2s
33546240/102967424 [========>.....................] - ETA: 1s 40189952/102967424 [==========>...................] - ETA: 1s 41934848/102967424 [===========>..................] - ETA: 1s 48799744/102967424 [=============>................] - ETA: 1s 50323456/102967424 [=============>................] - ETA: 1s 56967168/102967424 [===============>..............] - ETA: 1s 58712064/102967424 [================>.............] - ETA: 1s 65421312/102967424 [==================>...........] -
ETA: 0s 67100672/102967424 [==================>...........] - ETA: 0s 71352320/102967424 [===================>..........] - ETA: 0s 75489280/102967424 [====================>.........] - ETA: 0s 79675392/102967424 [======================>.......] - ETA: 0s 82124800/102967424 [======================>.......] - ETA: 0s 83877888/102967424 [=======================>......] - ETA: 0s 90521600/102967424 [=========================>....] - ETA: 0s 92266496/102967424
[=========================>....] - ETA: 0s 98910208/102967424 [===========================>..] - ETA: 0s 100646912/102967424 [============================>.] - ETA: 0s 102850560/102967424 [============================>.] - ETA: 0s 102967424/102967424 [==============================] - 3s 0us/step
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
index f199b9c724..0f8cafe8fc 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
@@ -437,7 +437,7 @@ Execute on TVM
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 16.4012 16.3867 16.8650 15.8796 0.3185
+ 13.8952 13.8335 14.5689 13.6890 0.2441
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_nano.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_nano.rst.txt
index 2056b4e1a2..f59430abe4 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_nano.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_nano.rst.txt
@@ -114,38 +114,47 @@ successfully on your device.
INFO:RPCServer:bind to 0.0.0.0:9091
-.. GENERATED FROM PYTHON SOURCE LINES 101-108
+.. GENERATED FROM PYTHON SOURCE LINES 101-106
Prepare the Pre-trained Model
-----------------------------
Back to the host machine, which should have a full TVM installed (with LLVM).
-We will use pre-trained model from
-`MXNet Gluon model zoo <https://mxnet.apache.org/api/python/gluon/model_zoo.html>`_.
-You can found more details about this part at tutorial :ref:`tutorial-from-mxnet`.
+We will use pre-trained model from torchvision
-.. GENERATED FROM PYTHON SOURCE LINES 108-122
+.. GENERATED FROM PYTHON SOURCE LINES 106-122
.. code-block:: default
- import sys
-
- from mxnet.gluon.model_zoo.vision import get_model
+ import torch
+ import torchvision
from PIL import Image
import numpy as np
# one line to get the model
- try:
- block = get_model("resnet18_v1", pretrained=True)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+ model_name = "resnet18"
+ model = getattr(torchvision.models, model_name)(pretrained=True)
+ model = model.eval()
+
+ # We grab the TorchScripted model via tracing
+ input_shape = [1, 3, 224, 224]
+ input_data = torch.randn(input_shape)
+ scripted_model = torch.jit.trace(model, input_data).eval()
+.. rst-class:: sphx-glr-script-out
+
+ .. code-block:: none
+
+ /venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
+ warnings.warn(
+ /venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
+ warnings.warn(msg)
+
@@ -220,9 +229,9 @@ It's as easy as several lines.
.. code-block:: default
- # We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
- shape_dict = {"data": x.shape}
- mod, params = relay.frontend.from_mxnet(block, shape_dict)
+ input_name = "input0"
+ shape_list = [(input_name, x.shape)]
+ mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
# we want a probability so add a softmax operator
func = mod["main"]
func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
@@ -231,6 +240,15 @@ It's as easy as several lines.
+.. rst-class:: sphx-glr-script-out
+
+ .. code-block:: none
+
+ /workspace/python/tvm/relay/frontend/pytorch_utils.py:47: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
+ return LooseVersion(torch_ver) > ver
+ /venv/apache-tvm-py3.8/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
+ other = LooseVersion(other)
+
@@ -350,7 +368,7 @@ to the remote device.
module = runtime.GraphModule(rlib["default"](dev))
# set input data
- module.set_input("data", tvm.nd.array(x.astype("float32")))
+ module.set_input(input_name, tvm.nd.array(x.astype("float32")))
# run
module.run()
# get output
@@ -366,7 +384,7 @@ to the remote device.
.. code-block:: none
- TVM prediction top-1: tiger cat
+ TVM prediction top-1: tabby, tabby cat
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_rasp.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_rasp.rst.txt
index b256fa9a71..b978e5e6a2 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_rasp.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_rasp.rst.txt
@@ -109,38 +109,47 @@ successfully on your device.
INFO:root:RPCServer: bind to 0.0.0.0:9090
-.. GENERATED FROM PYTHON SOURCE LINES 94-101
+.. GENERATED FROM PYTHON SOURCE LINES 94-99
Prepare the Pre-trained Model
-----------------------------
Back to the host machine, which should have a full TVM installed (with LLVM).
-We will use pre-trained model from
-`MXNet Gluon model zoo <https://mxnet.apache.org/api/python/gluon/model_zoo.html>`_.
-You can found more details about this part at tutorial :ref:`tutorial-from-mxnet`.
+We will use pre-trained model from torchvision
-.. GENERATED FROM PYTHON SOURCE LINES 101-115
+.. GENERATED FROM PYTHON SOURCE LINES 99-115
.. code-block:: default
- import sys
-
- from mxnet.gluon.model_zoo.vision import get_model
+ import torch
+ import torchvision
from PIL import Image
import numpy as np
# one line to get the model
- try:
- block = get_model("resnet18_v1", pretrained=True)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+ model_name = "resnet18"
+ model = getattr(torchvision.models, model_name)(pretrained=True)
+ model = model.eval()
+
+ # We grab the TorchScripted model via tracing
+ input_shape = [1, 3, 224, 224]
+ input_data = torch.randn(input_shape)
+ scripted_model = torch.jit.trace(model, input_data).eval()
+.. rst-class:: sphx-glr-script-out
+
+ .. code-block:: none
+
+ /venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
+ warnings.warn(
+ /venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
+ warnings.warn(msg)
+
@@ -207,7 +216,7 @@ the word human can understand.
.. GENERATED FROM PYTHON SOURCE LINES 151-153
-Now we would like to port the Gluon model to a portable computational graph.
+Now we would like to port the PyTorch model to a portable computational graph.
It's as easy as several lines.
.. GENERATED FROM PYTHON SOURCE LINES 153-161
@@ -215,9 +224,9 @@ It's as easy as several lines.
.. code-block:: default
- # We support MXNet static graph(symbol) and HybridBlock in mxnet.gluon
- shape_dict = {"data": x.shape}
- mod, params = relay.frontend.from_mxnet(block, shape_dict)
+ input_name = "input0"
+ shape_list = [(input_name, x.shape)]
+ mod, params = relay.frontend.from_pytorch(scripted_model, shape_list)
# we want a probability so add a softmax operator
func = mod["main"]
func = relay.Function(func.params, relay.nn.softmax(func.body), None, func.type_params, func.attrs)
@@ -226,6 +235,15 @@ It's as easy as several lines.
+.. rst-class:: sphx-glr-script-out
+
+ .. code-block:: none
+
+ /workspace/python/tvm/relay/frontend/pytorch_utils.py:47: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
+ return LooseVersion(torch_ver) > ver
+ /venv/apache-tvm-py3.8/lib/python3.8/site-packages/setuptools/_distutils/version.py:346: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
+ other = LooseVersion(other)
+
@@ -337,7 +355,7 @@ to the remote device.
dev = remote.cpu(0)
module = runtime.GraphModule(rlib["default"](dev))
# set input data
- module.set_input("data", tvm.nd.array(x.astype("float32")))
+ module.set_input(input_name, tvm.nd.array(x.astype("float32")))
# run
module.run()
# get output
@@ -353,7 +371,7 @@ to the remote device.
.. code-block:: none
- TVM prediction top-1: tiger cat
+ TVM prediction top-1: tabby, tabby cat
diff --git a/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt b/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
index 8788849d83..95bc4f1664 100644
--- a/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
@@ -130,7 +130,7 @@ Load pre-trained maskrcnn from torchvision and do tracing
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MaskRCNN_ResNet50_FPN_Weights.COCO_V1`. You can also use `weights=MaskRCNN_ResNet50_FPN_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth" to /workspace/.cache/torch/hub/checkpoints/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth
- 0%| | 0.00/170M [00:00<?, ?B/s] 4%|3 | 6.30M/170M [00:00<00:08, 20.7MB/s] 5%|4 | 8.28M/170M [00:00<00:09, 18.3MB/s] 8%|8 | 14.3M/170M [00:00<00:07, 23.0MB/s] 10%|9 | 16.6M/170M [00:00<00:06, 23.3MB/s] 14%|#4 | 24.0M/170M [00:00<00:05, 29.4MB/s] 19%|#8 | 32.0M/170M [00:01<00:04, 34.0MB/s] 21%|## | 35.2M/170M [00:01<00:04, 32.4MB/s] 24%|##3 | 40.0M/170M [00:01<00:04, 31.1MB/s] 28%|##7 | 47.0M/170M [00:01<00:03, 40.0MB/s] 30%|### | 51.1M/170M [00:01<00:03, 34.5MB/s] 33%|###2 | 56.0M/170M [00:01<00:03, 38.1MB/s] 38%|###7 | 64.0M/170M [00:02<00:02, 43.9MB/s] 42%|####2 | 72.0M/170M [00:02<00:01, 52.4MB/s] 46%|####6 | 78.3M/170M [00:02<00:01, 50.0MB/s] 49%|####9 | 83.3M/170M [00:02<00:01, 48.9MB/s] 52%|#####1 | 88.2M/170M [00:02<00:02, 41.1MB/s] 57%|#####6 | 96.0M/170M [00:02<00:01, 41.2MB/
s] 61%|######1 | 104M/170M [00:02<00:01, 43.7MB/s] 66%|######5 | 112M/170M [00:03<00:01, 49.2MB/s] 71%|####### | 120M/170M [00:03<00:01, 50.8MB/s] 75%|#######5 | 128M/170M [00:03<00:00, 45.2MB/s] 79%|#######9 | 134M/170M [00:03<00:00, 48.4MB/s] 82%|########1 | 139M/170M [00:03<00:00, 49.1MB/s] 85%|########4 | 144M/170M [00:03<00:00, 40.8MB/s] 88%|########8 | 150M/170M [00:03<00:00, 45.1MB/s] 91%|#########1| 155M/170M [00:04<00:00, 39.4MB/s] 94%|#########4| 160M/170M [00:04<00:00, 39.4MB/s] 98%|#########7| 166M/170M [00:04<00:00, 40.1MB/s] 100%|##########| 170M/170M [00:04<00:00, 39.5MB/s]
+ 0%| | 0.00/170M [00:00<?, ?B/s] 4%|3 | 6.30M/170M [00:00<00:07, 22.6MB/s] 6%|5 | 9.76M/170M [00:00<00:06, 27.1MB/s] 8%|8 | 14.3M/170M [00:00<00:05, 31.0MB/s] 10%|# | 17.5M/170M [00:00<00:05, 27.4MB/s] 13%|#3 | 22.3M/170M [00:00<00:05, 25.9MB/s] 15%|#4 | 24.9M/170M [00:01<00:06, 24.3MB/s] 19%|#8 | 32.0M/170M [00:01<00:04, 30.2MB/s] 24%|##3 | 40.0M/170M [00:01<00:03, 37.8MB/s] 27%|##7 | 46.3M/170M [00:01<00:03, 36.3MB/s] 29%|##9 | 49.8M/170M [00:01<00:03, 34.7MB/s] 33%|###2 | 56.0M/170M [00:01<00:02, 40.1MB/s] 37%|###6 | 62.3M/170M [00:01<00:02, 41.2MB/s] 39%|###9 | 66.3M/170M [00:02<00:03, 34.9MB/s] 42%|####2 | 72.0M/170M [00:02<00:02, 39.4MB/s] 46%|####6 | 78.3M/170M [00:02<00:02, 41.8MB/s] 49%|####8 | 82.5M/170M [00:02<00:02, 38.5MB/s] 54%|#####4 | 91.8M/170M [00:02<00:01, 52.3MB/
s] 57%|#####7 | 97.2M/170M [00:02<00:01, 42.0MB/s] 61%|######1 | 104M/170M [00:02<00:01, 47.2MB/s] 66%|######5 | 112M/170M [00:03<00:01, 48.4MB/s] 71%|####### | 120M/170M [00:03<00:00, 56.3MB/s] 74%|#######4 | 126M/170M [00:03<00:00, 54.4MB/s] 78%|#######7 | 132M/170M [00:03<00:00, 51.0MB/s] 81%|######## | 137M/170M [00:03<00:00, 45.1MB/s] 85%|########4 | 144M/170M [00:03<00:00, 51.3MB/s] 89%|########9 | 152M/170M [00:03<00:00, 50.0MB/s] 93%|#########3| 158M/170M [00:04<00:00, 50.1MB/s] 96%|#########6| 163M/170M [00:04<00:00, 46.4MB/s] 99%|#########8| 168M/170M [00:04<00:00, 35.4MB/s] 100%|##########| 170M/170M [00:04<00:00, 39.7MB/s]
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/nn/functional.py:3912: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
(torch.floor((input.size(i + 2).float() * torch.tensor(scale_factors[i], dtype=torch.float32)).float()))
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/ops/boxes.py:157: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
@@ -292,7 +292,7 @@ Get boxes with score larger than 0.9
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 3 minutes 38.224 seconds)
+ **Total running time of the script:** ( 3 minutes 22.657 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_object_detection_pytorch.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt b/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
index cffb956a5d..169b113471 100644
--- a/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
@@ -227,7 +227,7 @@ training. Other models require a full post training calibration.
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MobileNet_V2_Weights.IMAGENET1K_V1`. You can also use `weights=MobileNet_V2_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/mobilenet_v2-b0353104.pth" to /workspace/.cache/torch/hub/checkpoints/mobilenet_v2-b0353104.pth
- 0%| | 0.00/13.6M [00:00<?, ?B/s] 47%|####6 | 6.30M/13.6M [00:00<00:00, 30.0MB/s] 68%|######7 | 9.17M/13.6M [00:00<00:00, 29.9MB/s] 90%|########9 | 12.2M/13.6M [00:00<00:00, 27.6MB/s] 100%|##########| 13.6M/13.6M [00:00<00:00, 31.2MB/s]
+ 0%| | 0.00/13.6M [00:00<?, ?B/s] 59%|#####8 | 7.99M/13.6M [00:00<00:00, 58.4MB/s] 100%|##########| 13.6M/13.6M [00:00<00:00, 39.8MB/s]
@@ -409,7 +409,7 @@ Here we give an example of how to measure performance of TVM compiled models.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 88.9235 88.8774 89.8110 88.6164 0.2135
+ 86.5491 86.0988 90.4162 85.6121 1.1157
@@ -457,7 +457,7 @@ TODO
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 30.277 seconds)
+ **Total running time of the script:** ( 1 minutes 24.914 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_prequantized.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt b/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
index bad772385b..fce74cd0c8 100644
--- a/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
@@ -423,7 +423,7 @@ Here we give an example of how to measure performance of TVM compiled models.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 110.1117 110.0563 113.8865 109.2087 0.5471
+ 107.1608 106.5409 120.8698 104.3406 2.5093
diff --git a/docs/_sources/how_to/deploy_models/deploy_quantized.rst.txt b/docs/_sources/how_to/deploy_models/deploy_quantized.rst.txt
deleted file mode 100644
index ae72bdbdfe..0000000000
--- a/docs/_sources/how_to/deploy_models/deploy_quantized.rst.txt
+++ /dev/null
@@ -1,289 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "how_to/deploy_models/deploy_quantized.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_how_to_deploy_models_deploy_quantized.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/a269cb38341b190be980a0bd3ea8a625/deploy_quantized.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_how_to_deploy_models_deploy_quantized.py:
-
-
-Deploy a Quantized Model on Cuda
-================================
-**Author**: `Wuwei Lin <https://github.com/vinx13>`_
-
-This article is an introductory tutorial of automatic quantization with TVM.
-Automatic quantization is one of the quantization modes in TVM. More details on
-the quantization story in TVM can be found
-`here <https://discuss.tvm.apache.org/t/quantization-story/3920>`_.
-In this tutorial, we will import a GluonCV pre-trained model on ImageNet to
-Relay, quantize the Relay model and then perform the inference.
-
-.. GENERATED FROM PYTHON SOURCE LINES 29-47
-
-.. code-block:: default
-
-
- import logging
- import os
- import sys
-
- import tvm
- from tvm import te
- from tvm import relay
- import mxnet as mx
- from tvm.contrib.download import download_testdata
- from mxnet import gluon
-
-
- batch_size = 1
- model_name = "resnet18_v1"
- target = "cuda"
- dev = tvm.device(target)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 48-52
-
-Prepare the Dataset
--------------------
-We will demonstrate how to prepare the calibration dataset for quantization.
-We first download the validation set of ImageNet and pre-process the dataset.
-
-.. GENERATED FROM PYTHON SOURCE LINES 52-83
-
-.. code-block:: default
-
- calibration_rec = download_testdata(
- "http://data.mxnet.io.s3-website-us-west-1.amazonaws.com/data/val_256_q90.rec",
- "val_256_q90.rec",
- )
-
-
- def get_val_data(num_workers=4):
- mean_rgb = [123.68, 116.779, 103.939]
- std_rgb = [58.393, 57.12, 57.375]
-
- def batch_fn(batch):
- return batch.data[0].asnumpy(), batch.label[0].asnumpy()
-
- img_size = 299 if model_name == "inceptionv3" else 224
- val_data = mx.io.ImageRecordIter(
- path_imgrec=calibration_rec,
- preprocess_threads=num_workers,
- shuffle=False,
- batch_size=batch_size,
- resize=256,
- data_shape=(3, img_size, img_size),
- mean_r=mean_rgb[0],
- mean_g=mean_rgb[1],
- mean_b=mean_rgb[2],
- std_r=std_rgb[0],
- std_g=std_rgb[1],
- std_b=std_rgb[2],
- )
- return val_data, batch_fn
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 84-87
-
-The calibration dataset should be an iterable object. We define the
-calibration dataset as a generator object in Python. In this tutorial, we
-only use a few samples for calibration.
-
-.. GENERATED FROM PYTHON SOURCE LINES 87-101
-
-.. code-block:: default
-
-
- calibration_samples = 10
-
-
- def calibrate_dataset():
- val_data, batch_fn = get_val_data()
- val_data.reset()
- for i, batch in enumerate(val_data):
- if i * batch_size >= calibration_samples:
- break
- data, _ = batch_fn(batch)
- yield {"data": data}
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 102-105
-
-Import the model
-----------------
-We use the Relay MxNet frontend to import a model from the Gluon model zoo.
-
-.. GENERATED FROM PYTHON SOURCE LINES 105-113
-
-.. code-block:: default
-
- def get_model():
- gluon_model = gluon.model_zoo.vision.get_model(model_name, pretrained=True)
- img_size = 299 if model_name == "inceptionv3" else 224
- data_shape = (batch_size, 3, img_size, img_size)
- mod, params = relay.frontend.from_mxnet(gluon_model, {"data": data_shape})
- return mod, params
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 114-135
-
-Quantize the Model
-------------------
-In quantization, we need to find the scale for each weight and intermediate
-feature map tensor of each layer.
-
-For weights, the scales are directly calculated based on the value of the
-weights. Two modes are supported: `power2` and `max`. Both modes find the
-maximum value within the weight tensor first. In `power2` mode, the maximum
-is rounded down to power of two. If the scales of both weights and
-intermediate feature maps are power of two, we can leverage bit shifting for
-multiplications. This make it computationally more efficient. In `max` mode,
-the maximum is used as the scale. Without rounding, `max` mode might have
-better accuracy in some cases. When the scales are not powers of two, fixed
-point multiplications will be used.
-
-For intermediate feature maps, we can find the scales with data-aware
-quantization. Data-aware quantization takes a calibration dataset as the
-input argument. Scales are calculated by minimizing the KL divergence between
-distribution of activation before and after quantization.
-Alternatively, we can also use pre-defined global scales. This saves the time
-for calibration. But the accuracy might be impacted.
-
-.. GENERATED FROM PYTHON SOURCE LINES 135-147
-
-.. code-block:: default
-
-
-
- def quantize(mod, params, data_aware):
- if data_aware:
- with relay.quantize.qconfig(calibrate_mode="kl_divergence", weight_scale="max"):
- mod = relay.quantize.quantize(mod, params, dataset=calibrate_dataset())
- else:
- with relay.quantize.qconfig(calibrate_mode="global_scale", global_scale=8.0):
- mod = relay.quantize.quantize(mod, params)
- return mod
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 148-151
-
-Run Inference
--------------
-We create a Relay VM to build and execute the model.
-
-.. GENERATED FROM PYTHON SOURCE LINES 151-173
-
-.. code-block:: default
-
- def run_inference(mod):
- model = relay.create_executor("vm", mod, dev, target).evaluate()
- val_data, batch_fn = get_val_data()
- for i, batch in enumerate(val_data):
- data, label = batch_fn(batch)
- prediction = model(data)
- if i > 10: # only run inference on a few samples in this tutorial
- break
-
-
- def main():
- try:
- mod, params = get_model()
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- return
- mod = quantize(mod, params, data_aware=True)
- run_inference(mod)
-
-
- if __name__ == "__main__":
- main()
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- /workspace/python/tvm/relay/build_module.py:345: DeprecationWarning: Please use input parameter mod (tvm.IRModule) instead of deprecated parameter mod (tvm.relay.function.Function)
- warnings.warn(
-
-
-
-
-
-.. rst-class:: sphx-glr-timing
-
- **Total running time of the script:** ( 2 minutes 7.550 seconds)
-
-
-.. _sphx_glr_download_how_to_deploy_models_deploy_quantized.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: deploy_quantized.py <deploy_quantized.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: deploy_quantized.ipynb <deploy_quantized.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/how_to/deploy_models/index.rst.txt b/docs/_sources/how_to/deploy_models/index.rst.txt
index f43b3b3bd5..c668da15ab 100644
--- a/docs/_sources/how_to/deploy_models/index.rst.txt
+++ b/docs/_sources/how_to/deploy_models/index.rst.txt
@@ -150,23 +150,6 @@ backends.
</div>
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This article is an introductory tutorial of automatic quantization with TVM. Automatic quantiza...">
-
-.. only:: html
-
- .. image:: /how_to/deploy_models/images/thumb/sphx_glr_deploy_quantized_thumb.png
- :alt: Deploy a Quantized Model on Cuda
-
- :ref:`sphx_glr_how_to_deploy_models_deploy_quantized.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Deploy a Quantized Model on Cuda</div>
- </div>
-
-
.. raw:: html
<div class="sphx-glr-thumbcontainer" tooltip="This tutorial demonstrates how to take any pruned model, in this case `PruneBert from Hugging F...">
@@ -200,7 +183,6 @@ backends.
/how_to/deploy_models/deploy_object_detection_pytorch
/how_to/deploy_models/deploy_prequantized
/how_to/deploy_models/deploy_prequantized_tflite
- /how_to/deploy_models/deploy_quantized
/how_to/deploy_models/deploy_sparse
diff --git a/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt b/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
index 7ee098ea92..3e09255b71 100644
--- a/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
@@ -5,26 +5,24 @@
Computation times
=================
-**12:08.972** total execution time for **how_to_deploy_models** files:
+**09:26.288** total execution time for **how_to_deploy_models** files:
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_object_detection_pytorch.py` (``deploy_object_detection_pytorch.py``) | 03:38.224 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_object_detection_pytorch.py` (``deploy_object_detection_pytorch.py``) | 03:22.657 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_quantized.py` (``deploy_quantized.py``) | 02:07.550 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized.py` (``deploy_prequantized.py``) | 01:24.914 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized.py` (``deploy_prequantized.py``) | 01:30.277 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno.py` (``deploy_model_on_adreno.py``) | 01:16.927 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno.py` (``deploy_model_on_adreno.py``) | 01:20.092 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized_tflite.py` (``deploy_prequantized_tflite.py``) | 00:50.121 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized_tflite.py` (``deploy_prequantized_tflite.py``) | 00:52.393 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_android.py` (``deploy_model_on_android.py``) | 00:48.699 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_android.py` (``deploy_model_on_android.py``) | 00:51.936 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno_tvmc.py` (``deploy_model_on_adreno_tvmc.py``) | 00:45.841 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno_tvmc.py` (``deploy_model_on_adreno_tvmc.py``) | 00:46.861 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_rasp.py` (``deploy_model_on_rasp.py``) | 00:28.569 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_nano.py` (``deploy_model_on_nano.py``) | 00:30.974 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_nano.py` (``deploy_model_on_nano.py``) | 00:28.551 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_rasp.py` (``deploy_model_on_rasp.py``) | 00:30.660 | 0.0 MB |
-+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_sparse.py` (``deploy_sparse.py``) | 00:00.007 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_sparse.py` (``deploy_sparse.py``) | 00:00.008 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/extend_tvm/bring_your_own_datatypes.rst.txt b/docs/_sources/how_to/extend_tvm/bring_your_own_datatypes.rst.txt
deleted file mode 100644
index f86f24204b..0000000000
--- a/docs/_sources/how_to/extend_tvm/bring_your_own_datatypes.rst.txt
+++ /dev/null
@@ -1,735 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "how_to/extend_tvm/bring_your_own_datatypes.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_how_to_extend_tvm_bring_your_own_datatypes.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/b11795df0596a55e4982bf895d0c8c38/bring_your_own_datatypes.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_how_to_extend_tvm_bring_your_own_datatypes.py:
-
-
-Bring Your Own Datatypes to TVM
-===============================
-**Authors**: `Gus Smith <https://github.com/gussmith23>`_, `Andrew Liu <https://github.com/hypercubestart>`_
-
-In this tutorial, we will show you how to utilize the Bring Your Own Datatypes framework to use your own custom datatypes in TVM.
-Note that the Bring Your Own Datatypes framework currently only handles **software emulated versions of datatypes**.
-The framework does not support compiling for custom accelerator datatypes out-of-the-box.
-
-Datatype Libraries
-------------------
-
-The Bring Your Own Datatypes allows users to register their own datatype implementations alongside TVM's native datatypes (such as ``float``).
-In the wild, these datatype implementations often appear as libraries.
-For example:
-
-- `libposit <https://github.com/cjdelisle/libposit>`_, a posit library
-- `Stillwater Universal <https://github.com/stillwater-sc/universal>`_, a library with posits, fixed-point numbers, and other types
-- `SoftFloat <https://github.com/ucb-bar/berkeley-softfloat-3>`_, Berkeley's software implementation of IEEE 754 floating-point
-
-The Bring Your Own Datatypes enables users to plug these datatype implementations into TVM!
-
-In this section, we will use an example library we have already implemented, located at ``3rdparty/byodt/myfloat.cc``.
-This datatype, which we dubbed "myfloat", is really just a IEE-754 float under-the-hood, but it serves a useful example
-to show that any datatype can be used in the BYODT framework.
-
-Setup
------
-
-Since we do not use any 3rdparty library, there is no setup needed.
-
-If you would like to try this with your own datatype library, first bring the library's functions into the process space with ``CDLL``:
-
-.. code-block:: python
-
- ctypes.CDLL('my-datatype-lib.so', ctypes.RTLD_GLOBAL)
-
-.. GENERATED FROM PYTHON SOURCE LINES 57-61
-
-A Simple TVM Program
---------------------
-
-We'll begin by writing a simple program in TVM; afterwards, we will re-write it to use custom datatypes.
-
-.. GENERATED FROM PYTHON SOURCE LINES 61-73
-
-.. code-block:: default
-
- import sys
-
- import tvm
- from tvm import relay
-
- # Our basic program: Z = X + Y
- x = relay.var("x", shape=(3,), dtype="float32")
- y = relay.var("y", shape=(3,), dtype="float32")
- z = x + y
- program = relay.Function([x, y], z)
- module = tvm.IRModule.from_expr(program)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 74-75
-
-Now, we create random inputs to feed into this program using numpy:
-
-.. GENERATED FROM PYTHON SOURCE LINES 75-85
-
-.. code-block:: default
-
-
- import numpy as np
-
- np.random.seed(23) # for reproducibility
-
- x_input = np.random.rand(3).astype("float32")
- y_input = np.random.rand(3).astype("float32")
- print("x: {}".format(x_input))
- print("y: {}".format(y_input))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- x: [0.51729786 0.9469626 0.7654598 ]
- y: [0.28239584 0.22104536 0.6862221 ]
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 86-87
-
-Finally, we're ready to run the program:
-
-.. GENERATED FROM PYTHON SOURCE LINES 87-91
-
-.. code-block:: default
-
-
- z_output = relay.create_executor(mod=module).evaluate()(x_input, y_input)
- print("z: {}".format(z_output))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- z: [0.7996937 1.168008 1.4516819]
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 92-100
-
-Adding Custom Datatypes
------------------------
-Now, we will do the same, but we will use a custom datatype for our intermediate computation.
-
-We use the same input variables ``x`` and ``y`` as above, but before adding ``x + y``, we first cast both ``x`` and ``y`` to a custom datatype via the ``relay.cast(...)`` call.
-
-Note how we specify the custom datatype: we indicate it using the special ``custom[...]`` syntax.
-Additionally, note the "32" after the datatype: this is the bitwidth of the custom datatype. This tells TVM that each instance of ``myfloat`` is 32 bits wide.
-
-.. GENERATED FROM PYTHON SOURCE LINES 100-111
-
-.. code-block:: default
-
-
- try:
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- x_myfloat = relay.cast(x, dtype="custom[myfloat]32")
- y_myfloat = relay.cast(y, dtype="custom[myfloat]32")
- z_myfloat = x_myfloat + y_myfloat
- z = relay.cast(z_myfloat, dtype="float32")
- except tvm.TVMError as e:
- # Print last line of error
- print(str(e).split("\n")[-1])
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 112-115
-
-Trying to generate this program throws an error from TVM.
-TVM does not know how to handle any custom datatype out of the box!
-We first have to register the custom type with TVM, giving it a name and a type code:
-
-.. GENERATED FROM PYTHON SOURCE LINES 115-118
-
-.. code-block:: default
-
-
- tvm.target.datatype.register("myfloat", 150)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 119-122
-
-Note that the type code, 150, is currently chosen manually by the user.
-See ``TVMTypeCode::kCustomBegin`` in `include/tvm/runtime/c_runtime_api.h <https://github.com/apache/tvm/blob/main/include/tvm/runtime/data_type.h>`_.
-Now we can generate our program again:
-
-.. GENERATED FROM PYTHON SOURCE LINES 122-131
-
-.. code-block:: default
-
-
- x_myfloat = relay.cast(x, dtype="custom[myfloat]32")
- y_myfloat = relay.cast(y, dtype="custom[myfloat]32")
- z_myfloat = x_myfloat + y_myfloat
- z = relay.cast(z_myfloat, dtype="float32")
- program = relay.Function([x, y], z)
- module = tvm.IRModule.from_expr(program)
- module = relay.transform.InferType()(module)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 132-133
-
-Now we have a Relay program that uses myfloat!
-
-.. GENERATED FROM PYTHON SOURCE LINES 133-135
-
-.. code-block:: default
-
- print(program)
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- fn (%x: Tensor[(3), float32], %y: Tensor[(3), float32]) {
- %0 = cast(%x, dtype="custom[myfloat]32");
- %1 = cast(%y, dtype="custom[myfloat]32");
- %2 = add(%0, %1);
- cast(%2, dtype="float32")
- }
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 136-137
-
-Now that we can express our program without errors, let's try running it!
-
-.. GENERATED FROM PYTHON SOURCE LINES 137-145
-
-.. code-block:: default
-
- try:
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- z_output_myfloat = relay.create_executor("graph", mod=module).evaluate()(x_input, y_input)
- print("z: {}".format(y_myfloat))
- except tvm.TVMError as e:
- # Print last line of error
- print(str(e).split("\n")[-1])
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- InternalError: Check failed: (lower) is false: Cast lowering function for target llvm destination type 150 source type 2 not found
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 146-155
-
-Now, trying to compile this program throws an error.
-Let's dissect this error.
-
-The error is occurring during the process of lowering the custom datatype code to code that TVM can compile and run.
-TVM is telling us that it cannot find a *lowering function* for the ``Cast`` operation, when casting from source type 2 (``float``, in TVM), to destination type 150 (our custom datatype).
-When lowering custom datatypes, if TVM encounters an operation over a custom datatype, it looks for a user-registered *lowering function*, which tells it how to lower the operation to an operation over datatypes it understands.
-We have not told TVM how to lower ``Cast`` operations for our custom datatypes; thus, the source of this error.
-
-To fix this error, we simply need to specify a lowering function:
-
-.. GENERATED FROM PYTHON SOURCE LINES 155-168
-
-.. code-block:: default
-
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func(
- {
- (32, 32): "FloatToCustom32", # cast from float32 to myfloat32
- }
- ),
- "Cast",
- "llvm",
- "float",
- "myfloat",
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 169-180
-
-The ``register_op(...)`` call takes a lowering function, and a number of parameters which specify exactly the operation which should be lowered with the provided lowering function.
-In this case, the arguments we pass specify that this lowering function is for lowering a ``Cast`` from ``float`` to ``myfloat`` for target ``"llvm"``.
-
-The lowering function passed into this call is very general: it should take an operation of the specified type (in this case, `Cast`) and return another operation which only uses datatypes which TVM understands.
-
-In the general case, we expect users to implement operations over their custom datatypes using calls to an external library.
-In our example, our ``myfloat`` library implements a ``Cast`` from ``float`` to 32-bit ``myfloat`` in the function ``FloatToCustom32``.
-To provide for the general case, we have made a helper function, ``create_lower_func(...)``,
-which does just this: given a dictionary, it replaces the given operation with a ``Call`` to the appropriate function name provided based on the op and the bit widths.
-It additionally removes usages of the custom datatype by storing the custom datatype in an opaque ``uint`` of the appropriate width; in our case, a ``uint32_t``.
-For more information, see `the source code <https://github.com/apache/tvm/blob/main/python/tvm/target/datatype.py>`_.
-
-.. GENERATED FROM PYTHON SOURCE LINES 180-190
-
-.. code-block:: default
-
-
- # We can now re-try running the program:
- try:
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- z_output_myfloat = relay.create_executor("graph", mod=module).evaluate()(x_input, y_input)
- print("z: {}".format(z_output_myfloat))
- except tvm.TVMError as e:
- # Print last line of error
- print(str(e).split("\n")[-1])
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- InternalError: Check failed: (lower) is false: Add lowering function for target llvm type 150 not found
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 191-197
-
-This new error tells us that the ``Add`` lowering function is not found, which is good news, as it's no longer complaining about the ``Cast``!
-We know what to do from here: we just need to register the lowering functions for the other operations in our program.
-
-Note that for ``Add``, ``create_lower_func`` takes in a dict where the key is an integer.
-For ``Cast`` operations, we require a 2-tuple to specify the ``src_bit_length`` and the ``dest_bit_length``,
-while for all other operations, the bit length is the same between the operands so we only require one integer to specify ``bit_length``.
-
-.. GENERATED FROM PYTHON SOURCE LINES 197-223
-
-.. code-block:: default
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Add"}),
- "Add",
- "llvm",
- "myfloat",
- )
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({(32, 32): "Custom32ToFloat"}),
- "Cast",
- "llvm",
- "myfloat",
- "float",
- )
-
- # Now, we can run our program without errors.
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- z_output_myfloat = relay.create_executor(mod=module).evaluate()(x_input, y_input)
- print("z: {}".format(z_output_myfloat))
-
- print("x:\t\t{}".format(x_input))
- print("y:\t\t{}".format(y_input))
- print("z (float32):\t{}".format(z_output))
- print("z (myfloat32):\t{}".format(z_output_myfloat))
-
- # Perhaps as expected, the ``myfloat32`` results and ``float32`` are exactly the same!
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- z: [0.7996937 1.168008 1.4516819]
- x: [0.51729786 0.9469626 0.7654598 ]
- y: [0.28239584 0.22104536 0.6862221 ]
- z (float32): [0.7996937 1.168008 1.4516819]
- z (myfloat32): [0.7996937 1.168008 1.4516819]
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 224-233
-
-Running Models With Custom Datatypes
-------------------------------------
-
-We will first choose the model which we would like to run with myfloat.
-In this case we use `Mobilenet <https://arxiv.org/abs/1704.04861>`_.
-We choose Mobilenet due to its small size.
-In this alpha state of the Bring Your Own Datatypes framework, we have not implemented any software optimizations for running software emulations of custom datatypes; the result is poor performance due to many calls into our datatype emulation library.
-
-First let us define two helper functions to get the mobilenet model and a cat image.
-
-.. GENERATED FROM PYTHON SOURCE LINES 233-264
-
-.. code-block:: default
-
-
-
- def get_mobilenet():
- dshape = (1, 3, 224, 224)
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("mobilenet0.25", pretrained=True)
- shape_dict = {"data": dshape}
- return relay.frontend.from_mxnet(block, shape_dict)
-
-
- def get_cat_image():
- from tvm.contrib.download import download_testdata
- from PIL import Image
-
- url = "https://gist.githubusercontent.com/zhreshold/bcda4716699ac97ea44f791c24310193/raw/fa7ef0e9c9a5daea686d6473a62aacd1a5885849/cat.png"
- dst = "cat.png"
- real_dst = download_testdata(url, dst, module="data")
- img = Image.open(real_dst).resize((224, 224))
- # CoreML's standard model image format is BGR
- img_bgr = np.array(img)[:, :, ::-1]
- img = np.transpose(img_bgr, (2, 0, 1))[np.newaxis, :]
- return np.asarray(img, dtype="float32")
-
-
- try:
- module, params = get_mobilenet()
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Downloading /workspace/.mxnet/models/mobilenet0.25-9f83e440.zip29c92174-b498-41f0-b465-1c5f6c91f4c2 from https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/models/mobilenet0.25-9f83e440.zip...
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 265-266
-
-It's easy to execute MobileNet with native TVM:
-
-.. GENERATED FROM PYTHON SOURCE LINES 266-273
-
-.. code-block:: default
-
-
- ex = tvm.relay.create_executor("graph", mod=module, params=params)
- input = get_cat_image()
- result = ex.evaluate()(input).numpy()
- # print first 10 elements
- print(result.flatten()[:10])
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- [ -7.5350165 2.0368009 -12.706646 -5.63786 -12.684058 4.0723605
- 2.618876 3.4049501 -9.867913 -24.53311 ]
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 274-275
-
-Now, we would like to change the model to use myfloat internally. To do so, we need to convert the network. To do this, we first define a function which will help us convert tensors:
-
-.. GENERATED FROM PYTHON SOURCE LINES 275-285
-
-.. code-block:: default
-
-
-
- def convert_ndarray(dst_dtype, array):
- """Converts an NDArray into the specified datatype"""
- x = relay.var("x", shape=array.shape, dtype=str(array.dtype))
- cast = relay.Function([x], x.astype(dst_dtype))
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- return relay.create_executor("graph").evaluate(cast)(array)
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 286-287
-
-Now, to actually convert the entire network, we have written `a pass in Relay <https://github.com/gussmith23/tvm/blob/ea174c01c54a2529e19ca71e125f5884e728da6e/python/tvm/relay/frontend/change_datatype.py#L21>`_ which simply converts all nodes within the model to use the new datatype.
-
-.. GENERATED FROM PYTHON SOURCE LINES 287-322
-
-.. code-block:: default
-
-
- from tvm.relay.frontend.change_datatype import ChangeDatatype
-
- src_dtype = "float32"
- dst_dtype = "custom[myfloat]32"
-
- module = relay.transform.InferType()(module)
-
- # Currently, custom datatypes only work if you run simplify_inference beforehand
- module = tvm.relay.transform.SimplifyInference()(module)
-
- # Run type inference before changing datatype
- module = tvm.relay.transform.InferType()(module)
-
- # Change datatype from float to myfloat and re-infer types
- cdtype = ChangeDatatype(src_dtype, dst_dtype)
- expr = cdtype.visit(module["main"])
- module = tvm.relay.transform.InferType()(module)
-
- # We also convert the parameters:
- params = {k: convert_ndarray(dst_dtype, v) for k, v in params.items()}
-
- # We also need to convert our input:
- input = convert_ndarray(dst_dtype, input)
-
- # Finally, we can try to run the converted model:
- try:
- # Vectorization is not implemented with custom datatypes.
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- result_myfloat = tvm.relay.create_executor("graph", mod=module).evaluate(expr)(
- input, **params
- )
- except tvm.TVMError as e:
- print(str(e).split("\n")[-1])
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- InternalError: Check failed: (lower) is false: FloatImm lowering function for target llvm type 150 not found
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 323-327
-
-When we attempt to run the model, we get a familiar error telling us that more functions need to be registered for myfloat.
-
-Because this is a neural network, many more operations are required.
-Here, we register all the needed functions:
-
-.. GENERATED FROM PYTHON SOURCE LINES 327-395
-
-.. code-block:: default
-
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "FloatToCustom32"}),
- "FloatImm",
- "llvm",
- "myfloat",
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.lower_ite, "Call", "llvm", "myfloat", intrinsic_name="tir.if_then_else"
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.lower_call_pure_extern,
- "Call",
- "llvm",
- "myfloat",
- intrinsic_name="tir.call_pure_extern",
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Mul"}),
- "Mul",
- "llvm",
- "myfloat",
- )
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Div"}),
- "Div",
- "llvm",
- "myfloat",
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Sqrt"}),
- "Call",
- "llvm",
- "myfloat",
- intrinsic_name="tir.sqrt",
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Sub"}),
- "Sub",
- "llvm",
- "myfloat",
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Exp"}),
- "Call",
- "llvm",
- "myfloat",
- intrinsic_name="tir.exp",
- )
-
- tvm.target.datatype.register_op(
- tvm.target.datatype.create_lower_func({32: "Custom32Max"}),
- "Max",
- "llvm",
- "myfloat",
- )
-
- tvm.target.datatype.register_min_func(
- tvm.target.datatype.create_min_lower_func({32: "MinCustom32"}, "myfloat"),
- "myfloat",
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 396-405
-
-Note we are making use of two new functions: ``register_min_func`` and ``create_min_lower_func``.
-
-``register_min_func`` takes in an integer ``num_bits`` for the bit length, and should return an operation
-representing the minimum finite representable value for the custom data type with the specified bit length.
-
-Similar to ``register_op`` and ``create_lower_func``, the ``create_min_lower_func`` handles the general case
-where the minimum representable custom datatype value is implemented using calls to an external library.
-
-Now we can finally run the model:
-
-.. GENERATED FROM PYTHON SOURCE LINES 405-416
-
-.. code-block:: default
-
-
- # Vectorization is not implemented with custom datatypes.
- with tvm.transform.PassContext(config={"tir.disable_vectorize": True}):
- result_myfloat = relay.create_executor(mod=module).evaluate(expr)(input, **params)
- result_myfloat = convert_ndarray(src_dtype, result_myfloat).numpy()
- # print first 10 elements
- print(result_myfloat.flatten()[:10])
-
- # Again, note that the output using 32-bit myfloat exactly the same as 32-bit floats,
- # because myfloat is exactly a float!
- np.testing.assert_array_equal(result, result_myfloat)
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- [ -7.5350165 2.0368009 -12.706646 -5.63786 -12.684058 4.0723605
- 2.618876 3.4049501 -9.867913 -24.53311 ]
-
-
-
-
-
-.. _sphx_glr_download_how_to_extend_tvm_bring_your_own_datatypes.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: bring_your_own_datatypes.py <bring_your_own_datatypes.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: bring_your_own_datatypes.ipynb <bring_your_own_datatypes.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/how_to/extend_tvm/index.rst.txt b/docs/_sources/how_to/extend_tvm/index.rst.txt
index b0e0f70750..2fd8784f6c 100644
--- a/docs/_sources/how_to/extend_tvm/index.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/index.rst.txt
@@ -66,23 +66,6 @@ extended.
</div>
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we will show you how to utilize the Bring Your Own Datatypes framework to use...">
-
-.. only:: html
-
- .. image:: /how_to/extend_tvm/images/thumb/sphx_glr_bring_your_own_datatypes_thumb.png
- :alt: Bring Your Own Datatypes to TVM
-
- :ref:`sphx_glr_how_to_extend_tvm_bring_your_own_datatypes.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Bring Your Own Datatypes to TVM</div>
- </div>
-
-
.. raw:: html
</div>
@@ -94,7 +77,6 @@ extended.
/how_to/extend_tvm/low_level_custom_pass
/how_to/extend_tvm/use_pass_infra
/how_to/extend_tvm/use_pass_instrument
- /how_to/extend_tvm/bring_your_own_datatypes
diff --git a/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt b/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
index feda4166a3..42ec7b7533 100644
--- a/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
@@ -5,14 +5,12 @@
Computation times
=================
-**00:59.090** total execution time for **how_to_extend_tvm** files:
+**00:03.766** total execution time for **how_to_extend_tvm** files:
-+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_bring_your_own_datatypes.py` (``bring_your_own_datatypes.py``) | 00:55.017 | 0.0 MB |
-+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_use_pass_instrument.py` (``use_pass_instrument.py``) | 00:02.852 | 0.0 MB |
-+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_use_pass_infra.py` (``use_pass_infra.py``) | 00:01.214 | 0.0 MB |
-+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_low_level_custom_pass.py` (``low_level_custom_pass.py``) | 00:00.007 | 0.0 MB |
-+-------------------------------------------------------------------------------------------------+-----------+--------+
++-------------------------------------------------------------------------------------------+-----------+--------+
+| :ref:`sphx_glr_how_to_extend_tvm_use_pass_instrument.py` (``use_pass_instrument.py``) | 00:02.637 | 0.0 MB |
++-------------------------------------------------------------------------------------------+-----------+--------+
+| :ref:`sphx_glr_how_to_extend_tvm_use_pass_infra.py` (``use_pass_infra.py``) | 00:01.122 | 0.0 MB |
++-------------------------------------------------------------------------------------------+-----------+--------+
+| :ref:`sphx_glr_how_to_extend_tvm_low_level_custom_pass.py` (``low_level_custom_pass.py``) | 00:00.007 | 0.0 MB |
++-------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt b/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
index bcdada63f8..fda3ec58eb 100644
--- a/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
@@ -220,10 +220,10 @@ profile the execution time of each passes.
.. code-block:: none
Printing results of timing profile...
- InferType: 25554us [25554us] (48.24%; 48.24%)
- FoldScaleAxis: 27420us [9us] (51.76%; 51.76%)
- FoldConstant: 27410us [1862us] (51.74%; 99.97%)
- InferType: 25549us [25549us] (48.23%; 93.21%)
+ InferType: 24356us [24356us] (48.60%; 48.60%)
+ FoldScaleAxis: 25763us [7us] (51.40%; 51.40%)
+ FoldConstant: 25756us [1806us] (51.39%; 99.97%)
+ InferType: 23950us [23950us] (47.79%; 92.99%)
@@ -262,10 +262,10 @@ Refer to following sections and :py:func:`tvm.instrument.pass_instrument` for th
.. code-block:: none
Printing results of timing profile...
- InferType: 25525us [25525us] (48.48%; 48.48%)
- FoldScaleAxis: 27120us [8us] (51.52%; 51.52%)
- FoldConstant: 27112us [1892us] (51.50%; 99.97%)
- InferType: 25220us [25220us] (47.91%; 93.02%)
+ InferType: 23521us [23521us] (48.60%; 48.60%)
+ FoldScaleAxis: 24873us [5us] (51.40%; 51.40%)
+ FoldConstant: 24868us [1664us] (51.39%; 99.98%)
+ InferType: 23204us [23204us] (47.95%; 93.31%)
diff --git a/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt b/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
index 3e33334338..a74f05d061 100644
--- a/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
@@ -331,7 +331,7 @@ latency of convolution.
.. code-block:: none
- Convolution: 39.436862 ms
+ Convolution: 53.497695 ms
diff --git a/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt b/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
index cc2446be41..83afce3a75 100644
--- a/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
@@ -598,7 +598,7 @@ be able to run on our build server
.. code-block:: none
- conv2d with tensor core: 12.265677 ms
+ conv2d with tensor core: 12.266832 ms
diff --git a/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt b/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
index a2988a6122..5838882089 100644
--- a/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
@@ -134,8 +134,8 @@ Then we write a baseline implementation, the simplest way to write a matrix mult
.. code-block:: none
- Numpy running time: 0.018760
- Baseline: 3.282078
+ Numpy running time: 0.013818
+ Baseline: 3.313510
@@ -227,7 +227,7 @@ fill 32 * 32 * sizeof(float) which is 4KB in the cache whose total size is 32KB
.. code-block:: none
- Opt1: 0.316699
+ Opt1: 0.289857
@@ -318,7 +318,7 @@ In this tutorial, we chose to vectorize the inner loop row data since it is cach
.. code-block:: none
- Opt2: 0.295198
+ Opt2: 0.258958
@@ -406,7 +406,7 @@ the access pattern for A matrix is more cache friendly.
.. code-block:: none
- Opt3: 0.115228
+ Opt3: 0.110938
@@ -523,7 +523,7 @@ flattening.
.. code-block:: none
- Opt4: 0.106512
+ Opt4: 0.103325
@@ -635,7 +635,7 @@ write to C when all the block results are ready.
.. code-block:: none
- Opt5: 0.112095
+ Opt5: 0.095177
@@ -748,7 +748,7 @@ Furthermore, we can also utilize multi-core processors to do the thread-level pa
.. code-block:: none
- Opt6: 0.133301
+ Opt6: 0.112953
diff --git a/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt b/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
index babfaeba31..ae3b10cdaf 100644
--- a/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:34.044** total execution time for **how_to_optimize_operators** files:
+**00:31.515** total execution time for **how_to_optimize_operators** files:
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_gemm.py` (``opt_gemm.py``) | 00:30.957 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_gemm.py` (``opt_gemm.py``) | 00:28.423 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_tensorcore.py` (``opt_conv_tensorcore.py``) | 00:01.904 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_tensorcore.py` (``opt_conv_tensorcore.py``) | 00:01.871 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_cuda.py` (``opt_conv_cuda.py``) | 00:01.183 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_cuda.py` (``opt_conv_cuda.py``) | 00:01.220 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
index f6fabf665f..0a844ef088 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
@@ -5,18 +5,18 @@
Computation times
=================
-**03:40.495** total execution time for **how_to_tune_with_autoscheduler** files:
+**03:27.124** total execution time for **how_to_tune_with_autoscheduler** files:
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_x86.py` (``tune_network_x86.py``) | 01:35.218 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_x86.py` (``tune_network_x86.py``) | 01:29.493 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_cuda.py` (``tune_network_cuda.py``) | 01:13.548 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_cuda.py` (``tune_network_cuda.py``) | 01:09.698 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_arm.py` (``tune_network_arm.py``) | 00:18.191 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_arm.py` (``tune_network_arm.py``) | 00:17.129 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_conv2d_layer_cuda.py` (``tune_conv2d_layer_cuda.py``) | 00:16.770 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_mali.py` (``tune_network_mali.py``) | 00:15.729 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_mali.py` (``tune_network_mali.py``) | 00:16.666 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_conv2d_layer_cuda.py` (``tune_conv2d_layer_cuda.py``) | 00:14.978 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_sparse_x86.py` (``tune_sparse_x86.py``) | 00:00.101 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_sparse_x86.py` (``tune_sparse_x86.py``) | 00:00.097 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_conv2d_layer_cuda.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_conv2d_layer_cuda.rst.txt
index a77741320d..d85b3f249a 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_conv2d_layer_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_conv2d_layer_cuda.rst.txt
@@ -766,7 +766,7 @@ We build the binary and check its correctness and performance.
.. code-block:: none
- Execution time of this operator: 0.357 ms
+ Execution time of this operator: 0.334 ms
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt
index bb1e3f7c8f..e12ab57ae9 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_arm.rst.txt
@@ -89,7 +89,7 @@ We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 74-151
+.. GENERATED FROM PYTHON SOURCE LINES 74-138
.. code-block:: default
@@ -142,19 +142,6 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet50_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
elif name == "mlp":
mod, params = relay.testing.mlp.get_workload(
batch_size=batch_size, dtype=dtype, image_shape=image_shape, num_classes=1000
@@ -177,7 +164,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 152-176
+.. GENERATED FROM PYTHON SOURCE LINES 139-163
Start RPC Tracker
-----------------
@@ -204,7 +191,7 @@ The expected output is
INFO:RPCTracker:bind to 0.0.0.0:9190
-.. GENERATED FROM PYTHON SOURCE LINES 178-220
+.. GENERATED FROM PYTHON SOURCE LINES 165-207
Register Devices to RPC Tracker
-----------------------------------
@@ -249,7 +236,7 @@ the output can be
You can register multiple devices to the tracker to accelerate the measurement in tuning.
-.. GENERATED FROM PYTHON SOURCE LINES 222-228
+.. GENERATED FROM PYTHON SOURCE LINES 209-215
Set Tuning Options
------------------
@@ -258,7 +245,7 @@ as example with a 64bit OS (Ubuntu 20.04). In your setting, you should modify th
and device_key accordingly.
set :code:`use_ndk` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 228-255
+.. GENERATED FROM PYTHON SOURCE LINES 215-242
.. code-block:: default
@@ -296,7 +283,7 @@ set :code:`use_ndk` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 256-265
+.. GENERATED FROM PYTHON SOURCE LINES 243-252
Extract Search Tasks
--------------------
@@ -308,20 +295,16 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 265-283
+.. GENERATED FROM PYTHON SOURCE LINES 252-266
.. code-block:: default
# Extract tasks from the network
print("Get model...")
- try:
- mod, params, input_shape, output_shape = get_network(
- network, batch_size, layout, dtype=dtype, use_sparse=use_sparse
- )
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+ mod, params, input_shape, output_shape = get_network(
+ network, batch_size, layout, dtype=dtype, use_sparse=use_sparse
+ )
print("Extract tasks...")
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
@@ -550,7 +533,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 284-303
+.. GENERATED FROM PYTHON SOURCE LINES 267-286
Tuning and Evaluation
---------------------
@@ -572,7 +555,7 @@ After auto-tuning, we can compile the network with the best schedules we found.
All measurement records are dumped into the log file during auto-tuning,
so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 303-368
+.. GENERATED FROM PYTHON SOURCE LINES 286-351
.. code-block:: default
@@ -648,7 +631,7 @@ so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 369-420
+.. GENERATED FROM PYTHON SOURCE LINES 352-403
.. note:: Explaining the printed information during tuning
@@ -702,7 +685,7 @@ so we can read the log file and load the best schedules.
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 422-428
+.. GENERATED FROM PYTHON SOURCE LINES 405-411
.. note:: Terminate the tuning earlier
@@ -711,7 +694,7 @@ so we can read the log file and load the best schedules.
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 430-446
+.. GENERATED FROM PYTHON SOURCE LINES 413-429
Other Tips
----------
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
index 4987e515bc..9c156d9b43 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
@@ -85,7 +85,7 @@ We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 68-142
+.. GENERATED FROM PYTHON SOURCE LINES 68-128
.. code-block:: default
@@ -138,20 +138,6 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
-
return mod, params, input_shape, output_shape
@@ -170,7 +156,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 143-152
+.. GENERATED FROM PYTHON SOURCE LINES 129-138
Extract Search Tasks
--------------------
@@ -182,18 +168,14 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 152-166
+.. GENERATED FROM PYTHON SOURCE LINES 138-148
.. code-block:: default
# Extract tasks from the network
print("Extract tasks...")
- try:
- mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+ mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
for idx, task in enumerate(tasks):
@@ -486,7 +468,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 167-189
+.. GENERATED FROM PYTHON SOURCE LINES 149-171
Begin Tuning
------------
@@ -511,7 +493,7 @@ Now, we set some options for tuning and launch the search tasks
:any:`auto_scheduler.LocalRPCMeasureContext` for more parameters.
-.. GENERATED FROM PYTHON SOURCE LINES 189-211
+.. GENERATED FROM PYTHON SOURCE LINES 171-193
.. code-block:: default
@@ -544,7 +526,7 @@ Now, we set some options for tuning and launch the search tasks
-.. GENERATED FROM PYTHON SOURCE LINES 212-265
+.. GENERATED FROM PYTHON SOURCE LINES 194-247
.. note:: Explain the printed information during tuning
@@ -600,7 +582,7 @@ Now, we set some options for tuning and launch the search tasks
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 267-273
+.. GENERATED FROM PYTHON SOURCE LINES 249-255
.. note:: Terminate the tuning earlier
@@ -609,7 +591,7 @@ Now, we set some options for tuning and launch the search tasks
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 276-281
+.. GENERATED FROM PYTHON SOURCE LINES 258-263
Compile and Evaluate
--------------------
@@ -617,7 +599,7 @@ After auto-tuning, we can compile the network with the best schedules we found.
All measurement records are dumped into the log file during auto-tuning,
so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 281-299
+.. GENERATED FROM PYTHON SOURCE LINES 263-281
.. code-block:: default
@@ -651,12 +633,12 @@ so we can read the log file and load the best schedules.
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 3.2944 3.2928 3.2999 3.2906 0.0039
+ 3.1960 3.1960 3.1964 3.1956 0.0003
-.. GENERATED FROM PYTHON SOURCE LINES 300-316
+.. GENERATED FROM PYTHON SOURCE LINES 282-298
Other Tips
----------
@@ -678,7 +660,7 @@ Other Tips
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 13.548 seconds)
+ **Total running time of the script:** ( 1 minutes 9.698 seconds)
.. _sphx_glr_download_how_to_tune_with_autoscheduler_tune_network_cuda.py:
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt
index 8d20b2c7fa..225dabb3fb 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_mali.rst.txt
@@ -88,7 +88,7 @@ We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 71-150
+.. GENERATED FROM PYTHON SOURCE LINES 71-137
.. code-block:: default
@@ -141,19 +141,6 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet50_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
return mod, params, input_shape, output_shape
@@ -178,7 +165,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 151-156
+.. GENERATED FROM PYTHON SOURCE LINES 138-143
Start an RPC Tracker and Register Devices to the Tracker
--------------------------------------------------------
@@ -186,7 +173,7 @@ Please refer to the "Start RPC Tracker" and "Register Devices to RPC Tracker" se
in this :ref:`tutorial <tutorials-autotvm-start-rpc-tracker>` to start an RPC tracker
and register devices to the tracker.
-.. GENERATED FROM PYTHON SOURCE LINES 156-161
+.. GENERATED FROM PYTHON SOURCE LINES 143-148
.. code-block:: default
@@ -202,7 +189,7 @@ and register devices to the tracker.
-.. GENERATED FROM PYTHON SOURCE LINES 162-171
+.. GENERATED FROM PYTHON SOURCE LINES 149-158
Extract Search Tasks
--------------------
@@ -214,18 +201,14 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 171-184
+.. GENERATED FROM PYTHON SOURCE LINES 158-167
.. code-block:: default
# Extract tasks from the network
print("Extract tasks...")
- try:
- mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+ mod, params, input_shape, output_shape = get_network(network, batch_size, layout, dtype=dtype)
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
for idx, task in enumerate(tasks):
@@ -436,7 +419,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 185-211
+.. GENERATED FROM PYTHON SOURCE LINES 168-194
.. note:: How to get the hardware parameters from remote device
@@ -465,7 +448,7 @@ The task scheduler will just optimize this objective.
)
-.. GENERATED FROM PYTHON SOURCE LINES 213-229
+.. GENERATED FROM PYTHON SOURCE LINES 196-212
Tuning and Evaluate
-------------------
@@ -484,7 +467,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
:any:`auto_scheduler.LocalRunner` for more parameters.
-.. GENERATED FROM PYTHON SOURCE LINES 229-282
+.. GENERATED FROM PYTHON SOURCE LINES 212-265
.. code-block:: default
@@ -548,7 +531,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
-.. GENERATED FROM PYTHON SOURCE LINES 283-341
+.. GENERATED FROM PYTHON SOURCE LINES 266-324
.. note:: Explain the printed information during tuning
@@ -609,7 +592,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 343-349
+.. GENERATED FROM PYTHON SOURCE LINES 326-332
.. note:: Terminate the tuning earlier
@@ -618,7 +601,7 @@ Now, we set some options for tuning, launch the search tasks and evaluate the en
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 351-367
+.. GENERATED FROM PYTHON SOURCE LINES 334-350
Other Tips
----------
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
index 9c97d9e472..aba948fd77 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
@@ -87,7 +87,7 @@ We also implemented more optimizations for NHWC layout with the auto-scheduler.
So it is recommended to convert your models to NHWC layout to use the auto-scheduler.
You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout conversion in TVM.
-.. GENERATED FROM PYTHON SOURCE LINES 71-159
+.. GENERATED FROM PYTHON SOURCE LINES 71-146
.. code-block:: default
@@ -140,19 +140,6 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299) if layout == "NCHW" else (batch_size, 299, 299, 3)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- assert layout == "NCHW"
-
- block = get_model("resnet50_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
elif name == "mlp":
mod, params = relay.testing.mlp.get_workload(
batch_size=batch_size, dtype=dtype, image_shape=image_shape, num_classes=1000
@@ -186,7 +173,7 @@ You can use :ref:`ConvertLayout <convert-layout-usage>` pass to do the layout co
-.. GENERATED FROM PYTHON SOURCE LINES 160-169
+.. GENERATED FROM PYTHON SOURCE LINES 147-156
Extract Search Tasks
--------------------
@@ -198,24 +185,20 @@ as :code:`sum(latency[t] * weight[t])`, where :code:`latency[t]` is the
latency of a task and :code:`weight[t]` is the weight of the task.
The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 169-191
+.. GENERATED FROM PYTHON SOURCE LINES 156-174
.. code-block:: default
# Extract tasks from the network
print("Get model...")
- try:
- mod, params, input_shape, output_shape = get_network(
- network,
- batch_size,
- layout,
- dtype=dtype,
- use_sparse=use_sparse,
- )
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
+ mod, params, input_shape, output_shape = get_network(
+ network,
+ batch_size,
+ layout,
+ dtype=dtype,
+ use_sparse=use_sparse,
+ )
print("Extract tasks...")
tasks, task_weights = auto_scheduler.extract_tasks(mod["main"], params, target)
@@ -496,7 +479,7 @@ The task scheduler will just optimize this objective.
-.. GENERATED FROM PYTHON SOURCE LINES 192-208
+.. GENERATED FROM PYTHON SOURCE LINES 175-191
Begin Tuning
------------
@@ -515,7 +498,7 @@ Now, we set some options for tuning and launch the search tasks
:any:`auto_scheduler.LocalRunner` for more parameters.
-.. GENERATED FROM PYTHON SOURCE LINES 208-242
+.. GENERATED FROM PYTHON SOURCE LINES 191-225
.. code-block:: default
@@ -560,7 +543,7 @@ Now, we set some options for tuning and launch the search tasks
-.. GENERATED FROM PYTHON SOURCE LINES 243-301
+.. GENERATED FROM PYTHON SOURCE LINES 226-284
.. note:: Explain the printed information during tuning
@@ -621,7 +604,7 @@ Now, we set some options for tuning and launch the search tasks
errors are isolated from the main process.
-.. GENERATED FROM PYTHON SOURCE LINES 303-309
+.. GENERATED FROM PYTHON SOURCE LINES 286-292
.. note:: Terminate the tuning earlier
@@ -630,7 +613,7 @@ Now, we set some options for tuning and launch the search tasks
you should be able to do the compilation (the secion below).
-.. GENERATED FROM PYTHON SOURCE LINES 312-317
+.. GENERATED FROM PYTHON SOURCE LINES 295-300
Compile and Evaluate
--------------------
@@ -638,7 +621,7 @@ After auto-tuning, we can compile the network with the best schedules we found.
All measurement records are dumped into the log file during auto-tuning,
so we can read the log file and load the best schedules.
-.. GENERATED FROM PYTHON SOURCE LINES 317-335
+.. GENERATED FROM PYTHON SOURCE LINES 300-318
.. code-block:: default
@@ -672,12 +655,12 @@ so we can read the log file and load the best schedules.
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 760.1610 759.7629 761.9910 758.7290 1.3611
+ 701.1497 699.5665 705.8347 698.0478 3.3704
-.. GENERATED FROM PYTHON SOURCE LINES 336-352
+.. GENERATED FROM PYTHON SOURCE LINES 319-335
Other Tips
----------
@@ -699,7 +682,7 @@ Other Tips
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 35.218 seconds)
+ **Total running time of the script:** ( 1 minutes 29.493 seconds)
.. _sphx_glr_download_how_to_tune_with_autoscheduler_tune_network_x86.py:
diff --git a/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt b/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
index df4438ff58..ed6b965a6b 100644
--- a/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
@@ -5,14 +5,14 @@
Computation times
=================
-**00:24.100** total execution time for **how_to_tune_with_autotvm** files:
+**00:22.622** total execution time for **how_to_tune_with_autotvm** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_conv2d_cuda.py` (``tune_conv2d_cuda.py``) | 00:24.061 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_conv2d_cuda.py` (``tune_conv2d_cuda.py``) | 00:22.586 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_x86.py` (``tune_relay_x86.py``) | 00:00.023 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_x86.py` (``tune_relay_x86.py``) | 00:00.021 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_cuda.py` (``tune_relay_cuda.py``) | 00:00.006 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_cuda.py` (``tune_relay_cuda.py``) | 00:00.005 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_mobile_gpu.py` (``tune_relay_mobile_gpu.py``) | 00:00.005 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
index a1896168ab..86fb14a128 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
@@ -326,7 +326,7 @@ and measure running time.
Best config:
,None
- Time cost of this operator: 0.037176
+ Time cost of this operator: 0.037134
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_relay_arm.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_relay_arm.rst.txt
index 89b6cc38b9..31a9045073 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_relay_arm.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_relay_arm.rst.txt
@@ -100,7 +100,7 @@ First we need to define the network in relay frontend API.
We can load some pre-defined network from :code:`relay.testing`.
We can also load models from MXNet, ONNX and TensorFlow.
-.. GENERATED FROM PYTHON SOURCE LINES 82-125
+.. GENERATED FROM PYTHON SOURCE LINES 82-114
.. code-block:: default
@@ -130,17 +130,6 @@ We can also load models from MXNet, ONNX and TensorFlow.
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
@@ -154,7 +143,7 @@ We can also load models from MXNet, ONNX and TensorFlow.
-.. GENERATED FROM PYTHON SOURCE LINES 126-150
+.. GENERATED FROM PYTHON SOURCE LINES 115-139
Start RPC Tracker
-----------------
@@ -181,7 +170,7 @@ The expected output is
INFO:RPCTracker:bind to 0.0.0.0:9190
-.. GENERATED FROM PYTHON SOURCE LINES 152-194
+.. GENERATED FROM PYTHON SOURCE LINES 141-183
Register Devices to RPC Tracker
-----------------------------------
@@ -226,7 +215,7 @@ the output can be
You can register multiple devices to the tracker to accelerate the measurement in tuning.
-.. GENERATED FROM PYTHON SOURCE LINES 196-201
+.. GENERATED FROM PYTHON SOURCE LINES 185-190
Set Tuning Options
------------------
@@ -234,7 +223,7 @@ Before tuning, we should apply some configurations. Here I use an RK3399 board
as example. In your setting, you should modify the target and device_key accordingly.
set :code:`use_android` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 201-236
+.. GENERATED FROM PYTHON SOURCE LINES 190-225
.. code-block:: default
@@ -280,7 +269,7 @@ set :code:`use_android` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 237-249
+.. GENERATED FROM PYTHON SOURCE LINES 226-238
.. note:: How to set tuning options
@@ -295,7 +284,7 @@ set :code:`use_android` to True if you use android phone.
optimization in general. For example, on ARM CPU A53 2.0GHz, we find it could boost 1.6x
performance of depthwise convolution on Mobilenet V1 model.
-.. GENERATED FROM PYTHON SOURCE LINES 252-258
+.. GENERATED FROM PYTHON SOURCE LINES 241-247
Begin Tuning
------------
@@ -304,7 +293,7 @@ Here, we provide a simple utility function to tune a list of tasks.
This function is just an initial implementation which tunes them in sequential order.
We will introduce a more sophisticated tuning scheduler in the future.
-.. GENERATED FROM PYTHON SOURCE LINES 258-332
+.. GENERATED FROM PYTHON SOURCE LINES 247-321
.. code-block:: default
@@ -389,11 +378,11 @@ We will introduce a more sophisticated tuning scheduler in the future.
-.. GENERATED FROM PYTHON SOURCE LINES 333-334
+.. GENERATED FROM PYTHON SOURCE LINES 322-323
Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 334-387
+.. GENERATED FROM PYTHON SOURCE LINES 323-376
.. code-block:: default
@@ -457,7 +446,7 @@ Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 388-415
+.. GENERATED FROM PYTHON SOURCE LINES 377-404
Sample Output
-------------
@@ -487,7 +476,7 @@ It takes about 2 hours on a 32T AMD Ryzen Threadripper.
Evaluate inference time cost...
Mean inference time (std dev): 162.59 ms (0.06 ms)
-.. GENERATED FROM PYTHON SOURCE LINES 417-433
+.. GENERATED FROM PYTHON SOURCE LINES 406-422
.. note:: **Experiencing Difficulties?**
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_relay_cuda.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_relay_cuda.rst.txt
index 9c18c078d3..53a4a134d9 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_relay_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_relay_cuda.rst.txt
@@ -96,7 +96,7 @@ First we need to define the network in relay frontend API.
We can load some pre-defined network from :code:`tvm.relay.testing`.
We can also load models from MXNet, ONNX and TensorFlow.
-.. GENERATED FROM PYTHON SOURCE LINES 81-124
+.. GENERATED FROM PYTHON SOURCE LINES 81-113
.. code-block:: default
@@ -126,17 +126,6 @@ We can also load models from MXNet, ONNX and TensorFlow.
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
@@ -150,13 +139,13 @@ We can also load models from MXNet, ONNX and TensorFlow.
-.. GENERATED FROM PYTHON SOURCE LINES 125-128
+.. GENERATED FROM PYTHON SOURCE LINES 114-117
Set Tuning Options
------------------
Before tuning, we apply some configurations.
-.. GENERATED FROM PYTHON SOURCE LINES 128-148
+.. GENERATED FROM PYTHON SOURCE LINES 117-137
.. code-block:: default
@@ -194,7 +183,7 @@ Before tuning, we apply some configurations.
-.. GENERATED FROM PYTHON SOURCE LINES 149-159
+.. GENERATED FROM PYTHON SOURCE LINES 138-148
.. note:: How to set tuning options
@@ -207,7 +196,7 @@ Before tuning, we apply some configurations.
accelerate the tuning process. (see the 'Scale up measurement` section below).
-.. GENERATED FROM PYTHON SOURCE LINES 162-168
+.. GENERATED FROM PYTHON SOURCE LINES 151-157
Begin Tuning
------------
@@ -216,7 +205,7 @@ Here, we provide a simple utility function to tune a list of tasks.
This function is just an initial implementation which tunes them in sequential order.
We will introduce a more sophisticated tuning scheduler in the future.
-.. GENERATED FROM PYTHON SOURCE LINES 168-242
+.. GENERATED FROM PYTHON SOURCE LINES 157-231
.. code-block:: default
@@ -301,11 +290,11 @@ We will introduce a more sophisticated tuning scheduler in the future.
-.. GENERATED FROM PYTHON SOURCE LINES 243-244
+.. GENERATED FROM PYTHON SOURCE LINES 232-233
Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 244-280
+.. GENERATED FROM PYTHON SOURCE LINES 233-269
.. code-block:: default
@@ -352,7 +341,7 @@ Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 281-314
+.. GENERATED FROM PYTHON SOURCE LINES 270-303
Sample Output
-------------
@@ -388,7 +377,7 @@ The tuning target is NVIDIA 1080 Ti.
As a reference baseline, the time cost of MXNet + TensorRT on resnet-18 is 1.30ms. So we are a little faster.
-.. GENERATED FROM PYTHON SOURCE LINES 316-332
+.. GENERATED FROM PYTHON SOURCE LINES 305-321
.. note:: **Experiencing Difficulties?**
@@ -407,11 +396,11 @@ As a reference baseline, the time cost of MXNet + TensorRT on resnet-18 is 1.30m
Finally, always feel free to ask our community for help on https://discuss.tvm.apache.org
-.. GENERATED FROM PYTHON SOURCE LINES 335-336
+.. GENERATED FROM PYTHON SOURCE LINES 324-325
.. _tutorials-autotvm-scale-up-rpc-tracker:
-.. GENERATED FROM PYTHON SOURCE LINES 338-391
+.. GENERATED FROM PYTHON SOURCE LINES 327-380
Scale up measurement by using multiple devices
----------------------------------------------
@@ -467,7 +456,7 @@ For example, if we have four 1080ti, two titanx and one gfx900, the output can b
Finally, we need to change the tuning option to use RPCRunner. Use the code below
to replace the corresponding part above.
-.. GENERATED FROM PYTHON SOURCE LINES 391-410
+.. GENERATED FROM PYTHON SOURCE LINES 380-399
.. code-block:: default
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_relay_mobile_gpu.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_relay_mobile_gpu.rst.txt
index e0611d77aa..d57a91d83d 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_relay_mobile_gpu.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_relay_mobile_gpu.rst.txt
@@ -99,7 +99,7 @@ First we need to define the network in relay frontend API.
We can load some pre-defined network from :code:`relay.testing`.
We can also load models from MXNet, ONNX and TensorFlow.
-.. GENERATED FROM PYTHON SOURCE LINES 81-124
+.. GENERATED FROM PYTHON SOURCE LINES 81-113
.. code-block:: default
@@ -129,17 +129,6 @@ We can also load models from MXNet, ONNX and TensorFlow.
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={"data": input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
@@ -153,11 +142,11 @@ We can also load models from MXNet, ONNX and TensorFlow.
-.. GENERATED FROM PYTHON SOURCE LINES 125-126
+.. GENERATED FROM PYTHON SOURCE LINES 114-115
.. _tutorials-autotvm-start-rpc-tracker:
-.. GENERATED FROM PYTHON SOURCE LINES 128-152
+.. GENERATED FROM PYTHON SOURCE LINES 117-141
Start RPC Tracker
-----------------
@@ -184,7 +173,7 @@ The expected output is
INFO:RPCTracker:bind to 0.0.0.0:9190
-.. GENERATED FROM PYTHON SOURCE LINES 154-196
+.. GENERATED FROM PYTHON SOURCE LINES 143-185
Register Devices to RPC Tracker
-----------------------------------
@@ -229,7 +218,7 @@ the output can be
You can register multiple devices to the tracker to accelerate the measurement in tuning.
-.. GENERATED FROM PYTHON SOURCE LINES 198-203
+.. GENERATED FROM PYTHON SOURCE LINES 187-192
Set Tuning Options
------------------
@@ -237,7 +226,7 @@ Before tuning, we should apply some configurations. Here I use an RK3399 board
as example. In your setting, you should modify the target and device_key accordingly.
set :code:`use_android` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 203-237
+.. GENERATED FROM PYTHON SOURCE LINES 192-226
.. code-block:: default
@@ -282,7 +271,7 @@ set :code:`use_android` to True if you use android phone.
-.. GENERATED FROM PYTHON SOURCE LINES 238-246
+.. GENERATED FROM PYTHON SOURCE LINES 227-235
.. note:: How to set tuning options
@@ -293,7 +282,7 @@ set :code:`use_android` to True if you use android phone.
set timeout larger.
-.. GENERATED FROM PYTHON SOURCE LINES 249-255
+.. GENERATED FROM PYTHON SOURCE LINES 238-244
Begin Tuning
------------
@@ -302,7 +291,7 @@ Here, we provide a simple utility function to tune a list of tasks.
This function is just an initial implementation which tunes them in sequential order.
We will introduce a more sophisticated tuning scheduler in the future.
-.. GENERATED FROM PYTHON SOURCE LINES 255-329
+.. GENERATED FROM PYTHON SOURCE LINES 244-318
.. code-block:: default
@@ -387,11 +376,11 @@ We will introduce a more sophisticated tuning scheduler in the future.
-.. GENERATED FROM PYTHON SOURCE LINES 330-331
+.. GENERATED FROM PYTHON SOURCE LINES 319-320
Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 331-386
+.. GENERATED FROM PYTHON SOURCE LINES 320-375
.. code-block:: default
@@ -457,7 +446,7 @@ Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 387-419
+.. GENERATED FROM PYTHON SOURCE LINES 376-408
Sample Output
-------------
@@ -492,7 +481,7 @@ One sample output is listed below. It takes about 3 hours on a 32T AMD Ryzen Thr
Mean inference time (std dev): 128.05 ms (7.74 ms)
-.. GENERATED FROM PYTHON SOURCE LINES 421-437
+.. GENERATED FROM PYTHON SOURCE LINES 410-426
.. note:: **Experiencing Difficulties?**
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_relay_x86.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_relay_x86.rst.txt
index 8962d96f50..ae9ca5089b 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_relay_x86.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_relay_x86.rst.txt
@@ -68,7 +68,7 @@ We can also load models from MXNet, ONNX and TensorFlow.
In this tutorial, we choose resnet-18 as tuning example.
-.. GENERATED FROM PYTHON SOURCE LINES 51-117
+.. GENERATED FROM PYTHON SOURCE LINES 51-106
.. code-block:: default
@@ -98,17 +98,6 @@ In this tutorial, we choose resnet-18 as tuning example.
elif name == "inception_v3":
input_shape = (batch_size, 3, 299, 299)
mod, params = relay.testing.inception_v3.get_workload(batch_size=batch_size, dtype=dtype)
- elif name == "mxnet":
- # an example for mxnet model
- from mxnet.gluon.model_zoo.vision import get_model
-
- block = get_model("resnet18_v1", pretrained=True)
- mod, params = relay.frontend.from_mxnet(block, shape={input_name: input_shape}, dtype=dtype)
- net = mod["main"]
- net = relay.Function(
- net.params, relay.nn.softmax(net.body), None, net.type_params, net.attrs
- )
- mod = tvm.IRModule.from_expr(net)
else:
raise ValueError("Unsupported network: " + name)
@@ -145,7 +134,7 @@ In this tutorial, we choose resnet-18 as tuning example.
-.. GENERATED FROM PYTHON SOURCE LINES 118-134
+.. GENERATED FROM PYTHON SOURCE LINES 107-123
Configure tensor tuning settings and create tasks
-------------------------------------------------
@@ -164,7 +153,7 @@ times and use the average of results. In addition, we need to flush the cache
for the weight tensors between repeated measurements. This can make the measured
latency of one operator closer to its actual latency during end-to-end inference.
-.. GENERATED FROM PYTHON SOURCE LINES 134-216
+.. GENERATED FROM PYTHON SOURCE LINES 123-205
.. code-block:: default
@@ -257,11 +246,11 @@ latency of one operator closer to its actual latency during end-to-end inference
-.. GENERATED FROM PYTHON SOURCE LINES 217-218
+.. GENERATED FROM PYTHON SOURCE LINES 206-207
Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 218-273
+.. GENERATED FROM PYTHON SOURCE LINES 207-262
.. code-block:: default
@@ -327,7 +316,7 @@ Finally, we launch tuning jobs and evaluate the end-to-end performance.
-.. GENERATED FROM PYTHON SOURCE LINES 274-322
+.. GENERATED FROM PYTHON SOURCE LINES 263-311
Sample Output
-------------
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
index 8b0d727f1d..e4337e7c9b 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
@@ -360,10 +360,10 @@ Timing the untuned program
########## Build without Autotuning ##########
Node Name Ops Time(us) Time(%) Shape Inputs Outputs Measurements(us)
--------- --- -------- ------- ----- ------ ------- ----------------
- tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 304.4 98.71 (1, 2, 10, 10, 3) 2 1 [304.4]
- tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 3.008 0.976 (1, 6, 10, 10) 1 1 [3.008]
- tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.969 0.314 (1, 1, 10, 10, 3) 1 1 [0.969]
- Total_time - 308.378 - - - - -
+ tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 294.4 98.723 (1, 2, 10, 10, 3) 2 1 [294.4]
+ tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 2.869 0.962 (1, 6, 10, 10) 1 1 [2.869]
+ tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.939 0.315 (1, 1, 10, 10, 3) 1 1 [0.939]
+ Total_time - 298.208 - - - - -
@@ -428,10 +428,10 @@ Timing the tuned program
########## Build with Autotuning ##########
Node Name Ops Time(us) Time(%) Shape Inputs Outputs Measurements(us)
--------- --- -------- ------- ----- ------ ------- ----------------
- tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 101.8 97.383 (1, 6, 10, 10, 1) 2 1 [101.8]
- tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 1.777 1.7 (1, 6, 10, 10) 1 1 [1.777]
- tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.959 0.918 (1, 1, 10, 10, 3) 1 1 [0.959]
- Total_time - 104.536 - - - - -
+ tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 99.625 97.529 (1, 6, 10, 10, 1) 2 1 [99.625]
+ tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 1.703 1.667 (1, 6, 10, 10) 1 1 [1.703]
+ tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.821 0.804 (1, 3, 10, 10, 1) 1 1 [0.821]
+ Total_time - 102.149 - - - - -
@@ -439,7 +439,7 @@ Timing the tuned program
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 27.824 seconds)
+ **Total running time of the script:** ( 1 minutes 22.930 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_autotune.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
index c29a75ec46..4104e97aa6 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
@@ -118,7 +118,7 @@ download a cat image and preprocess it to use as the model input.
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/quantization/utils.py:310: UserWarning: must run observer before calling calculate_qparams. Returning default values.
warnings.warn(
Downloading: "https://download.pytorch.org/models/quantized/mobilenet_v2_qnnpack_37f702c5.pth" to /workspace/.cache/torch/hub/checkpoints/mobilenet_v2_qnnpack_37f702c5.pth
- 0%| | 0.00/3.42M [00:00<?, ?B/s] 61%|###### | 2.09M/3.42M [00:00<00:00, 12.4MB/s] 100%|##########| 3.42M/3.42M [00:00<00:00, 19.4MB/s]
+ 0%| | 0.00/3.42M [00:00<?, ?B/s] 61%|###### | 2.09M/3.42M [00:00<00:00, 13.6MB/s] 100%|##########| 3.42M/3.42M [00:00<00:00, 21.5MB/s]
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/_utils.py:314: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
device=storage.device,
/workspace/python/tvm/relay/frontend/pytorch_utils.py:47: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
@@ -326,7 +326,7 @@ Look up prediction top 1 index in 1000 class synset.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 31.348 seconds)
+ **Total running time of the script:** ( 1 minutes 25.303 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_pytorch.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
index 5b41ef0e14..7b96ee7c8c 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
@@ -217,7 +217,7 @@ take about **2 minutes** to download the Stanford Cars, while COCO 2017 validati
.. code-block:: none
- '/tmp/tmpbffr4r4d/images/random'
+ '/tmp/tmpvxxcw420/images/random'
@@ -317,8 +317,8 @@ objects to other stuff? We can display some examples from our datasets using ``m
.. code-block:: none
- /tmp/tmpbffr4r4d/images/target contains 8144 images
- /tmp/tmpbffr4r4d/images/random contains 5000 images
+ /tmp/tmpvxxcw420/images/target contains 8144 images
+ /tmp/tmpvxxcw420/images/random contains 5000 images
@@ -493,13 +493,13 @@ the time on our validation set).
.. code-block:: none
Epoch 1/3
- 328/328 - 40s - loss: 0.2225 - accuracy: 0.9244 - val_loss: 0.1222 - val_accuracy: 0.9569 - 40s/epoch - 123ms/step
+ 328/328 - 39s - loss: 0.2332 - accuracy: 0.9227 - val_loss: 0.1303 - val_accuracy: 0.9520 - 39s/epoch - 117ms/step
Epoch 2/3
- 328/328 - 35s - loss: 0.1037 - accuracy: 0.9633 - val_loss: 0.1382 - val_accuracy: 0.9430 - 35s/epoch - 108ms/step
+ 328/328 - 34s - loss: 0.1017 - accuracy: 0.9624 - val_loss: 0.0958 - val_accuracy: 0.9649 - 34s/epoch - 104ms/step
Epoch 3/3
- 328/328 - 35s - loss: 0.0686 - accuracy: 0.9741 - val_loss: 0.0970 - val_accuracy: 0.9698 - 35s/epoch - 108ms/step
+ 328/328 - 34s - loss: 0.0661 - accuracy: 0.9766 - val_loss: 0.0981 - val_accuracy: 0.9671 - 34s/epoch - 104ms/step
- <keras.callbacks.History object at 0x7f6998227040>
+ <keras.callbacks.History object at 0x7fd4b3472640>
@@ -860,7 +860,7 @@ Arduino tutorial for how to do that `on GitHub <https://github.com/guberti/tvm-a
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 4 minutes 55.007 seconds)
+ **Total running time of the script:** ( 4 minutes 49.107 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_train.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
index 0b59285c1e..865b58cf59 100644
--- a/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
@@ -5,20 +5,20 @@
Computation times
=================
-**08:24.098** total execution time for **how_to_work_with_microtvm** files:
+**08:04.731** total execution time for **how_to_work_with_microtvm** files:
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_train.py` (``micro_train.py``) | 04:55.007 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_train.py` (``micro_train.py``) | 04:49.107 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_pytorch.py` (``micro_pytorch.py``) | 01:31.348 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_pytorch.py` (``micro_pytorch.py``) | 01:25.303 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_autotune.py` (``micro_autotune.py``) | 01:27.824 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_autotune.py` (``micro_autotune.py``) | 01:22.930 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_aot.py` (``micro_aot.py``) | 00:12.319 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_aot.py` (``micro_aot.py``) | 00:11.538 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_custom_ide.py` (``micro_custom_ide.py``) | 00:09.142 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_custom_ide.py` (``micro_custom_ide.py``) | 00:08.595 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_tflite.py` (``micro_tflite.py``) | 00:08.458 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_tflite.py` (``micro_tflite.py``) | 00:07.258 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_work_with_microtvm_micro_ethosu.py` (``micro_ethosu.py``) | 00:00.000 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
index a99db4e8fb..c6f0fa08ec 100644
--- a/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
@@ -5,14 +5,14 @@
Computation times
=================
-**00:39.346** total execution time for **how_to_work_with_relay** files:
+**00:37.898** total execution time for **how_to_work_with_relay** files:
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_pipeline_executor.py` (``using_pipeline_executor.py``) | 00:34.242 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_pipeline_executor.py` (``using_pipeline_executor.py``) | 00:32.946 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_external_lib.py` (``using_external_lib.py``) | 00:03.185 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_external_lib.py` (``using_external_lib.py``) | 00:03.121 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_build_gcn.py` (``build_gcn.py``) | 00:01.913 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_build_gcn.py` (``build_gcn.py``) | 00:01.825 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_relay_viz.py` (``using_relay_viz.py``) | 00:00.006 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_relay_viz.py` (``using_relay_viz.py``) | 00:00.007 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_schedules/intrin_math.rst.txt b/docs/_sources/how_to/work_with_schedules/intrin_math.rst.txt
index 5d92e67232..b15bc2701e 100644
--- a/docs/_sources/how_to/work_with_schedules/intrin_math.rst.txt
+++ b/docs/_sources/how_to/work_with_schedules/intrin_math.rst.txt
@@ -281,7 +281,7 @@ The following example customizes CUDA lowering rule for :code:`exp`.
.. code-block:: none
- <function my_cuda_math_rule at 0x7f6b6e0a6820>
+ <function my_cuda_math_rule at 0x7fd4cea1e9d0>
diff --git a/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
index c3d8883468..8964815010 100644
--- a/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
@@ -5,22 +5,22 @@
Computation times
=================
-**00:05.721** total execution time for **how_to_work_with_schedules** files:
+**00:05.469** total execution time for **how_to_work_with_schedules** files:
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_intrin_math.py` (``intrin_math.py``) | 00:02.590 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_intrin_math.py` (``intrin_math.py``) | 00:02.571 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tensorize.py` (``tensorize.py``) | 00:01.372 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tensorize.py` (``tensorize.py``) | 00:01.174 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_reduction.py` (``reduction.py``) | 00:00.739 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_reduction.py` (``reduction.py``) | 00:00.730 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_scan.py` (``scan.py``) | 00:00.730 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_scan.py` (``scan.py``) | 00:00.713 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_extern_op.py` (``extern_op.py``) | 00:00.119 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_extern_op.py` (``extern_op.py``) | 00:00.118 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_schedule_primitives.py` (``schedule_primitives.py``) | 00:00.072 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_schedule_primitives.py` (``schedule_primitives.py``) | 00:00.069 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tedd.py` (``tedd.py``) | 00:00.067 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tedd.py` (``tedd.py``) | 00:00.064 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tuple_inputs.py` (``tuple_inputs.py``) | 00:00.032 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tuple_inputs.py` (``tuple_inputs.py``) | 00:00.030 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/autotvm/index.rst.txt b/docs/_sources/topic/vta/tutorials/autotvm/index.rst.txt
deleted file mode 100644
index 6ef5d82bef..0000000000
--- a/docs/_sources/topic/vta/tutorials/autotvm/index.rst.txt
+++ /dev/null
@@ -1,59 +0,0 @@
-
-
-.. _sphx_glr_topic_vta_tutorials_autotvm:
-
-Auto tuning
--------------
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="Auto-tuning a ALU fused op on VTA">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/autotvm/images/thumb/sphx_glr_tune_alu_vta_thumb.png
- :alt: Auto-tuning a ALU fused op on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_autotvm_tune_alu_vta.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Auto-tuning a ALU fused op on VTA</div>
- </div>
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="Auto-tuning for a specific accelerator design is critical for getting the best performance for ...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/autotvm/images/thumb/sphx_glr_tune_relay_vta_thumb.png
- :alt: Auto-tuning a convolutional network on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_autotvm_tune_relay_vta.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Auto-tuning a convolutional network on VTA</div>
- </div>
-
-
-.. raw:: html
-
- </div>
-
-
-.. toctree::
- :hidden:
-
- /topic/vta/tutorials/autotvm/tune_alu_vta
- /topic/vta/tutorials/autotvm/tune_relay_vta
-
diff --git a/docs/_sources/topic/vta/tutorials/autotvm/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/autotvm/sg_execution_times.rst.txt
deleted file mode 100644
index af392ea9e8..0000000000
--- a/docs/_sources/topic/vta/tutorials/autotvm/sg_execution_times.rst.txt
+++ /dev/null
@@ -1,14 +0,0 @@
-
-:orphan:
-
-.. _sphx_glr_topic_vta_tutorials_autotvm_sg_execution_times:
-
-Computation times
-=================
-**00:35.924** total execution time for **topic_vta_tutorials_autotvm** files:
-
-+---------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_autotvm_tune_relay_vta.py` (``tune_relay_vta.py``) | 00:35.916 | 0.0 MB |
-+---------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_autotvm_tune_alu_vta.py` (``tune_alu_vta.py``) | 00:00.008 | 0.0 MB |
-+---------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/autotvm/tune_alu_vta.rst.txt b/docs/_sources/topic/vta/tutorials/autotvm/tune_alu_vta.rst.txt
deleted file mode 100644
index 9ac5191331..0000000000
--- a/docs/_sources/topic/vta/tutorials/autotvm/tune_alu_vta.rst.txt
+++ /dev/null
@@ -1,443 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/autotvm/tune_alu_vta.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_autotvm_tune_alu_vta.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/178b6f23dffc01ac92f2cf95f41a5679/tune_alu_vta.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_autotvm_tune_alu_vta.py:
-
-
-Auto-tuning a ALU fused op on VTA
----------------------------------
-
-.. GENERATED FROM PYTHON SOURCE LINES 21-43
-
-.. code-block:: default
-
-
- import os
- import sys
- from mxnet.gluon.model_zoo import vision
- import numpy as np
- from PIL import Image
-
- from tvm import topi
- import tvm
- from tvm import te
- from tvm import rpc, autotvm, relay
- from tvm.contrib import download
- from tvm.autotvm.measure.measure_methods import request_remote
- from tvm.autotvm.tuner import XGBTuner, GATuner, RandomTuner, GridSearchTuner
- from tvm.autotvm import record
-
- import vta
- from vta.testing import simulator
- from vta.top import graph_pack
- import copy
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 44-47
-
-Compile network
----------------
-Perform vta-specific compilation with Relay from a Gluon model
-
-.. GENERATED FROM PYTHON SOURCE LINES 47-82
-
-.. code-block:: default
-
- def compile_network(env, target, model, start_pack, stop_pack):
-
- # Populate the shape and data type dictionary
- dtype_dict = {"data": "float32"}
- shape_dict = {"data": (env.BATCH, 3, 224, 224)}
-
- # Get off the shelf gluon model, and convert to relay
- gluon_model = vision.get_model(model, pretrained=True)
- mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)
-
- # Update shape and type dictionary
- shape_dict.update({k: v.shape for k, v in params.items()})
- dtype_dict.update({k: str(v.dtype) for k, v in params.items()})
-
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with relay.build_config(opt_level=3):
- with relay.quantize.qconfig(global_scale=8.0, skip_conv_layers=[0]):
- mod = relay.quantize.quantize(mod, params=params)
-
- # Perform graph packing and constant folding for VTA target
- if target.device_name == "vta":
- assert env.BLOCK_IN == env.BLOCK_OUT
- relay_prog = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=start_pack,
- stop_name=stop_pack,
- )
-
- return relay_prog, params
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 83-87
-
-Set Tuning Options
-------------------
-Before tuning, we should apply some configurations.
-Here we use an Pynq-Z1 board as an example.
-
-.. GENERATED FROM PYTHON SOURCE LINES 87-237
-
-.. code-block:: default
-
-
- # Tracker host and port can be set by your environment
- tracker_host = os.environ.get("TVM_TRACKER_HOST", "0.0.0.0")
- tracker_port = int(os.environ.get("TVM_TRACKER_PORT", 9190))
-
- # Load VTA parameters from the vta/config/vta_config.json file
- env = vta.get_env()
-
- # This target is used for cross compilation. You can query it by :code:`gcc -v` on your device.
- # Set ``device=arm_cpu`` to run inference on the CPU
- # or ``device=vta`` to run inference on the FPGA.
- device = "vta"
- target = env.target if device == "vta" else env.target_vta_cpu
-
- # Name of Gluon model to compile
- # The ``start_pack`` and ``stop_pack`` labels indicate where
- # to start and end the graph packing relay pass: in other words
- # where to start and finish offloading to VTA.
- network = "resnet50_v2"
- start_pack = "nn.max_pool2d"
- stop_pack = "nn.global_avg_pool2d"
-
- # Tuning option
- log_file = "%s.alu.%s.log" % (device, network)
- tuning_option = {
- "log_filename": log_file,
- "tuner": "random",
- "n_trial": 1000,
- "early_stopping": None,
- "measure_option": autotvm.measure_option(
- builder=autotvm.LocalBuilder(n_parallel=1),
- runner=autotvm.RPCRunner(
- env.TARGET,
- host=tracker_host,
- port=tracker_port,
- number=5,
- timeout=60,
- # check_correctness=True, # TODO: re-enable when check_correctness works again.
- ),
- ),
- }
-
-
- def log_to_file(file_out, protocol="json"):
- """Log the tuning records into file.
- The rows of the log are stored in the format of autotvm.record.encode.
- for lhs == rhs, we add an extra rhs = [] record
-
- Parameters
- ----------
- file_out : str
- The file to log to.
- protocol: str, optional
- The log protocol. Can be 'json' or 'pickle'
-
- Returns
- -------
- callback : callable
- Callback function to do the logging.
- """
-
- def _callback(_, inputs, results):
- with open(file_out, "a") as f:
- for inp, result in zip(inputs, results):
- f.write(record.encode(inp, result, protocol) + "\n")
-
- # we only consider task with same lhs and rhs
- if inp.task.args[0] == inp.task.args[1]:
- args = list(inp.task.args)
- args[1] = (args[0][0], (), args[0][2])
- inp_copy = copy.deepcopy(inp)
- inp_copy.task.args = tuple(args)
- f.write(record.encode(inp_copy, result, protocol) + "\n")
-
- return _callback
-
-
- def tune_tasks(
- tasks,
- measure_option,
- tuner="xgb",
- n_trial=10,
- early_stopping=None,
- log_filename="tuning.log",
- use_transfer_learning=True,
- ):
-
- # create tmp log file
- tmp_log_file = log_filename + ".tmp"
- if os.path.exists(tmp_log_file):
- os.remove(tmp_log_file)
-
- for i, tsk in enumerate(reversed(tasks)):
- prefix = "[Task %2d/%2d] " % (i + 1, len(tasks))
-
- # create tuner
- if tuner == "xgb":
- tuner_obj = XGBTuner(tsk, loss_type="reg")
- elif tuner == "xgb_knob":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="knob")
- elif tuner == "xgb_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="itervar")
- elif tuner == "xgb_curve":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="curve")
- elif tuner == "xgb_rank":
- tuner_obj = XGBTuner(tsk, loss_type="rank")
- elif tuner == "xgb_rank_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="knob")
- elif tuner == "xgb_rank_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="itervar")
- elif tuner == "xgb_rank_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="curve")
- elif tuner == "xgb_rank_binary":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary")
- elif tuner == "xgb_rank_binary_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="knob")
- elif tuner == "xgb_rank_binary_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="itervar")
- elif tuner == "xgb_rank_binary_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="curve")
- elif tuner == "ga":
- tuner_obj = GATuner(tsk, pop_size=50)
- elif tuner == "random":
- tuner_obj = RandomTuner(tsk)
- elif tuner == "gridsearch":
- tuner_obj = GridSearchTuner(tsk)
- else:
- raise ValueError("Invalid tuner: " + tuner)
-
- if use_transfer_learning:
- if os.path.isfile(tmp_log_file):
- tuner_obj.load_history(autotvm.record.load_from_file(tmp_log_file))
-
- # do tuning
- tsk_trial = min(n_trial, len(tsk.config_space))
- tuner_obj.tune(
- n_trial=tsk_trial,
- early_stopping=early_stopping,
- measure_option=measure_option,
- callbacks=[
- autotvm.callback.progress_bar(tsk_trial, prefix=prefix),
- log_to_file(tmp_log_file),
- ],
- )
-
- # pick best records to a cache file
- autotvm.record.pick_best(tmp_log_file, log_filename)
- os.remove(tmp_log_file)
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 238-239
-
-Register VTA-specific tuning tasks
-
-.. GENERATED FROM PYTHON SOURCE LINES 239-287
-
-.. code-block:: default
-
- def register_vta_tuning_tasks():
- from tvm.autotvm.task import TaskExtractEnv
-
- @tvm.te.tag_scope(tag=topi.tag.ELEMWISE)
- def my_clip(x, a_min, a_max):
- """Unlike topi's current clip, put min and max into two stages."""
- const_min = tvm.tir.const(a_min, x.dtype)
- const_max = tvm.tir.const(a_max, x.dtype)
- x = te.compute(x.shape, lambda *i: tvm.te.min(x(*i), const_max), name="clipA")
- x = te.compute(x.shape, lambda *i: tvm.te.max(x(*i), const_min), name="clipB")
- return x
-
- # init autotvm env to register VTA operator
- TaskExtractEnv()
-
- @autotvm.template("add.vta")
- def _topi_add(*args, **kwargs):
- assert not kwargs, "Do not support kwargs in template function call"
- A, B = args[:2]
-
- with tvm.target.vta():
- res = vta.top.op.add_packed(*args, **kwargs)
- res = my_clip(res, 0, 127)
- res = topi.cast(res, "int8")
-
- if tvm.target.Target.current().device_name == "vta":
- s = vta.top.op.schedule_add_packed([res])
- else:
- s = te.create_schedule([res.op])
- return s, [A, B, res]
-
- @autotvm.template("multiply.vta")
- def _topi_multiply(*args, **kwargs):
- assert not kwargs, "Do not support kwargs in template function call"
- A, B = args[:2]
-
- with tvm.target.vta():
- res = vta.top.op.multiply_packed(*args, **kwargs)
- res = my_clip(res, 0, 127)
- res = topi.cast(res, "int8")
-
- if tvm.target.Target.current().device_name == "vta":
- s = vta.top.op.schedule_multiply_packed([res])
- else:
- s = te.create_schedule([res.op])
- return s, [A, B, res]
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 288-289
-
-Finally, we launch tuning jobs and evaluate the end-to-end performance.
-
-.. GENERATED FROM PYTHON SOURCE LINES 289-346
-
-.. code-block:: default
-
- def tune_and_evaluate(tuning_opt):
-
- if env.TARGET != "intelfocl":
- print("ALU only op only available for intelfocl target")
- return
-
- # Register VTA tuning tasks
- register_vta_tuning_tasks()
-
- # Perform task extraction on Relay program
- print("Extract tasks...")
- relay_prog, params = compile_network(env, target, network, start_pack, stop_pack)
- mod = tvm.IRModule.from_expr(relay_prog)
- tasks = autotvm.task.extract_from_program(
- mod,
- params=params,
- ops=(
- relay.op.get("add"),
- relay.op.get("multiply"),
- ),
- target=tvm.target.Target(target, host=env.target_host),
- )
-
- # filter out non-packed alu task
- tasks = list(filter(lambda t: len(t.args[0][1]) > 4, tasks))
- # filter out float alu task
- tasks = list(filter(lambda t: t.args[0][2] != "float32", tasks))
-
- # We should have extracted 10 convolution tasks
- tasks_set = {}
- print("Extracted {} alu tasks:".format(len(tasks)))
- for tsk in tasks:
- print("tsk = ", tsk)
-
- if len(tsk.args[1][1]) == 0:
- args = list(tsk.args)
- args[1] = args[0]
- tsk.args = tuple(args)
-
- if (tsk.name, tsk.args) in tasks_set:
- print("task {} already exists".format(tsk))
- tasks_set[(tsk.name, tsk.args)] = tsk
-
- tasks = list(tasks_set.values())
- print("After merged, final #tasks={}, tasks = {}".format(len(tasks), tasks))
-
- # run tuning tasks
- print("Tuning...")
- tune_tasks(tasks, **tuning_opt)
-
-
- # Run the tuning and evaluate the results
- try:
- tune_and_evaluate(tuning_option)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- ALU only op only available for intelfocl target
-
-
-
-
-
-.. _sphx_glr_download_topic_vta_tutorials_autotvm_tune_alu_vta.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: tune_alu_vta.py <tune_alu_vta.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: tune_alu_vta.ipynb <tune_alu_vta.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/autotvm/tune_relay_vta.rst.txt b/docs/_sources/topic/vta/tutorials/autotvm/tune_relay_vta.rst.txt
deleted file mode 100644
index 9394fc7117..0000000000
--- a/docs/_sources/topic/vta/tutorials/autotvm/tune_relay_vta.rst.txt
+++ /dev/null
@@ -1,666 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/autotvm/tune_relay_vta.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_autotvm_tune_relay_vta.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/b1b0cbd807166348a0eabbad6bfbbdaf/tune_relay_vta.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_autotvm_tune_relay_vta.py:
-
-
-Auto-tuning a convolutional network on VTA
-==========================================
-**Author**: `Lianmin Zheng <https://github.com/merrymercy>`_, `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-Auto-tuning for a specific accelerator design is critical for getting the best
-performance for any given operator. This is a tutorial showcases how to tune a
-whole convolutional network on VTA.
-
-The operator implementation for VTA in TVM is written in template form.
-The template has many tunable knobs (tile factor, virtual threads, etc).
-We will tune all convolution operators in the neural network. After tuning,
-we produce a log file which stores the best schedule parameters for all tuned
-operators. When the TVM compiler compiles these operators, it will query this
-log file to get the best knob parameters.
-
-.. GENERATED FROM PYTHON SOURCE LINES 36-55
-
-Install dependencies
---------------------
-To use the autotvm package in tvm, we need to install some extra dependencies.
-(change "3" to "2" if you use python2):
-
-.. code-block:: bash
-
- pip3 install --user psutil xgboost tornado mxnet requests "Pillow<7" cloudpickle
-
-To make TVM run faster during tuning, it is recommended to use cython
-as FFI of TVM. In the root directory of TVM, execute
-(change "3" to "2" if you use python2):
-
-.. code-block:: bash
-
- pip3 install --user cython
- sudo make cython3
-
-Now return to python code. Import packages.
-
-.. GENERATED FROM PYTHON SOURCE LINES 55-75
-
-.. code-block:: default
-
-
- import os
- import sys
-
- from mxnet.gluon.model_zoo import vision
- import numpy as np
- from PIL import Image
-
- from tvm import topi
- import tvm
- from tvm import te
- from tvm import rpc, autotvm, relay
- from tvm.contrib import graph_executor, utils, download
- from tvm.autotvm.measure.measure_methods import request_remote
- from tvm.autotvm.tuner import XGBTuner, GATuner, RandomTuner, GridSearchTuner
-
- import vta
- from vta.testing import simulator
- from vta.top import graph_pack
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 76-79
-
-Compile network
----------------
-Perform vta-specific compilation with Relay from a Gluon model
-
-.. GENERATED FROM PYTHON SOURCE LINES 79-116
-
-.. code-block:: default
-
-
-
- def compile_network(env, target, model, start_pack, stop_pack):
-
- # Populate the shape and data type dictionary
- dtype_dict = {"data": "float32"}
- shape_dict = {"data": (env.BATCH, 3, 224, 224)}
-
- # Get off the shelf gluon model, and convert to relay
- gluon_model = vision.get_model(model, pretrained=True)
- mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)
-
- # Update shape and type dictionary
- shape_dict.update({k: v.shape for k, v in params.items()})
- dtype_dict.update({k: str(v.dtype) for k, v in params.items()})
-
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with tvm.transform.PassContext(opt_level=3):
- with relay.quantize.qconfig(global_scale=8.0, skip_conv_layers=[0]):
- mod = relay.quantize.quantize(mod, params=params)
-
- # Perform graph packing and constant folding for VTA target
- if target.device_name == "vta":
- assert env.BLOCK_IN == env.BLOCK_OUT
- relay_prog = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=start_pack,
- stop_name=stop_pack,
- )
-
- return relay_prog, params
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 117-141
-
-Start RPC Tracker
------------------
-TVM uses an RPC session to communicate with Pynq boards.
-During tuning, the tuner will send the generated code to the board and
-measure the speed of code on the board.
-
-To scale up tuning, TVM uses an RPC Tracker to manage multiple devices.
-The RPC Tracker is a centralized controller node. We can register all devices to
-the tracker. For example, if we have 10 Pynq boards, we can register all of them
-to the tracker, and run 10 measurements in parallel, accelerating the tuning process.
-
-To start an RPC tracker, run this command on the host machine. The tracker is
-required during the whole tuning process, so we need to open a new terminal for
-this command:
-
-.. code-block:: bash
-
- python -m tvm.exec.rpc_tracker --host=0.0.0.0 --port=9190
-
-The expected output is:
-
-.. code-block:: bash
-
- INFO:RPCTracker:bind to 0.0.0.0:9190
-
-.. GENERATED FROM PYTHON SOURCE LINES 143-177
-
-Register devices to RPC Tracker
------------------------------------
-Now we can register our devices to the tracker. The first step is to
-build the TVM runtime for the Pynq devices.
-
-Follow :ref:`vta-index`
-to build the TVM runtime on the device. Then register the device to the tracker with:
-
-.. code-block:: bash
-
- python -m tvm.exec.rpc_server --tracker=[HOST_IP]:9190 --key=pynq
-
-(replace :code:`[HOST_IP]` with the IP address of your host machine)
-
-After registering devices, we can confirm it by querying the rpc_tracker:
-
-.. code-block:: bash
-
- python -m tvm.exec.query_rpc_tracker --host=0.0.0.0 --port=9190
-
-For example, if we have 6 Pynq boards and 11 Raspberry Pi 3B,
-the output can be
-
-.. code-block:: bash
-
- Queue Status
- ----------------------------------
- key total free pending
- ----------------------------------
- pynq 6 6 0
- rpi3b 11 11 0
- ----------------------------------
-
-You can register multiple devices to the tracker to accelerate tuning.
-
-.. GENERATED FROM PYTHON SOURCE LINES 179-183
-
-Set Tuning Options
-------------------
-Before tuning, we should apply some configurations.
-Here we use an Pynq-Z1 board as an example.
-
-.. GENERATED FROM PYTHON SOURCE LINES 183-226
-
-.. code-block:: default
-
-
- # Tracker host and port can be set by your environment
- tracker_host = os.environ.get("TVM_TRACKER_HOST", "127.0.0.1")
- tracker_port = int(os.environ.get("TVM_TRACKER_PORT", 9190))
-
- # Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
- env = vta.get_env()
-
- # This target is used for cross compilation. You can query it by :code:`gcc -v` on your device.
- # Set ``device=arm_cpu`` to run inference on the CPU
- # or ``device=vta`` to run inference on the FPGA.
- device = "vta"
- target = env.target if device == "vta" else env.target_vta_cpu
-
- # Name of Gluon model to compile
- # The ``start_pack`` and ``stop_pack`` labels indicate where
- # to start and end the graph packing relay pass: in other words
- # where to start and finish offloading to VTA.
- network = "resnet18_v1"
- start_pack = "nn.max_pool2d"
- stop_pack = "nn.global_avg_pool2d"
-
- # Tuning option
- log_file = "%s.%s.log" % (device, network)
- tuning_option = {
- "log_filename": log_file,
- "tuner": "random",
- "n_trial": 1000,
- "early_stopping": None,
- "measure_option": autotvm.measure_option(
- builder=autotvm.LocalBuilder(),
- runner=autotvm.RPCRunner(
- env.TARGET,
- host=tracker_host,
- port=tracker_port,
- number=5,
- timeout=60,
- module_loader=vta.module_loader(),
- # check_correctness=True, # TODO: re-enable when check_correctness works again.
- ),
- ),
- }
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 227-235
-
-.. note:: How to set tuning options
-
- In general, the default values provided here work well.
- If you have enough time budget, you can set :code:`n_trial`, :code:`early_stopping`
- to larger values, makes the tuning run for longer.
- If your device is under-powered or your conv2d operators are large, consider
- setting a longer timeout.
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 238-247
-
-Begin Tuning
-------------
-Now we can extract tuning tasks from the network and begin tuning.
-Here, we provide a simple utility function to tune a list of tasks.
-This function is just an initial implementation which tunes them in sequential order.
-We will introduce a more sophisticated tuning scheduler in the future.
-
-Given that the tuning will be done on Pynq FPGA boards, make sure that
-the ```TARGET`` entry in the ``vta_config.json`` file is set to ``pynq``.
-
-.. GENERATED FROM PYTHON SOURCE LINES 247-323
-
-.. code-block:: default
-
-
-
- # You can skip the implementation of this function for this tutorial.
- def tune_tasks(
- tasks,
- measure_option,
- tuner="xgb",
- n_trial=1000,
- early_stopping=None,
- log_filename="tuning.log",
- use_transfer_learning=True,
- ):
-
- # create tmp log file
- tmp_log_file = log_filename + ".tmp"
- if os.path.exists(tmp_log_file):
- os.remove(tmp_log_file)
-
- for i, tsk in enumerate(reversed(tasks)):
- prefix = "[Task %2d/%2d] " % (i + 1, len(tasks))
-
- # create tuner
- if tuner == "xgb":
- tuner_obj = XGBTuner(tsk, loss_type="reg")
- elif tuner == "xgb_knob":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="knob")
- elif tuner == "xgb_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="itervar")
- elif tuner == "xgb_curve":
- tuner_obj = XGBTuner(tsk, loss_type="reg", feature_type="curve")
- elif tuner == "xgb_rank":
- tuner_obj = XGBTuner(tsk, loss_type="rank")
- elif tuner == "xgb_rank_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="knob")
- elif tuner == "xgb_rank_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="itervar")
- elif tuner == "xgb_rank_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank", feature_type="curve")
- elif tuner == "xgb_rank_binary":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary")
- elif tuner == "xgb_rank_binary_knob":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="knob")
- elif tuner == "xgb_rank_binary_itervar":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="itervar")
- elif tuner == "xgb_rank_binary_curve":
- tuner_obj = XGBTuner(tsk, loss_type="rank-binary", feature_type="curve")
- elif tuner == "ga":
- tuner_obj = GATuner(tsk, pop_size=50)
- elif tuner == "random":
- tuner_obj = RandomTuner(tsk)
- elif tuner == "gridsearch":
- tuner_obj = GridSearchTuner(tsk)
- else:
- raise ValueError("Invalid tuner: " + tuner)
-
- if use_transfer_learning:
- if os.path.isfile(tmp_log_file):
- tuner_obj.load_history(autotvm.record.load_from_file(tmp_log_file))
-
- # do tuning
- tsk_trial = min(n_trial, len(tsk.config_space))
- tuner_obj.tune(
- n_trial=tsk_trial,
- early_stopping=early_stopping,
- measure_option=measure_option,
- callbacks=[
- autotvm.callback.progress_bar(tsk_trial, prefix=prefix),
- autotvm.callback.log_to_file(tmp_log_file),
- ],
- )
-
- # pick best records to a cache file
- autotvm.record.pick_best(tmp_log_file, log_filename)
- os.remove(tmp_log_file)
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 324-325
-
-Register VTA-specific tuning tasks
-
-.. GENERATED FROM PYTHON SOURCE LINES 325-360
-
-.. code-block:: default
-
-
-
- def register_vta_tuning_tasks():
- from tvm.autotvm.task import TaskExtractEnv
-
- @tvm.te.tag_scope(tag=topi.tag.ELEMWISE)
- def my_clip(x, a_min, a_max):
- """Unlike topi's current clip, put min and max into two stages."""
- const_min = tvm.tir.const(a_min, x.dtype)
- const_max = tvm.tir.const(a_max, x.dtype)
- x = te.compute(x.shape, lambda *i: tvm.te.min(x(*i), const_max), name="clipA")
- x = te.compute(x.shape, lambda *i: tvm.te.max(x(*i), const_min), name="clipB")
- return x
-
- # init autotvm env to register VTA operator
- TaskExtractEnv()
-
- @autotvm.template("conv2d_packed.vta")
- def _topi_nn_conv2d(*args, **kwargs):
- assert not kwargs, "Do not support kwargs in template function call"
- A, W = args[:2]
-
- with tvm.target.vta():
- res = vta.top.conv2d_packed(*args, **kwargs)
- res = topi.right_shift(res, 8)
- res = my_clip(res, 0, 127)
- res = topi.cast(res, "int8")
-
- if tvm.target.Target.current().device_name == "vta":
- s = vta.top.schedule_conv2d_packed([res])
- else:
- s = te.create_schedule([res.op])
- return s, [A, W, res]
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 361-362
-
-Finally, we launch tuning jobs and evaluate the end-to-end performance.
-
-.. GENERATED FROM PYTHON SOURCE LINES 362-482
-
-.. code-block:: default
-
-
-
- def tune_and_evaluate(tuning_opt):
-
- # Register VTA tuning tasks
- register_vta_tuning_tasks()
-
- # Perform task extraction on Relay program
- print("Extract tasks...")
- relay_prog, params = compile_network(env, target, network, start_pack, stop_pack)
- mod = tvm.IRModule.from_expr(relay_prog)
- tasks = autotvm.task.extract_from_program(
- mod,
- params=params,
- ops=(relay.op.get("nn.conv2d"),),
- target=target,
- target_host=env.target_host,
- )
-
- # filter out non-packed conv2d task
- tasks = list(filter(lambda t: len(t.args[0][1]) > 4 and "conv" in t.name, tasks))
-
- # We should have extracted 10 convolution tasks
- assert len(tasks) == 10
- print("Extracted {} conv2d tasks:".format(len(tasks)))
- for tsk in tasks:
- inp = tsk.args[0][1]
- wgt = tsk.args[1][1]
- batch = inp[0] * inp[4]
- in_filter = inp[1] * inp[5]
- out_filter = wgt[0] * wgt[4]
- height, width = inp[2], inp[3]
- hkernel, wkernel = wgt[2], wgt[3]
- hstride, wstride = tsk.args[2][0], tsk.args[2][1]
- hpad, wpad = tsk.args[3][0], tsk.args[3][1]
- print(
- "({}, {}, {}, {}, {}, {}, {}, {}, {}, {}, {})".format(
- batch,
- height,
- width,
- in_filter,
- out_filter,
- hkernel,
- wkernel,
- hpad,
- wpad,
- hstride,
- wstride,
- )
- )
-
- # We do not run the tuning in our webpage server since it takes too long.
- # Comment the following line to run it by yourself.
- return
-
- # run tuning tasks
- print("Tuning...")
- tune_tasks(tasks, **tuning_opt)
-
- # evaluate with tuning history
- if env.TARGET != "sim":
- # Get remote from fleet node
- remote = autotvm.measure.request_remote(
- env.TARGET, tracker_host, tracker_port, timeout=10000
- )
- # Reconfigure the JIT runtime and FPGA.
- vta.reconfig_runtime(remote)
- vta.program_fpga(remote, bitstream=None)
- else:
- # In simulation mode, host the RPC server locally.
- remote = rpc.LocalSession()
-
- # compile kernels with history best records
- with autotvm.tophub.context(target, extra_files=[log_file]):
- # Compile network
- print("Compile...")
- if target.device_name != "vta":
- with tvm.transform.PassContext(opt_level=3, disabled_pass={"AlterOpLayout"}):
- lib = relay.build(
- relay_prog, target=target, params=params, target_host=env.target_host
- )
- else:
- with vta.build_config(opt_level=3, disabled_pass={"AlterOpLayout"}):
- lib = relay.build(
- relay_prog, target=target, params=params, target_host=env.target_host
- )
-
- # Export library
- print("Upload...")
- temp = utils.tempdir()
- lib.export_library(temp.relpath("graphlib.tar"))
- remote.upload(temp.relpath("graphlib.tar"))
- lib = remote.load_module("graphlib.tar")
-
- # Generate the graph executor
- ctx = remote.ext_dev(0) if device == "vta" else remote.cpu(0)
- m = graph_executor.GraphModule(lib["default"](ctx))
-
- # upload parameters to device
- image = tvm.nd.array((np.random.uniform(size=(1, 3, 224, 224))).astype("float32"))
- m.set_input("data", image)
-
- # evaluate
- print("Evaluate inference time cost...")
- timer = m.module.time_evaluator("run", ctx, number=1, repeat=10)
- tcost = timer()
- prof_res = np.array(tcost.results) * 1000 # convert to millisecond
- print(
- "Mean inference time (std dev): %.2f ms (%.2f ms)"
- % (np.mean(prof_res), np.std(prof_res))
- )
-
-
- # Run the tuning and evaluate the results
- try:
- tune_and_evaluate(tuning_option)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Extract tasks...
- /workspace/python/tvm/target/target.py:329: UserWarning: target_host parameter is going to be deprecated. Please pass in tvm.target.Target(target, host=target_host) instead.
- warnings.warn(
- Extracted 10 conv2d tasks:
- (1, 56, 56, 64, 64, 3, 3, 1, 1, 1, 1)
- (1, 56, 56, 64, 128, 1, 1, 0, 0, 2, 2)
- (1, 56, 56, 64, 128, 3, 3, 1, 1, 2, 2)
- (1, 28, 28, 128, 128, 3, 3, 1, 1, 1, 1)
- (1, 28, 28, 128, 256, 1, 1, 0, 0, 2, 2)
- (1, 28, 28, 128, 256, 3, 3, 1, 1, 2, 2)
- (1, 14, 14, 256, 256, 3, 3, 1, 1, 1, 1)
- (1, 14, 14, 256, 512, 1, 1, 0, 0, 2, 2)
- (1, 14, 14, 256, 512, 3, 3, 1, 1, 2, 2)
- (1, 7, 7, 512, 512, 3, 3, 1, 1, 1, 1)
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 483-520
-
-Sample Output
--------------
-The tuning needs to compile many programs and extract feature from them.
-So a high performance CPU is recommended.
-One sample output is listed below.
-It takes about 2 hours on a 16T CPU, and 6 Pynq boards.
-
-.. code-block:: bash
-
- Extract tasks...
- [Warning] Invalid shape during AutoTVM task creation
- Extracted 10 conv2d tasks:
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (32, 16, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 16, 14, 14, 1, 16, 'int8'), (32, 16, 1, 1, 16, 16, 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 8, 28, 28, 1, 16), 'int8'), ('TENSOR', (16, 8, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 8, 28, 28, 1, 16, 'int8'), (16, 8, 1, 1, 16, 16, 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 4, 56, 56, 1, 16), 'int8'), ('TENSOR', (8, 4, 1, 1, 16, 16), 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 4, 56, 56, 1, 16, 'int8'), (8, 4, 1, 1, 16, 16, 'int8'), (2, 2), (0, 0), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 4, 56, 56, 1, 16), 'int8'), ('TENSOR', (4, 4, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 4, 56, 56, 1, 16, 'int8'), (4, 4, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 8, 28, 28, 1, 16), 'int8'), ('TENSOR', (8, 8, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 8, 28, 28, 1, 16, 'int8'), (8, 8, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 4, 56, 56, 1, 16), 'int8'), ('TENSOR', (8, 4, 3, 3, 16, 16), 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 4, 56, 56, 1, 16, 'int8'), (8, 4, 3, 3, 16, 16, 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (16, 16, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 16, 14, 14, 1, 16, 'int8'), (16, 16, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 8, 28, 28, 1, 16), 'int8'), ('TENSOR', (16, 8, 3, 3, 16, 16), 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 8, 28, 28, 1, 16, 'int8'), (16, 8, 3, 3, 16, 16, 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 32, 7, 7, 1, 16), 'int8'), ('TENSOR', (32, 32, 3, 3, 16, 16), 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 32, 7, 7, 1, 16, 'int8'), (32, 32, 3, 3, 16, 16, 'int8'), (1, 1), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Task(func_name=topi_nn_conv2d, args=(('TENSOR', (1, 16, 14, 14, 1, 16), 'int8'), ('TENSOR', (32, 16, 3, 3, 16, 16), 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'), kwargs={}, workload=('conv2d', (1, 16, 14, 14, 1, 16, 'int8'), (32, 16, 3, 3, 16, 16, 'int8'), (2, 2), (1, 1), (1, 1), 'NCHW1n16c', 'int32'))
- Tuning...
- [Task 1/10] Current/Best: 0.72/ 23.24 GFLOPS | Progress: (480/1000) | 640.31 s Done.
- [Task 2/10] Current/Best: 0.00/ 27.69 GFLOPS | Progress: (576/1000) | 810.09 s Done.
- [Task 3/10] Current/Best: 0.00/ 22.97 GFLOPS | Progress: (1000/1000) | 1125.37 s Done.
- [Task 4/10] Current/Best: 0.00/ 31.26 GFLOPS | Progress: (1000/1000) | 1025.52 s Done.
- [Task 5/10] Current/Best: 0.00/ 15.15 GFLOPS | Progress: (1000/1000) | 1236.58 s Done.
- [Task 6/10] Current/Best: 0.00/ 22.74 GFLOPS | Progress: (1000/1000) | 906.60 s Done.
- [Task 7/10] Current/Best: 0.00/ 15.27 GFLOPS | Progress: (1000/1000) | 1056.25 s Done.
- [Task 8/10] Current/Best: 0.00/ 2.18 GFLOPS | Progress: (1000/1000) | 2275.29 s Done.
- [Task 9/10] Current/Best: 2.23/ 3.99 GFLOPS | Progress: (1000/1000) | 2527.25 s Done.
- [Task 10/10] Current/Best: 1.56/ 6.32 GFLOPS | Progress: (480/1000) | 1304.84 s Done.
- Compile...
- Upload...
- Evaluate inference time cost...
- Mean inference time (std dev): 621.79 ms (0.14 ms)
-
-.. GENERATED FROM PYTHON SOURCE LINES 522-538
-
-.. note:: **Experiencing Difficulties?**
-
- The auto tuning module is error-prone. If you always see " 0.00/ 0.00 GFLOPS",
- then there must be something wrong.
-
- First, make sure you set the correct configuration of your device.
- Then, you can print debug information by adding these lines in the beginning
- of the script. It will print every measurement result, where you can find useful
- error messages.
-
- .. code-block:: python
-
- import logging
- logging.getLogger('autotvm').setLevel(logging.DEBUG)
-
- Finally, always feel free to ask our community for help on https://discuss.tvm.apache.org
-
-
-.. _sphx_glr_download_topic_vta_tutorials_autotvm_tune_relay_vta.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: tune_relay_vta.py <tune_relay_vta.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: tune_relay_vta.ipynb <tune_relay_vta.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/frontend/deploy_classification.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/deploy_classification.rst.txt
deleted file mode 100644
index 60f846fa71..0000000000
--- a/docs/_sources/topic/vta/tutorials/frontend/deploy_classification.rst.txt
+++ /dev/null
@@ -1,442 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/frontend/deploy_classification.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_frontend_deploy_classification.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/95395e118195f25266654dd8fbf487d4/deploy_classification.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_frontend_deploy_classification.py:
-
-
-Deploy Pretrained Vision Model from MxNet on VTA
-================================================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This tutorial provides an end-to-end demo, on how to run ImageNet classification
-inference onto the VTA accelerator design to perform ImageNet classification tasks.
-It showcases Relay as a front end compiler that can perform quantization (VTA
-only supports int8/32 inference) as well as graph packing (in order to enable
-tensorization in the core) to massage the compute graph for the hardware target.
-
-.. GENERATED FROM PYTHON SOURCE LINES 30-40
-
-Install dependencies
---------------------
-To use the autotvm package in tvm, we need to install some extra dependencies.
-(change "3" to "2" if you use python2):
-
-.. code-block:: bash
-
- pip3 install --user mxnet requests "Pillow<7"
-
-Now return to the python code. Import packages.
-
-.. GENERATED FROM PYTHON SOURCE LINES 40-68
-
-.. code-block:: default
-
-
- from __future__ import absolute_import, print_function
-
- import argparse, json, os, requests, sys, time
- from io import BytesIO
- from os.path import join, isfile
- import sys
- from PIL import Image
-
- from mxnet.gluon.model_zoo import vision
- import numpy as np
- from matplotlib import pyplot as plt
-
- import tvm
- from tvm import te
- from tvm import rpc, autotvm, relay
- from tvm.contrib import graph_executor, utils, download
- from tvm.contrib.debugger import debug_executor
- from tvm.relay import transform
-
- import vta
- from vta.testing import simulator
- from vta.top import graph_pack
-
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 69-72
-
-Define the platform and model targets
--------------------------------------
-Execute on CPU vs. VTA, and define the model.
-
-.. GENERATED FROM PYTHON SOURCE LINES 72-98
-
-.. code-block:: default
-
-
- # Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
- env = vta.get_env()
-
- # Set ``device=arm_cpu`` to run inference on the CPU
- # or ``device=vta`` to run inference on the FPGA.
- device = "vta"
- target = env.target if device == "vta" else env.target_vta_cpu
-
- # Dictionary lookup for when to start/end bit packing
- pack_dict = {
- "resnet18_v1": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet34_v1": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet18_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet34_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet50_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- "resnet101_v2": ["nn.max_pool2d", "nn.global_avg_pool2d"],
- }
-
- # Name of Gluon model to compile
- # The ``start_pack`` and ``stop_pack`` labels indicate where
- # to start and end the graph packing relay pass: in other words
- # where to start and finish offloading to VTA.
- model = "resnet18_v1"
- assert model in pack_dict
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 99-103
-
-Obtain an execution remote
---------------------------
-When target is 'pynq', reconfigure FPGA and runtime.
-Otherwise, if target is 'sim', execute locally.
-
-.. GENERATED FROM PYTHON SOURCE LINES 103-143
-
-.. code-block:: default
-
-
- if env.TARGET not in ["sim", "tsim", "intelfocl"]:
-
- # Get remote from tracker node if environment variable is set.
- # To set up the tracker, you'll need to follow the "Auto-tuning
- # a convolutional network for VTA" tutorial.
- tracker_host = os.environ.get("TVM_TRACKER_HOST", None)
- tracker_port = os.environ.get("TVM_TRACKER_PORT", None)
- # Otherwise if you have a device you want to program directly from
- # the host, make sure you've set the variables below to the IP of
- # your board.
- device_host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- device_port = os.environ.get("VTA_RPC_PORT", "9091")
- if not tracker_host or not tracker_port:
- remote = rpc.connect(device_host, int(device_port))
- else:
- remote = autotvm.measure.request_remote(
- env.TARGET, tracker_host, int(tracker_port), timeout=10000
- )
-
- # Reconfigure the JIT runtime and FPGA.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- reconfig_start = time.time()
- vta.reconfig_runtime(remote)
- vta.program_fpga(remote, bitstream=None)
- reconfig_time = time.time() - reconfig_start
- print("Reconfigured FPGA and RPC runtime in {0:.2f}s!".format(reconfig_time))
-
- # In simulation mode, host the RPC server locally.
- else:
- remote = rpc.LocalSession()
-
- if env.TARGET in ["intelfocl"]:
- # program intelfocl aocx
- vta.program_fpga(remote, bitstream="vta.bitstream")
-
- # Get execution context from remote
- ctx = remote.ext_dev(0) if device == "vta" else remote.cpu(0)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 144-158
-
-Build the inference graph executor
-----------------------------------
-Grab vision model from Gluon model zoo and compile with Relay.
-The compilation steps are:
-
-1. Front end translation from MxNet into Relay module.
-2. Apply 8-bit quantization: here we skip the first conv layer,
- and dense layer which will both be executed in fp32 on the CPU.
-3. Perform graph packing to alter the data layout for tensorization.
-4. Perform constant folding to reduce number of operators (e.g. eliminate batch norm multiply).
-5. Perform relay build to object file.
-6. Load the object file onto remote (FPGA device).
-7. Generate graph executor, `m`.
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 158-238
-
-.. code-block:: default
-
-
- # Load pre-configured AutoTVM schedules
- with autotvm.tophub.context(target):
-
- # Populate the shape and data type dictionary for ImageNet classifier input
- dtype_dict = {"data": "float32"}
- shape_dict = {"data": (env.BATCH, 3, 224, 224)}
-
- # Get off the shelf gluon model, and convert to relay
- try:
- gluon_model = vision.get_model(model, pretrained=True)
- except RuntimeError:
- print("Downloads from mxnet no longer supported", file=sys.stderr)
- sys.exit(0)
-
- # Measure build start time
- build_start = time.time()
-
- # Start front end compilation
- mod, params = relay.frontend.from_mxnet(gluon_model, shape_dict)
-
- # Update shape and type dictionary
- shape_dict.update({k: v.shape for k, v in params.items()})
- dtype_dict.update({k: str(v.dtype) for k, v in params.items()})
-
- if target.device_name == "vta":
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with tvm.transform.PassContext(opt_level=3):
- with relay.quantize.qconfig(global_scale=8.0, skip_conv_layers=[0]):
- mod = relay.quantize.quantize(mod, params=params)
- # Perform graph packing and constant folding for VTA target
- assert env.BLOCK_IN == env.BLOCK_OUT
- # do device annotation if target is intelfocl or sim
- relay_prog = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=pack_dict[model][0],
- stop_name=pack_dict[model][1],
- device_annot=(env.TARGET == "intelfocl"),
- )
- else:
- relay_prog = mod["main"]
-
- # Compile Relay program with AlterOpLayout disabled
- if target.device_name != "vta":
- with tvm.transform.PassContext(opt_level=3, disabled_pass={"AlterOpLayout"}):
- graph, lib, params = relay.build(
- relay_prog, target=tvm.target.Target(target, host=env.target_host), params=params
- )
- else:
- if env.TARGET == "intelfocl":
- # multiple targets to run both on cpu and vta
- target = {"cpu": env.target_vta_cpu, "ext_dev": target}
- with vta.build_config(
- opt_level=3, disabled_pass={"AlterOpLayout", "tir.CommonSubexprElimTIR"}
- ):
- graph, lib, params = relay.build(
- relay_prog, target=tvm.target.Target(target, host=env.target_host), params=params
- )
-
- # Measure Relay build time
- build_time = time.time() - build_start
- print(model + " inference graph built in {0:.2f}s!".format(build_time))
-
- # Send the inference library over to the remote RPC server
- temp = utils.tempdir()
- lib.export_library(temp.relpath("graphlib.tar"))
- remote.upload(temp.relpath("graphlib.tar"))
- lib = remote.load_module("graphlib.tar")
-
- if env.TARGET == "intelfocl":
- ctxes = [remote.ext_dev(0), remote.cpu(0)]
- m = graph_executor.create(graph, lib, ctxes)
- else:
- # Graph runtime
- m = graph_executor.create(graph, lib, ctx)
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- /workspace/python/tvm/relay/build_module.py:345: DeprecationWarning: Please use input parameter mod (tvm.IRModule) instead of deprecated parameter mod (tvm.relay.function.Function)
- warnings.warn(
- /workspace/vta/tutorials/frontend/deploy_classification.py:217: DeprecationWarning: legacy graph executor behavior of producing json / lib / params will be removed in the next release. Please see documents of tvm.contrib.graph_executor.GraphModule for the new recommended usage.
- graph, lib, params = relay.build(
- resnet18_v1 inference graph built in 38.49s!
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 239-244
-
-Perform image classification inference
---------------------------------------
-We run classification on an image sample from ImageNet
-We just need to download the categories files, `synset.txt`
-and an input test image.
-
-.. GENERATED FROM PYTHON SOURCE LINES 244-314
-
-.. code-block:: default
-
-
- # Download ImageNet categories
- categ_url = "https://github.com/uwsampl/web-data/raw/main/vta/models/"
- categ_fn = "synset.txt"
- download.download(join(categ_url, categ_fn), categ_fn)
- synset = eval(open(categ_fn).read())
-
- # Download test image
- image_url = "https://homes.cs.washington.edu/~moreau/media/vta/cat.jpg"
- image_fn = "cat.png"
- download.download(image_url, image_fn)
-
- # Prepare test image for inference
- image = Image.open(image_fn).resize((224, 224))
- plt.imshow(image)
- plt.show()
- image = np.array(image) - np.array([123.0, 117.0, 104.0])
- image /= np.array([58.395, 57.12, 57.375])
- image = image.transpose((2, 0, 1))
- image = image[np.newaxis, :]
- image = np.repeat(image, env.BATCH, axis=0)
-
- # Set the network parameters and inputs
- m.set_input(**params)
- m.set_input("data", image)
-
- # Perform inference and gather execution statistics
- # More on: :py:method:`tvm.runtime.Module.time_evaluator`
- num = 4 # number of times we run module for a single measurement
- rep = 3 # number of measurements (we derive std dev from this)
- timer = m.module.time_evaluator("run", ctx, number=num, repeat=rep)
-
- if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
- timer()
- sim_stats = simulator.stats()
- print("\nExecution statistics:")
- for k, v in sim_stats.items():
- # Since we execute the workload many times, we need to normalize stats
- # Note that there is always one warm up run
- # Therefore we divide the overall stats by (num * rep + 1)
- print("\t{:<16}: {:>16}".format(k, v // (num * rep + 1)))
- else:
- tcost = timer()
- std = np.std(tcost.results) * 1000
- mean = tcost.mean * 1000
- print("\nPerformed inference in %.2fms (std = %.2f) for %d samples" % (mean, std, env.BATCH))
- print("Average per sample inference time: %.2fms" % (mean / env.BATCH))
-
- # Get classification results
- tvm_output = m.get_output(0, tvm.nd.empty((env.BATCH, 1000), "float32", remote.cpu(0)))
- for b in range(env.BATCH):
- top_categories = np.argsort(tvm_output.numpy()[b])
- # Report top-5 classification results
- print("\n{} prediction for sample {}".format(model, b))
- print("\t#1:", synset[top_categories[-1]])
- print("\t#2:", synset[top_categories[-2]])
- print("\t#3:", synset[top_categories[-3]])
- print("\t#4:", synset[top_categories[-4]])
- print("\t#5:", synset[top_categories[-5]])
- # This just checks that one of the 5 top categories
- # is one variety of cat; this is by no means an accurate
- # assessment of how quantization affects classification
- # accuracy but is meant to catch changes to the
- # quantization pass that would accuracy in the CI.
- cat_detected = False
- for k in top_categories[-5:]:
- if "cat" in synset[k]:
- cat_detected = True
- assert cat_detected
-
-
-
-.. image-sg:: /topic/vta/tutorials/frontend/images/sphx_glr_deploy_classification_001.png
- :alt: deploy classification
- :srcset: /topic/vta/tutorials/frontend/images/sphx_glr_deploy_classification_001.png
- :class: sphx-glr-single-img
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
-
- Execution statistics:
- inp_load_nbytes : 5549568
- wgt_load_nbytes : 12763136
- acc_load_nbytes : 6051840
- uop_load_nbytes : 22872
- out_store_nbytes: 2433536
- gemm_counter : 6623232
- alu_counter : 702464
-
- resnet18_v1 prediction for sample 0
- #1: tiger cat
- #2: tabby, tabby cat
- #3: Egyptian cat
- #4: lynx, catamount
- #5: Persian cat
-
-
-
-
-
-.. _sphx_glr_download_topic_vta_tutorials_frontend_deploy_classification.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: deploy_classification.py <deploy_classification.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: deploy_classification.ipynb <deploy_classification.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
index 5b17710c7c..9a7836c18e 100644
--- a/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
@@ -337,7 +337,7 @@ The compilation steps are:
/workspace/python/tvm/relay/build_module.py:345: DeprecationWarning: Please use input parameter mod (tvm.IRModule) instead of deprecated parameter mod (tvm.relay.function.Function)
warnings.warn(
- yolov3-tiny inference graph built in 26.45s!
+ yolov3-tiny inference graph built in 25.33s!
@@ -445,11 +445,6 @@ Download test image
-.. rst-class:: sphx-glr-timing
-
- **Total running time of the script:** ( 1 minutes 0.508 seconds)
-
-
.. _sphx_glr_download_topic_vta_tutorials_frontend_deploy_detection.py:
.. only:: html
diff --git a/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt
index 17ea5a4591..a532819d8a 100644
--- a/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt
@@ -14,23 +14,6 @@ Compile Deep Learning Models
<div class="sphx-glr-thumbnails">
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an end-to-end demo, on how to run ImageNet classification inference onto...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/frontend/images/thumb/sphx_glr_deploy_classification_thumb.png
- :alt: Deploy Pretrained Vision Model from MxNet on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_classification.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Deploy Pretrained Vision Model from MxNet on VTA</div>
- </div>
-
-
.. raw:: html
<div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny inference onto the...">
@@ -56,6 +39,5 @@ Compile Deep Learning Models
.. toctree::
:hidden:
- /topic/vta/tutorials/frontend/deploy_classification
/topic/vta/tutorials/frontend/deploy_detection
diff --git a/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
index 0947477ba4..25465c267d 100644
--- a/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
@@ -5,10 +5,8 @@
Computation times
=================
-**02:00.046** total execution time for **topic_vta_tutorials_frontend** files:
+**00:57.919** total execution time for **topic_vta_tutorials_frontend** files:
-+------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py` (``deploy_detection.py``) | 01:00.508 | 0.0 MB |
-+------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_classification.py` (``deploy_classification.py``) | 00:59.538 | 0.0 MB |
-+------------------------------------------------------------------------------------------------------+-----------+--------+
++--------------------------------------------------------------------------------------------+-----------+--------+
+| :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py` (``deploy_detection.py``) | 00:57.919 | 0.0 MB |
++--------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/index.rst.txt b/docs/_sources/topic/vta/tutorials/index.rst.txt
index 225609a2e4..a8faf933fa 100644
--- a/docs/_sources/topic/vta/tutorials/index.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/index.rst.txt
@@ -69,23 +69,6 @@ Compile Deep Learning Models
<div class="sphx-glr-thumbnails">
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an end-to-end demo, on how to run ImageNet classification inference onto...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/frontend/images/thumb/sphx_glr_deploy_classification_thumb.png
- :alt: Deploy Pretrained Vision Model from MxNet on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_classification.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Deploy Pretrained Vision Model from MxNet on VTA</div>
- </div>
-
-
.. raw:: html
<div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny inference onto the...">
@@ -151,54 +134,6 @@ Optimize Tensor Operators
</div>
-.. raw:: html
-
- </div>
-
-Auto tuning
--------------
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="Auto-tuning a ALU fused op on VTA">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/autotvm/images/thumb/sphx_glr_tune_alu_vta_thumb.png
- :alt: Auto-tuning a ALU fused op on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_autotvm_tune_alu_vta.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Auto-tuning a ALU fused op on VTA</div>
- </div>
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="Auto-tuning for a specific accelerator design is critical for getting the best performance for ...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/autotvm/images/thumb/sphx_glr_tune_relay_vta_thumb.png
- :alt: Auto-tuning a convolutional network on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_autotvm_tune_relay_vta.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Auto-tuning a convolutional network on VTA</div>
- </div>
-
-
.. raw:: html
</div>
@@ -210,7 +145,6 @@ Auto tuning
/topic/vta/tutorials/frontend/index.rst
/topic/vta/tutorials/optimize/index.rst
- /topic/vta/tutorials/autotvm/index.rst
diff --git a/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
index 8e8c2ab08f..ad060973c4 100644
--- a/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:03.365** total execution time for **topic_vta_tutorials_optimize** files:
+**00:03.192** total execution time for **topic_vta_tutorials_optimize** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py` (``convolution_opt.py``) | 00:02.848 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py` (``convolution_opt.py``) | 00:02.702 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py` (``matrix_multiply_opt.py``) | 00:00.517 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py` (``matrix_multiply_opt.py``) | 00:00.490 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
index de5e2ddd08..fdd510057e 100644
--- a/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:00.843** total execution time for **topic_vta_tutorials** files:
+**00:00.817** total execution time for **topic_vta_tutorials** files:
+---------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_matrix_multiply.py` (``matrix_multiply.py``) | 00:00.430 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_matrix_multiply.py` (``matrix_multiply.py``) | 00:00.424 | 0.0 MB |
+---------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_vta_get_started.py` (``vta_get_started.py``) | 00:00.412 | 0.0 MB |
+| :ref:`sphx_glr_topic_vta_tutorials_vta_get_started.py` (``vta_get_started.py``) | 00:00.393 | 0.0 MB |
+---------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt b/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
index 96891a6076..c2994268f1 100644
--- a/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
+++ b/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
@@ -318,7 +318,7 @@ We build the binary and check its correctness and performance.
.. code-block:: none
- Execution time of this operator: 93.179 ms
+ Execution time of this operator: 90.564 ms
@@ -434,7 +434,7 @@ operations.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 21.479 seconds)
+ **Total running time of the script:** ( 1 minutes 15.527 seconds)
.. _sphx_glr_download_tutorial_auto_scheduler_matmul_x86.py:
diff --git a/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt b/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
index 40da7cb90b..c135b92b76 100644
--- a/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
+++ b/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
@@ -454,16 +454,16 @@ reduce variance, we take 5 measurements and average them.
waiting for device...
device available
Get devices for measurement successfully!
- No: 1 GFLOPS: 1.02/1.02 result: MeasureResult(costs=(0.26421220180000005,), error_no=MeasureErrorNo.NO_ERROR, all_cost=4.487396955490112, timestamp=1707954085.0204551) [('tile_y', [-1, 256]), ('tile_x', [-1, 2])],None,18
- No: 2 GFLOPS: 11.93/11.93 result: MeasureResult(costs=(0.0224943478,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6642606258392334, timestamp=1707954085.6474626) [('tile_y', [-1, 16]), ('tile_x', [-1, 512])],None,94
- No: 3 GFLOPS: 9.10/11.93 result: MeasureResult(costs=(0.0295089428,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7391059398651123, timestamp=1707954086.3778603) [('tile_y', [-1, 2]), ('tile_x', [-1, 64])],None,61
- No: 4 GFLOPS: 11.36/11.93 result: MeasureResult(costs=(0.023639946000000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6362216472625732, timestamp=1707954087.020166) [('tile_y', [-1, 128]), ('tile_x', [-1, 512])],None,97
- No: 5 GFLOPS: 13.89/13.89 result: MeasureResult(costs=(0.019326832,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.5792238712310791, timestamp=1707954087.8077765) [('tile_y', [-1, 16]), ('tile_x', [-1, 32])],None,54
- No: 6 GFLOPS: 9.89/13.89 result: MeasureResult(costs=(0.0271417682,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7133009433746338, timestamp=1707954088.5055072) [('tile_y', [-1, 4]), ('tile_x', [-1, 32])],None,52
- No: 7 GFLOPS: 10.31/13.89 result: MeasureResult(costs=(0.026033086200000007,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7047078609466553, timestamp=1707954089.1907346) [('tile_y', [-1, 32]), ('tile_x', [-1, 256])],None,85
- No: 8 GFLOPS: 9.29/13.89 result: MeasureResult(costs=(0.0289075382,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.8394291400909424, timestamp=1707954089.9252644) [('tile_y', [-1, 8]), ('tile_x', [-1, 128])],None,73
- No: 9 GFLOPS: 3.65/13.89 result: MeasureResult(costs=(0.07356237260000001,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.4034011363983154, timestamp=1707954091.4507942) [('tile_y', [-1, 128]), ('tile_x', [-1, 8])],None,37
- No: 10 GFLOPS: 11.41/13.89 result: MeasureResult(costs=(0.0235165548,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6241216659545898, timestamp=1707954092.0924337) [('tile_y', [-1, 128]), ('tile_x', [-1, 32])],None,57
+ No: 1 GFLOPS: 11.83/11.83 result: MeasureResult(costs=(0.022691141800000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6199727058410645, timestamp=1708011917.8810415) [('tile_y', [-1, 256]), ('tile_x', [-1, 512])],None,98
+ No: 2 GFLOPS: 10.03/11.83 result: MeasureResult(costs=(0.026763245799999995,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6931114196777344, timestamp=1708011918.5738919) [('tile_y', [-1, 4]), ('tile_x', [-1, 4])],None,22
+ No: 3 GFLOPS: 10.16/11.83 result: MeasureResult(costs=(0.026412748599999997,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7376604080200195, timestamp=1708011919.2599878) [('tile_y', [-1, 512]), ('tile_x', [-1, 128])],None,79
+ No: 4 GFLOPS: 8.76/11.83 result: MeasureResult(costs=(0.0306544982,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7452092170715332, timestamp=1708011920.0208437) [('tile_y', [-1, 1]), ('tile_x', [-1, 8])],None,30
+ No: 5 GFLOPS: 1.40/11.83 result: MeasureResult(costs=(0.1923943968,), error_no=MeasureErrorNo.NO_ERROR, all_cost=3.3096492290496826, timestamp=1708011923.5289671) [('tile_y', [-1, 2]), ('tile_x', [-1, 1])],None,1
+ No: 6 GFLOPS: 9.17/11.83 result: MeasureResult(costs=(0.029269668999999998,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.710364818572998, timestamp=1708011924.2544389) [('tile_y', [-1, 1]), ('tile_x', [-1, 32])],None,50
+ No: 7 GFLOPS: 14.41/14.41 result: MeasureResult(costs=(0.018628427,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.5510485172271729, timestamp=1708011924.8063998) [('tile_y', [-1, 16]), ('tile_x', [-1, 32])],None,54
+ No: 8 GFLOPS: 12.25/14.41 result: MeasureResult(costs=(0.0219187654,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.665583610534668, timestamp=1708011925.411672) [('tile_y', [-1, 64]), ('tile_x', [-1, 32])],None,56
+ No: 9 GFLOPS: 9.09/14.41 result: MeasureResult(costs=(0.029538170000000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6980679035186768, timestamp=1708011926.2184541) [('tile_y', [-1, 2]), ('tile_x', [-1, 16])],None,41
+ No: 10 GFLOPS: 1.06/14.41 result: MeasureResult(costs=(0.25395357939999996,), error_no=MeasureErrorNo.NO_ERROR, all_cost=4.2798850536346436, timestamp=1708011930.5383048) [('tile_y', [-1, 256]), ('tile_x', [-1, 2])],None,18
diff --git a/docs/_sources/tutorial/autotvm_relay_x86.rst.txt b/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
index dece0c9333..730adf9b76 100644
--- a/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
+++ b/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
@@ -311,7 +311,7 @@ standard deviation.
.. code-block:: none
- {'mean': 499.7387120300482, 'median': 499.10965995004517, 'std': 3.497109291631726}
+ {'mean': 467.02020688007906, 'median': 466.2648031999197, 'std': 1.6591059255929859}
@@ -582,30 +582,30 @@ the tuning data to.
.. code-block:: none
- [Task 1/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 1/25] Current/Best: 12.70/ 22.06 GFLOPS | Progress: (4/20) | 9.27 s [Task 1/25] Current/Best: 11.06/ 22.36 GFLOPS | Progress: (8/20) | 12.89 s [Task 1/25] Current/Best: 11.30/ 22.36 GFLOPS | Progress: (12/20) | 19.19 s [Task 1/25] Current/Best: 15.41/ 23.88 GFLOPS | Progress: (16/20) | 21.18 s [Task 1/25] Current/Best: 5.24/ 23.88 GFLOPS | Progress: (20/20) | 24.82 s Done.
- [Task 2/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 2/25] Current/Best: 16.72/ 16.72 GFLOPS | Progress: (4/20) | 4.81 s [Task 2/25] Current/Best: 15.16/ 16.72 GFLOPS | Progress: (8/20) | 7.44 s [Task 2/25] Current/Best: 5.85/ 17.51 GFLOPS | Progress: (12/20) | 9.08 s [Task 2/25] Current/Best: 14.97/ 21.35 GFLOPS | Progress: (16/20) | 10.82 s [Task 2/25] Current/Best: 15.53/ 21.35 GFLOPS | Progress: (20/20) | 12.57 s Done.
- [Task 3/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 3/25] Current/Best: 11.05/ 19.26 GFLOPS | Progress: (4/20) | 4.72 s [Task 3/25] Current/Best: 24.06/ 24.06 GFLOPS | Progress: (8/20) | 6.78 s [Task 3/25] Current/Best: 20.63/ 24.06 GFLOPS | Progress: (12/20) | 9.14 s [Task 3/25] Current/Best: 6.53/ 24.06 GFLOPS | Progress: (16/20) | 11.56 s [Task 3/25] Current/Best: 16.12/ 24.06 GFLOPS | Progress: (20/20) | 13.85 s Done.
- [Task 4/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 4/25] Current/Best: 11.35/ 15.15 GFLOPS | Progress: (4/20) | 5.63 s [Task 4/25] Current/Best: 9.98/ 15.15 GFLOPS | Progress: (8/20) | 8.18 s [Task 4/25] Current/Best: 17.20/ 17.20 GFLOPS | Progress: (12/20) | 14.63 s [Task 4/25] Current/Best: 5.89/ 17.20 GFLOPS | Progress: (16/20) | 21.01 s [Task 4/25] Current/Best: 4.96/ 17.20 GFLOPS | Progress: (20/20) | 26.29 s Done.
- [Task 5/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 5/25] Current/Best: 10.41/ 18.54 GFLOPS | Progress: (4/20) | 4.57 s [Task 5/25] Current/Best: 17.88/ 18.54 GFLOPS | Progress: (8/20) | 6.87 s [Task 5/25] Current/Best: 14.44/ 20.65 GFLOPS | Progress: (12/20) | 8.84 s [Task 5/25] Current/Best: 6.94/ 21.41 GFLOPS | Progress: (16/20) | 11.11 s [Task 5/25] Current/Best: 16.65/ 21.41 GFLOPS | Progress: (20/20) | 13.18 s Done.
- [Task 6/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 6/25] Current/Best: 15.26/ 15.26 GFLOPS | Progress: (4/20) | 5.50 s [Task 6/25] Current/Best: 5.87/ 15.26 GFLOPS | Progress: (8/20) | 8.16 s [Task 6/25] Current/Best: 11.70/ 18.43 GFLOPS | Progress: (12/20) | 11.40 s [Task 6/25] Current/Best: 18.92/ 18.92 GFLOPS | Progress: (16/20) | 16.04 s [Task 6/25] Current/Best: 12.38/ 18.92 GFLOPS | Progress: (20/20) | 18.49 s Done.
- [Task 7/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 7/25] Current/Best: 16.39/ 21.61 GFLOPS | Progress: (4/20) | 4.65 s [Task 7/25] Current/Best: 6.29/ 21.61 GFLOPS | Progress: (8/20) | 7.21 s [Task 7/25] Current/Best: 7.80/ 21.61 GFLOPS | Progress: (12/20) | 9.93 s [Task 7/25] Current/Best: 21.06/ 21.61 GFLOPS | Progress: (16/20) | 12.33 s [Task 7/25] Current/Best: 15.42/ 21.61 GFLOPS | Progress: (20/20) | 14.86 s Done.
- [Task 8/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 8/25] Current/Best: 8.01/ 13.61 GFLOPS | Progress: (4/20) | 5.42 s [Task 8/25] Current/Best: 7.88/ 13.61 GFLOPS | Progress: (8/20) | 8.42 s [Task 8/25] Current/Best: 16.02/ 18.34 GFLOPS | Progress: (12/20) | 11.52 s [Task 8/25] Current/Best: 15.17/ 18.34 GFLOPS | Progress: (16/20) | 18.25 s [Task 8/25] Current/Best: 12.82/ 18.34 GFLOPS | Progress: (20/20) | 20.99 s Done.
- [Task 9/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 9/25] Current/Best: 7.05/ 18.37 GFLOPS | Progress: (4/20) | 11.73 s [Task 9/25] Current/Best: 15.47/ 20.90 GFLOPS | Progress: (8/20) | 14.67 s [Task 9/25] Current/Best: 13.85/ 21.17 GFLOPS | Progress: (12/20) | 18.05 s [Task 9/25] Current/Best: 6.55/ 21.17 GFLOPS | Progress: (16/20) | 20.12 s [Task 9/25] Current/Best: 16.44/ 21.17 GFLOPS | Progress: (20/20) | 23.12 s Done.
- [Task 10/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 10/25] Current/Best: 5.22/ 21.25 GFLOPS | Progress: (4/20) | 4.44 s [Task 10/25] Current/Best: 14.01/ 21.25 GFLOPS | Progress: (8/20) | 6.46 s [Task 10/25] Current/Best: 14.19/ 21.25 GFLOPS | Progress: (12/20) | 8.63 s [Task 10/25] Current/Best: 16.03/ 21.25 GFLOPS | Progress: (16/20) | 10.43 s [Task 10/25] Current/Best: 13.65/ 21.25 GFLOPS | Progress: (20/20) | 12.46 s Done.
- [Task 11/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 11/25] Current/Best: 23.86/ 23.86 GFLOPS | Progress: (4/20) | 5.38 s [Task 11/25] Current/Best: 12.62/ 23.86 GFLOPS | Progress: (8/20) | 7.83 s [Task 11/25] Current/Best: 19.15/ 23.86 GFLOPS | Progress: (12/20) | 9.84 s [Task 11/25] Current/Best: 13.02/ 23.86 GFLOPS | Progress: (16/20) | 13.98 s [Task 11/25] Current/Best: 19.21/ 23.86 GFLOPS | Progress: (20/20) | 16.53 s Done.
- [Task 12/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 12/25] Current/Best: 4.01/ 16.88 GFLOPS | Progress: (4/20) | 5.28 s [Task 12/25] Current/Best: 19.34/ 19.34 GFLOPS | Progress: (8/20) | 8.03 s [Task 12/25] Current/Best: 12.48/ 19.34 GFLOPS | Progress: (12/20) | 10.76 s [Task 12/25] Current/Best: 4.55/ 19.34 GFLOPS | Progress: (16/20) | 16.33 s [Task 12/25] Current/Best: 8.84/ 19.34 GFLOPS | Progress: (20/20) | 20.68 s Done.
- [Task 13/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 13/25] Current/Best: 2.73/ 18.69 GFLOPS | Progress: (4/20) | 6.28 s [Task 13/25] Current/Best: 17.28/ 20.40 GFLOPS | Progress: (8/20) | 8.40 s [Task 13/25] Current/Best: 15.53/ 22.62 GFLOPS | Progress: (12/20) | 10.87 s [Task 13/25] Current/Best: 5.37/ 22.62 GFLOPS | Progress: (16/20) | 13.81 s [Task 13/25] Current/Best: 7.33/ 22.62 GFLOPS | Progress: (20/20) | 16.36 s Done.
- [Task 14/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 14/25] Current/Best: 9.80/ 13.93 GFLOPS | Progress: (4/20) | 7.46 s [Task 14/25] Current/Best: 16.21/ 16.21 GFLOPS | Progress: (8/20) | 11.04 s [Task 14/25] Current/Best: 3.02/ 17.75 GFLOPS | Progress: (12/20) | 16.37 s [Task 14/25] Current/Best: 15.99/ 17.75 GFLOPS | Progress: (16/20) | 27.69 s [Task 14/25] Current/Best: 22.09/ 22.09 GFLOPS | Progress: (20/20) | 30.90 s [Task 15/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 15/25] Current/Best: 17.73/ 18.11 GFLOPS | Progress: (4/20) | 4.21 s [Task 15/25] Current/Best: 6.90/ 18.11 GFLOPS | Progress: (8/20) | 6.30 s [Task 15/25] Current/Best: 11.27/ 18.11 GFLOPS | Progress: (12/20) | 16.80 s [Task 15/25] Current/Best: 8.64/ 19.74 GFLOPS | Progress: (16/20) | 18.49 s [Task 15/25] Current/Best: 6.57/ 19.74 GFLOPS | Progress: (20/2
0) | 22.46 s Done.
- [Task 16/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 16/25] Current/Best: 15.76/ 15.76 GFLOPS | Progress: (4/20) | 6.08 s [Task 16/25] Current/Best: 17.35/ 18.67 GFLOPS | Progress: (8/20) | 7.83 s [Task 16/25] Current/Best: 19.65/ 19.65 GFLOPS | Progress: (12/20) | 9.57 s [Task 16/25] Current/Best: 14.42/ 19.65 GFLOPS | Progress: (16/20) | 11.33 s [Task 16/25] Current/Best: 15.76/ 19.65 GFLOPS | Progress: (20/20) | 13.70 s Done.
- [Task 17/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 17/25] Current/Best: 9.94/ 19.80 GFLOPS | Progress: (4/20) | 5.44 s [Task 17/25] Current/Best: 22.37/ 22.37 GFLOPS | Progress: (8/20) | 8.77 s [Task 17/25] Current/Best: 6.15/ 22.37 GFLOPS | Progress: (12/20) | 11.92 s [Task 17/25] Current/Best: 9.69/ 22.37 GFLOPS | Progress: (16/20) | 14.33 s [Task 17/25] Current/Best: 6.18/ 22.37 GFLOPS | Progress: (20/20) | 18.66 s Done.
- [Task 18/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 18/25] Current/Best: 12.87/ 12.87 GFLOPS | Progress: (4/20) | 5.86 s [Task 18/25] Current/Best: 11.61/ 14.48 GFLOPS | Progress: (8/20) | 8.74 s [Task 18/25] Current/Best: 17.27/ 17.27 GFLOPS | Progress: (12/20) | 10.85 s [Task 18/25] Current/Best: 6.16/ 17.57 GFLOPS | Progress: (16/20) | 14.08 s [Task 18/25] Current/Best: 17.10/ 17.57 GFLOPS | Progress: (20/20) | 16.14 s Done.
- [Task 19/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 19/25] Current/Best: 15.44/ 15.44 GFLOPS | Progress: (4/20) | 6.95 s [Task 19/25] Current/Best: 18.01/ 18.10 GFLOPS | Progress: (8/20) | 12.91 s [Task 19/25] Current/Best: 20.75/ 22.02 GFLOPS | Progress: (12/20) | 15.18 s [Task 19/25] Current/Best: 9.90/ 22.02 GFLOPS | Progress: (16/20) | 19.15 s [Task 19/25] Current/Best: 10.74/ 22.02 GFLOPS | Progress: (20/20) | 22.42 s Done.
- [Task 20/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 20/25] Current/Best: 9.21/ 22.49 GFLOPS | Progress: (4/20) | 7.46 s [Task 20/25] Current/Best: 17.58/ 22.49 GFLOPS | Progress: (8/20) | 17.16 s [Task 20/25] Current/Best: 12.28/ 22.49 GFLOPS | Progress: (12/20) | 24.93 s [Task 20/25] Current/Best: 5.42/ 22.49 GFLOPS | Progress: (16/20) | 34.54 s [Task 20/25] Current/Best: 12.47/ 22.49 GFLOPS | Progress: (20/20) | 46.03 s [Task 21/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 21/25] Current/Best: 10.71/ 16.02 GFLOPS | Progress: (4/20) | 5.76 s [Task 21/25] Current/Best: 6.86/ 16.02 GFLOPS | Progress: (8/20) | 11.19 s [Task 21/25] Current/Best: 17.92/ 17.92 GFLOPS | Progress: (12/20) | 12.99 s [Task 21/25] Current/Best: 12.05/ 18.37 GFLOPS | Progress: (16/20) | 19.31 s [Task 21/25] Current/Best: 3.16/ 18.37 GFLOPS | Progress: (20/
20) | 23.09 s Done.
- [Task 22/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 22/25] Current/Best: 10.73/ 13.63 GFLOPS | Progress: (4/20) | 4.67 s Done.
+ [Task 1/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 1/25] Current/Best: 14.48/ 22.81 GFLOPS | Progress: (4/20) | 7.93 s [Task 1/25] Current/Best: 11.61/ 22.81 GFLOPS | Progress: (8/20) | 10.29 s [Task 1/25] Current/Best: 16.27/ 22.81 GFLOPS | Progress: (12/20) | 12.77 s [Task 1/25] Current/Best: 7.66/ 22.81 GFLOPS | Progress: (16/20) | 15.88 s [Task 1/25] Current/Best: 23.69/ 23.69 GFLOPS | Progress: (20/20) | 18.97 s Done.
+ [Task 2/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 2/25] Current/Best: 22.43/ 22.43 GFLOPS | Progress: (4/20) | 4.09 s [Task 2/25] Current/Best: 18.32/ 22.43 GFLOPS | Progress: (8/20) | 5.87 s [Task 2/25] Current/Best: 10.74/ 22.65 GFLOPS | Progress: (12/20) | 8.25 s [Task 2/25] Current/Best: 20.38/ 22.65 GFLOPS | Progress: (16/20) | 9.66 s [Task 2/25] Current/Best: 6.73/ 22.65 GFLOPS | Progress: (20/20) | 11.02 s Done.
+ [Task 3/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 3/25] Current/Best: 17.26/ 20.65 GFLOPS | Progress: (4/20) | 4.78 s [Task 3/25] Current/Best: 24.20/ 24.20 GFLOPS | Progress: (8/20) | 7.10 s [Task 3/25] Current/Best: 20.21/ 24.20 GFLOPS | Progress: (12/20) | 9.07 s [Task 3/25] Current/Best: 6.24/ 24.20 GFLOPS | Progress: (16/20) | 11.79 s [Task 3/25] Current/Best: 17.33/ 24.20 GFLOPS | Progress: (20/20) | 14.14 s Done.
+ [Task 4/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 4/25] Current/Best: 6.66/ 14.04 GFLOPS | Progress: (4/20) | 4.87 s [Task 4/25] Current/Best: 13.58/ 16.75 GFLOPS | Progress: (8/20) | 7.43 s [Task 4/25] Current/Best: 13.70/ 17.42 GFLOPS | Progress: (12/20) | 9.75 s [Task 4/25] Current/Best: 5.45/ 17.42 GFLOPS | Progress: (16/20) | 12.68 s [Task 4/25] Current/Best: 21.45/ 21.45 GFLOPS | Progress: (20/20) | 14.86 s Done.
+ [Task 5/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 5/25] Current/Best: 5.05/ 16.37 GFLOPS | Progress: (4/20) | 4.60 s [Task 5/25] Current/Best: 18.66/ 18.66 GFLOPS | Progress: (8/20) | 6.67 s [Task 5/25] Current/Best: 3.12/ 18.66 GFLOPS | Progress: (12/20) | 9.34 s [Task 5/25] Current/Best: 4.43/ 20.34 GFLOPS | Progress: (16/20) | 11.24 s [Task 5/25] Current/Best: 5.85/ 20.34 GFLOPS | Progress: (20/20) | 13.30 s Done.
+ [Task 6/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 6/25] Current/Best: 12.96/ 18.00 GFLOPS | Progress: (4/20) | 4.38 s [Task 6/25] Current/Best: 15.53/ 18.23 GFLOPS | Progress: (8/20) | 6.74 s [Task 6/25] Current/Best: 10.12/ 18.92 GFLOPS | Progress: (12/20) | 9.92 s [Task 6/25] Current/Best: 10.62/ 18.92 GFLOPS | Progress: (16/20) | 13.43 s [Task 6/25] Current/Best: 15.26/ 18.92 GFLOPS | Progress: (20/20) | 16.18 s Done.
+ [Task 7/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 7/25] Current/Best: 10.08/ 18.90 GFLOPS | Progress: (4/20) | 5.70 s [Task 7/25] Current/Best: 17.90/ 18.90 GFLOPS | Progress: (8/20) | 8.12 s [Task 7/25] Current/Best: 10.48/ 18.90 GFLOPS | Progress: (12/20) | 10.61 s [Task 7/25] Current/Best: 15.08/ 18.90 GFLOPS | Progress: (16/20) | 14.33 s [Task 7/25] Current/Best: 20.70/ 20.70 GFLOPS | Progress: (20/20) | 16.83 s Done.
+ [Task 8/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 8/25] Current/Best: 15.08/ 15.08 GFLOPS | Progress: (4/20) | 4.76 s [Task 8/25] Current/Best: 11.55/ 15.08 GFLOPS | Progress: (8/20) | 8.70 s [Task 8/25] Current/Best: 13.43/ 16.64 GFLOPS | Progress: (12/20) | 11.64 s [Task 8/25] Current/Best: 12.91/ 16.64 GFLOPS | Progress: (16/20) | 18.36 s [Task 8/25] Current/Best: 15.15/ 16.64 GFLOPS | Progress: (20/20) | 20.98 s Done.
+ [Task 9/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 9/25] Current/Best: 17.34/ 17.34 GFLOPS | Progress: (4/20) | 8.46 s [Task 9/25] Current/Best: 14.24/ 17.45 GFLOPS | Progress: (8/20) | 10.86 s [Task 9/25] Current/Best: 8.95/ 18.96 GFLOPS | Progress: (12/20) | 13.61 s [Task 9/25] Current/Best: 19.76/ 19.76 GFLOPS | Progress: (16/20) | 24.66 s [Task 9/25] Current/Best: 6.61/ 19.76 GFLOPS | Progress: (20/20) | 27.69 s [Task 10/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 10/25] Current/Best: 13.05/ 15.74 GFLOPS | Progress: (4/20) | 4.58 s [Task 10/25] Current/Best: 18.46/ 18.46 GFLOPS | Progress: (8/20) | 7.05 s [Task 10/25] Current/Best: 18.56/ 18.56 GFLOPS | Progress: (12/20) | 8.90 s [Task 10/25] Current/Best: 16.70/ 18.56 GFLOPS | Progress: (16/20) | 11.02 s [Task 10/25] Current/Best: 16.46/ 18.56 GFLOPS | Progress: (20/20
) | 14.82 s Done.
+ [Task 11/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 11/25] Current/Best: 3.23/ 24.71 GFLOPS | Progress: (4/20) | 5.15 s [Task 11/25] Current/Best: 6.30/ 24.99 GFLOPS | Progress: (8/20) | 7.19 s [Task 11/25] Current/Best: 15.60/ 24.99 GFLOPS | Progress: (12/20) | 9.15 s [Task 11/25] Current/Best: 13.20/ 24.99 GFLOPS | Progress: (16/20) | 11.89 s [Task 11/25] Current/Best: 24.48/ 24.99 GFLOPS | Progress: (20/20) | 13.88 s Done.
+ [Task 12/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 12/25] Current/Best: 3.73/ 18.70 GFLOPS | Progress: (4/20) | 4.84 s [Task 12/25] Current/Best: 16.81/ 18.70 GFLOPS | Progress: (8/20) | 10.24 s [Task 12/25] Current/Best: 4.30/ 18.70 GFLOPS | Progress: (12/20) | 14.55 s [Task 12/25] Current/Best: 12.84/ 18.70 GFLOPS | Progress: (16/20) | 20.30 s [Task 12/25] Current/Best: 17.59/ 18.70 GFLOPS | Progress: (20/20) | 23.77 s Done.
+ [Task 13/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 13/25] Current/Best: 16.44/ 20.41 GFLOPS | Progress: (4/20) | 4.64 s [Task 13/25] Current/Best: 7.91/ 20.41 GFLOPS | Progress: (8/20) | 9.33 s [Task 13/25] Current/Best: 13.32/ 20.41 GFLOPS | Progress: (12/20) | 13.13 s [Task 13/25] Current/Best: 8.34/ 22.24 GFLOPS | Progress: (16/20) | 16.38 s [Task 13/25] Current/Best: 16.18/ 22.24 GFLOPS | Progress: (20/20) | 18.60 s Done.
+ [Task 14/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 14/25] Current/Best: 13.96/ 20.30 GFLOPS | Progress: (4/20) | 9.32 s [Task 14/25] Current/Best: 5.93/ 20.30 GFLOPS | Progress: (8/20) | 15.78 s [Task 14/25] Current/Best: 10.14/ 20.30 GFLOPS | Progress: (12/20) | 24.23 s [Task 14/25] Current/Best: 10.11/ 20.30 GFLOPS | Progress: (16/20) | 30.68 s [Task 14/25] Current/Best: 18.80/ 20.30 GFLOPS | Progress: (20/20) | 32.90 s Done.
+ [Task 15/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 15/25] Current/Best: 9.21/ 17.71 GFLOPS | Progress: (4/20) | 6.11 s [Task 15/25] Current/Best: 12.66/ 19.27 GFLOPS | Progress: (8/20) | 10.15 s [Task 15/25] Current/Best: 12.53/ 19.59 GFLOPS | Progress: (12/20) | 13.23 s [Task 15/25] Current/Best: 7.34/ 22.61 GFLOPS | Progress: (16/20) | 14.81 s [Task 15/25] Current/Best: 17.36/ 22.61 GFLOPS | Progress: (20/20) | 16.74 s Done.
+ [Task 16/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 16/25] Current/Best: 11.05/ 20.11 GFLOPS | Progress: (4/20) | 5.34 s [Task 16/25] Current/Best: 10.93/ 20.11 GFLOPS | Progress: (8/20) | 7.51 s [Task 16/25] Current/Best: 11.42/ 20.11 GFLOPS | Progress: (12/20) | 10.65 s [Task 16/25] Current/Best: 12.34/ 20.11 GFLOPS | Progress: (16/20) | 12.92 s [Task 16/25] Current/Best: 21.03/ 21.03 GFLOPS | Progress: (20/20) | 14.48 s Done.
+ [Task 17/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 17/25] Current/Best: 20.70/ 20.70 GFLOPS | Progress: (4/20) | 4.53 s [Task 17/25] Current/Best: 3.19/ 20.70 GFLOPS | Progress: (8/20) | 7.91 s [Task 17/25] Current/Best: 12.70/ 20.70 GFLOPS | Progress: (12/20) | 10.53 s [Task 17/25] Current/Best: 20.69/ 21.23 GFLOPS | Progress: (16/20) | 14.12 s [Task 17/25] Current/Best: 21.72/ 21.72 GFLOPS | Progress: (20/20) | 16.36 s Done.
+ [Task 18/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 18/25] Current/Best: 14.43/ 15.41 GFLOPS | Progress: (4/20) | 5.08 s [Task 18/25] Current/Best: 13.19/ 17.61 GFLOPS | Progress: (8/20) | 7.55 s [Task 18/25] Current/Best: 17.91/ 18.34 GFLOPS | Progress: (12/20) | 11.29 s [Task 18/25] Current/Best: 21.16/ 21.16 GFLOPS | Progress: (16/20) | 14.75 s [Task 18/25] Current/Best: 5.68/ 21.16 GFLOPS | Progress: (20/20) | 16.88 s Done.
+ [Task 19/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 19/25] Current/Best: 9.18/ 18.42 GFLOPS | Progress: (4/20) | 6.49 s [Task 19/25] Current/Best: 18.55/ 18.75 GFLOPS | Progress: (8/20) | 9.94 s [Task 19/25] Current/Best: 10.68/ 19.33 GFLOPS | Progress: (12/20) | 13.63 s [Task 19/25] Current/Best: 10.93/ 21.17 GFLOPS | Progress: (16/20) | 17.39 s [Task 19/25] Current/Best: 6.37/ 21.17 GFLOPS | Progress: (20/20) | 20.79 s Done.
+ [Task 20/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 20/25] Current/Best: 16.42/ 16.42 GFLOPS | Progress: (4/20) | 14.19 s [Task 20/25] Current/Best: 18.12/ 18.12 GFLOPS | Progress: (8/20) | 18.40 s [Task 20/25] Current/Best: 18.82/ 19.52 GFLOPS | Progress: (12/20) | 20.40 s [Task 20/25] Current/Best: 18.82/ 19.52 GFLOPS | Progress: (16/20) | 29.10 s [Task 20/25] Current/Best: 10.78/ 19.52 GFLOPS | Progress: (20/20) | 31.96 s [Task 21/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s Done.
Done.
- [Task 22/25] Current/Best: 21.55/ 21.55 GFLOPS | Progress: (8/20) | 8.33 s [Task 22/25] Current/Best: 1.56/ 21.55 GFLOPS | Progress: (12/20) | 10.62 s [Task 22/25] Current/Best: 10.53/ 21.55 GFLOPS | Progress: (16/20) | 12.36 s [Task 22/25] Current/Best: 16.53/ 21.55 GFLOPS | Progress: (20/20) | 14.06 s Done.
- [Task 23/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 23/25] Current/Best: 11.70/ 19.86 GFLOPS | Progress: (4/20) | 5.14 s [Task 23/25] Current/Best: 9.51/ 19.86 GFLOPS | Progress: (8/20) | 11.37 s [Task 23/25] Current/Best: 11.59/ 20.75 GFLOPS | Progress: (12/20) | 15.37 s [Task 23/25] Current/Best: 6.12/ 20.75 GFLOPS | Progress: (16/20) | 19.44 s [Task 23/25] Current/Best: 15.21/ 20.75 GFLOPS | Progress: (20/20) | 22.78 s Done.
- [Task 24/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 24/25] Current/Best: 3.35/ 9.83 GFLOPS | Progress: (4/20) | 13.67 s [Task 24/25] Current/Best: 7.93/ 9.88 GFLOPS | Progress: (8/20) | 16.19 s [Task 24/25] Current/Best: 4.06/ 9.88 GFLOPS | Progress: (12/20) | 19.72 s [Task 24/25] Current/Best: 7.26/ 9.88 GFLOPS | Progress: (16/20) | 21.70 s [Task 24/25] Current/Best: 3.23/ 9.88 GFLOPS | Progress: (20/20) | 25.93 s [Task 25/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 25/25] Current/Best: 8.75/ 8.75 GFLOPS | Progress: (4/20) | 8.54 s [Task 25/25] Current/Best: 4.14/ 8.75 GFLOPS | Progress: (8/20) | 10.94 s [Task 25/25] Current/Best: 4.12/ 8.75 GFLOPS | Progress: (12/20) | 21.92 s [Task 25/25] Current/Best: 1.53/ 8.75 GFLOPS | Progress: (16/20) | 28.93 s [Task 25/25] Current/Best: 8.38/ 8.75 GFLOPS | Progress: (20
/20) | 39.89 s
+ [Task 21/25] Current/Best: 15.19/ 15.19 GFLOPS | Progress: (4/20) | 13.68 s [Task 21/25] Current/Best: 7.18/ 15.19 GFLOPS | Progress: (8/20) | 25.02 s [Task 21/25] Current/Best: 11.54/ 19.03 GFLOPS | Progress: (12/20) | 36.12 s [Task 21/25] Current/Best: 9.05/ 22.56 GFLOPS | Progress: (16/20) | 39.55 s [Task 21/25] Current/Best: 9.84/ 22.56 GFLOPS | Progress: (20/20) | 50.83 s [Task 22/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 22/25] Current/Best: 11.85/ 19.94 GFLOPS | Progress: (4/20) | 5.38 s [Task 22/25] Current/Best: 2.75/ 19.94 GFLOPS | Progress: (8/20) | 7.79 s [Task 22/25] Current/Best: 2.78/ 19.94 GFLOPS | Progress: (12/20) | 9.94 s [Task 22/25] Current/Best: 12.87/ 19.94 GFLOPS | Progress: (16/20) | 12.86 s [Task 22/25] Current/Best: 19.26/ 21.99 GFLOPS | Progress: (20/20) | 14.44 s Done.
+ [Task 23/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 23/25] Current/Best: 10.31/ 20.87 GFLOPS | Progress: (4/20) | 5.63 s [Task 23/25] Current/Best: 18.35/ 20.87 GFLOPS | Progress: (8/20) | 7.97 s [Task 23/25] Current/Best: 15.42/ 24.13 GFLOPS | Progress: (12/20) | 10.71 s [Task 23/25] Current/Best: 9.34/ 24.13 GFLOPS | Progress: (16/20) | 14.56 s [Task 23/25] Current/Best: 3.17/ 24.13 GFLOPS | Progress: (20/20) | 22.12 s Done.
+ [Task 24/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 24/25] Current/Best: 8.58/ 8.58 GFLOPS | Progress: (4/20) | 13.46 s [Task 24/25] Current/Best: 2.66/ 8.58 GFLOPS | Progress: (8/20) | 24.51 s [Task 24/25] Current/Best: 10.11/ 10.11 GFLOPS | Progress: (12/20) | 29.63 s [Task 24/25] Current/Best: 2.97/ 10.11 GFLOPS | Progress: (16/20) | 40.62 s [Task 24/25] Current/Best: 0.94/ 10.11 GFLOPS | Progress: (20/20) | 52.82 s Done.
+ [Task 25/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 25/25] Current/Best: 9.55/ 10.01 GFLOPS | Progress: (4/20) | 8.21 s [Task 25/25] Current/Best: 2.88/ 10.01 GFLOPS | Progress: (8/20) | 19.18 s [Task 25/25] Current/Best: 2.80/ 10.01 GFLOPS | Progress: (12/20) | 23.44 s [Task 25/25] Current/Best: 1.52/ 10.14 GFLOPS | Progress: (16/20) | 25.46 s [Task 25/25] Current/Best: 6.06/ 10.14 GFLOPS | Progress: (20/20) | 30.03 s
@@ -674,7 +674,6 @@ model using optimized operators to speed up our computations.
.. code-block:: none
Done.
- Done.
@@ -708,7 +707,7 @@ Verify that the optimized model runs and produces the same results:
.. code-block:: none
- class='n02123045 tabby, tabby cat' with probability=0.621104
+ class='n02123045 tabby, tabby cat' with probability=0.621105
class='n02123159 tiger cat' with probability=0.356378
class='n02124075 Egyptian cat' with probability=0.019712
class='n02129604 tiger, Panthera tigris' with probability=0.001215
@@ -766,8 +765,8 @@ improvement in comparing the optimized model to the unoptimized model.
.. code-block:: none
- optimized: {'mean': 406.5573813499941, 'median': 406.4214084500236, 'std': 0.8216902263569867}
- unoptimized: {'mean': 499.7387120300482, 'median': 499.10965995004517, 'std': 3.497109291631726}
+ optimized: {'mean': 392.4014270400221, 'median': 392.0195533499282, 'std': 1.9212977182785338}
+ unoptimized: {'mean': 467.02020688007906, 'median': 466.2648031999197, 'std': 1.6591059255929859}
@@ -790,7 +789,7 @@ profiling/benchmarking.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 13 minutes 4.355 seconds)
+ **Total running time of the script:** ( 13 minutes 6.228 seconds)
.. _sphx_glr_download_tutorial_autotvm_relay_x86.py:
diff --git a/docs/_sources/tutorial/cross_compilation_and_rpc.rst.txt b/docs/_sources/tutorial/cross_compilation_and_rpc.rst.txt
index 47721c1e7a..776629ea83 100644
--- a/docs/_sources/tutorial/cross_compilation_and_rpc.rst.txt
+++ b/docs/_sources/tutorial/cross_compilation_and_rpc.rst.txt
@@ -274,7 +274,7 @@ device and returns the measured cost. Network overhead is excluded.
.. code-block:: none
- 1.196e-07 secs/op
+ 1.128e-07 secs/op
diff --git a/docs/_sources/tutorial/intro_topi.rst.txt b/docs/_sources/tutorial/intro_topi.rst.txt
index 6177fc528d..2de5cc185b 100644
--- a/docs/_sources/tutorial/intro_topi.rst.txt
+++ b/docs/_sources/tutorial/intro_topi.rst.txt
@@ -270,7 +270,7 @@ As you can see, scheduled stages of computation have been accumulated and we can
.. code-block:: none
- [stage(a, placeholder(a, 0x2db0f4b0)), stage(b, placeholder(b, 0x298388d0)), stage(T_add, compute(T_add, body=[a[ax0, ax1, ax2] + b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T.Range(0, 10), "DataPar", ""), T.iter_var(ax2, T.Range(0, 10), "DataPar", "")], reduce_axis=[], tag=broadcast, attrs={})), stage(T_multiply, compute(T_multiply, body=[a[ax0, ax1, ax2] * b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T [...]
+ [stage(a, placeholder(a, 0x7acda80)), stage(b, placeholder(b, 0x88d9f10)), stage(T_add, compute(T_add, body=[a[ax0, ax1, ax2] + b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T.Range(0, 10), "DataPar", ""), T.iter_var(ax2, T.Range(0, 10), "DataPar", "")], reduce_axis=[], tag=broadcast, attrs={})), stage(T_multiply, compute(T_multiply, body=[a[ax0, ax1, ax2] * b[ax1, ax2]], axis=[T.iter_var(ax0, T.Range(0, 100), "DataPar", ""), T.iter_var(ax1, T.R [...]
diff --git a/docs/_sources/tutorial/sg_execution_times.rst.txt b/docs/_sources/tutorial/sg_execution_times.rst.txt
index 11eb74d0f6..6866b67aae 100644
--- a/docs/_sources/tutorial/sg_execution_times.rst.txt
+++ b/docs/_sources/tutorial/sg_execution_times.rst.txt
@@ -5,24 +5,24 @@
Computation times
=================
-**16:28.242** total execution time for **tutorial** files:
+**16:19.611** total execution time for **tutorial** files:
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_autotvm_relay_x86.py` (``autotvm_relay_x86.py``) | 13:04.355 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_autotvm_relay_x86.py` (``autotvm_relay_x86.py``) | 13:06.228 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_auto_scheduler_matmul_x86.py` (``auto_scheduler_matmul_x86.py``) | 01:21.479 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_auto_scheduler_matmul_x86.py` (``auto_scheduler_matmul_x86.py``) | 01:15.527 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_tensor_expr_get_started.py` (``tensor_expr_get_started.py``) | 01:00.064 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_tensor_expr_get_started.py` (``tensor_expr_get_started.py``) | 00:56.069 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_relay_quick_start.py` (``relay_quick_start.py``) | 00:42.642 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_relay_quick_start.py` (``relay_quick_start.py``) | 00:40.646 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_autotvm_matmul_x86.py` (``autotvm_matmul_x86.py``) | 00:17.623 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_autotvm_matmul_x86.py` (``autotvm_matmul_x86.py``) | 00:19.170 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_intro_topi.py` (``intro_topi.py``) | 00:00.973 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_intro_topi.py` (``intro_topi.py``) | 00:00.928 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_tensor_ir_blitz_course.py` (``tensor_ir_blitz_course.py``) | 00:00.887 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_tensor_ir_blitz_course.py` (``tensor_ir_blitz_course.py``) | 00:00.850 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_tutorial_cross_compilation_and_rpc.py` (``cross_compilation_and_rpc.py``) | 00:00.218 | 0.0 MB |
+| :ref:`sphx_glr_tutorial_cross_compilation_and_rpc.py` (``cross_compilation_and_rpc.py``) | 00:00.194 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_tutorial_uma.py` (``uma.py``) | 00:00.000 | 0.0 MB |
+------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/tutorial/tensor_expr_get_started.rst.txt b/docs/_sources/tutorial/tensor_expr_get_started.rst.txt
... 11202 lines suppressed ...