You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by tq...@apache.org on 2024/11/23 15:55:09 UTC
(tvm-site) branch asf-site updated: deploying docs (apache/tvm@27eed541ec647cc5d7b7ce18eee336177c594aa3)
This is an automated email from the ASF dual-hosted git repository.
tqchen pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/tvm-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new e2c5701db1 deploying docs (apache/tvm@27eed541ec647cc5d7b7ce18eee336177c594aa3)
e2c5701db1 is described below
commit e2c5701db19532e5dc14ed61883a4ffa31a50e98
Author: tvm-bot <95...@users.noreply.github.com>
AuthorDate: Sat Nov 23 15:55:00 2024 +0000
deploying docs (apache/tvm@27eed541ec647cc5d7b7ce18eee336177c594aa3)
---
.../convolution_opt.py | 458 --------
.../matrix_multiply.ipynb | 311 ------
.../tensorize.ipynb | 2 +-
.../428c6201e29ce74e73c6b41eee589f62/tensorize.py | 2 -
.../matrix_multiply_opt.ipynb | 176 ---
.../deploy_detection.py | 322 ------
.../deploy_detection.ipynb | 169 ---
.../matrix_multiply_opt.py | 374 -------
.../vta_get_started.ipynb | 373 -------
.../convolution_opt.ipynb | 194 ----
.../vta_get_started.py | 405 -------
.../matrix_multiply.py | 474 ---------
docs/_images/sphx_glr_convolution_opt_thumb.png | Bin 26794 -> 0 bytes
docs/_images/sphx_glr_deploy_detection_001.png | Bin 318961 -> 0 bytes
docs/_images/sphx_glr_deploy_detection_thumb.png | Bin 120997 -> 0 bytes
.../_images/sphx_glr_matrix_multiply_opt_thumb.png | Bin 26794 -> 0 bytes
docs/_images/sphx_glr_matrix_multiply_thumb.png | Bin 26794 -> 0 bytes
docs/_images/sphx_glr_vta_get_started_thumb.png | Bin 26794 -> 0 bytes
.../relax/tutorials/sg_execution_times.rst.txt | 6 +-
.../tensor_ir/tutorials/sg_execution_times.rst.txt | 6 +-
.../tensor_ir/tutorials/tir_creation.rst.txt | 20 +-
.../tensor_ir/tutorials/tir_transformation.rst.txt | 6 +-
docs/_sources/faq.rst.txt | 69 --
.../get_started/tutorials/ir_module.rst.txt | 8 +-
.../get_started/tutorials/quick_start.rst.txt | 4 +-
.../tutorials/sg_execution_times.rst.txt | 6 +-
.../how_to/compile_models/from_darknet.rst.txt | 2 +-
.../how_to/compile_models/from_keras.rst.txt | 2 +-
.../how_to/compile_models/from_oneflow.rst.txt | 2 +-
.../how_to/compile_models/from_paddle.rst.txt | 2 +-
.../how_to/compile_models/from_pytorch.rst.txt | 2 +-
.../how_to/compile_models/from_tensorflow.rst.txt | 2 +-
.../compile_models/sg_execution_times.rst.txt | 20 +-
.../deploy_models/deploy_model_on_adreno.rst.txt | 4 +-
.../deploy_model_on_adreno_tvmc.rst.txt | 4 +-
.../deploy_models/deploy_model_on_android.rst.txt | 2 +-
.../deploy_object_detection_pytorch.rst.txt | 4 +-
.../deploy_models/deploy_prequantized.rst.txt | 6 +-
.../deploy_prequantized_tflite.rst.txt | 2 +-
.../deploy_models/sg_execution_times.rst.txt | 18 +-
.../how_to/extend_tvm/sg_execution_times.rst.txt | 6 +-
.../how_to/extend_tvm/use_pass_instrument.rst.txt | 16 +-
docs/_sources/how_to/legacy_index.rst.txt | 1 -
.../optimize_operators/opt_conv_cuda.rst.txt | 2 +-
.../optimize_operators/opt_conv_tensorcore.rst.txt | 2 +-
.../how_to/optimize_operators/opt_gemm.rst.txt | 16 +-
.../optimize_operators/sg_execution_times.rst.txt | 8 +-
.../sg_execution_times.rst.txt | 14 +-
.../tune_network_cuda.rst.txt | 4 +-
.../tune_network_x86.rst.txt | 4 +-
.../tune_with_autotvm/sg_execution_times.rst.txt | 6 +-
.../tune_with_autotvm/tune_conv2d_cuda.rst.txt | 2 +-
.../tutorials/cross_compilation_and_rpc.rst.txt | 2 +-
.../how_to/tutorials/customize_opt.rst.txt | 4 +-
.../_sources/how_to/tutorials/optimize_llm.rst.txt | 2 +-
.../how_to/tutorials/sg_execution_times.rst.txt | 10 +-
.../work_with_microtvm/micro_autotune.rst.txt | 18 +-
.../work_with_microtvm/micro_pytorch.rst.txt | 4 +-
.../how_to/work_with_microtvm/micro_train.rst.txt | 16 +-
.../work_with_microtvm/sg_execution_times.rst.txt | 14 +-
.../work_with_relay/sg_execution_times.rst.txt | 8 +-
.../work_with_schedules/sg_execution_times.rst.txt | 16 +-
.../how_to/work_with_schedules/tensorize.rst.txt | 4 +-
docs/_sources/index.rst.txt | 1 -
.../reference/langref/relay_pattern.rst.txt | 2 +-
docs/_sources/topic/vta/dev/config.rst.txt | 73 --
docs/_sources/topic/vta/dev/hardware.rst.txt | 298 ------
docs/_sources/topic/vta/dev/index.rst.txt | 31 -
docs/_sources/topic/vta/index.rst.txt | 55 -
docs/_sources/topic/vta/install.rst.txt | 488 ---------
.../tutorials/frontend/deploy_detection.rst.txt | 473 ---------
.../topic/vta/tutorials/frontend/index.rst.txt | 43 -
.../tutorials/frontend/sg_execution_times.rst.txt | 12 -
docs/_sources/topic/vta/tutorials/index.rst.txt | 155 ---
.../topic/vta/tutorials/matrix_multiply.rst.txt | 791 --------------
.../vta/tutorials/optimize/convolution_opt.rst.txt | 806 --------------
.../topic/vta/tutorials/optimize/index.rst.txt | 59 --
.../tutorials/optimize/matrix_multiply_opt.rst.txt | 667 ------------
.../tutorials/optimize/sg_execution_times.rst.txt | 14 -
.../topic/vta/tutorials/sg_execution_times.rst.txt | 14 -
.../topic/vta/tutorials/vta_get_started.rst.txt | 717 -------------
.../tutorial/auto_scheduler_matmul_x86.rst.txt | 4 +-
docs/_sources/tutorial/autotvm_matmul_x86.rst.txt | 20 +-
docs/_sources/tutorial/autotvm_relay_x86.rst.txt | 58 +-
docs/_sources/tutorial/intro_topi.rst.txt | 2 +-
docs/_sources/tutorial/sg_execution_times.rst.txt | 20 +-
.../tutorial/tensor_expr_get_started.rst.txt | 47 +-
docs/api/links.html | 10 -
docs/api/python/auto_scheduler.html | 10 -
docs/api/python/autotvm.html | 10 -
docs/api/python/contrib.html | 10 -
docs/api/python/driver.html | 10 -
docs/api/python/error.html | 10 -
docs/api/python/graph_executor.html | 10 -
docs/api/python/index.html | 10 -
docs/api/python/ir.html | 10 -
docs/api/python/micro.html | 10 -
docs/api/python/ndarray.html | 10 -
docs/api/python/relay/analysis.html | 10 -
docs/api/python/relay/backend.html | 10 -
docs/api/python/relay/dataflow_pattern.html | 10 -
docs/api/python/relay/frontend.html | 10 -
docs/api/python/relay/image.html | 10 -
docs/api/python/relay/index.html | 10 -
docs/api/python/relay/nn.html | 10 -
docs/api/python/relay/testing.html | 10 -
docs/api/python/relay/transform.html | 10 -
docs/api/python/relay/vision.html | 10 -
docs/api/python/rpc.html | 10 -
docs/api/python/runtime.html | 10 -
docs/api/python/target.html | 10 -
docs/api/python/te.html | 10 -
docs/api/python/tir.html | 10 -
docs/api/python/topi.html | 10 -
docs/api/python/vta/index.html | 10 -
docs/arch/debugger.html | 1 -
docs/arch/device_target_interactions.html | 1 -
docs/arch/index.html | 1 -
.../arch/introduction_to_module_serialization.html | 1 -
docs/arch/pass_infra.html | 1 -
docs/arch/runtime.html | 1 -
docs/arch/runtimes/vulkan.html | 1 -
docs/commit_hash | 2 +-
docs/contribute/ci.html | 1 -
docs/contribute/code_guide.html | 1 -
docs/contribute/code_review.html | 1 -
docs/contribute/committer_guide.html | 1 -
docs/contribute/community.html | 1 -
docs/contribute/document.html | 1 -
docs/contribute/error_handling.html | 1 -
docs/contribute/git_howto.html | 1 -
docs/contribute/index.html | 5 +-
docs/contribute/pull_request.html | 1 -
docs/contribute/release_process.html | 1 -
docs/deep_dive/relax/abstraction.html | 1 -
docs/deep_dive/relax/index.html | 1 -
docs/deep_dive/relax/learning.html | 1 -
docs/deep_dive/relax/tutorials/index.html | 1 -
docs/deep_dive/relax/tutorials/relax_creation.html | 17 +-
.../relax/tutorials/relax_transformation.html | 16 +-
.../relax/tutorials/sg_execution_times.html | 7 +-
docs/deep_dive/tensor_ir/abstraction.html | 1 -
docs/deep_dive/tensor_ir/index.html | 1 -
docs/deep_dive/tensor_ir/learning.html | 1 -
docs/deep_dive/tensor_ir/tutorials/index.html | 1 -
.../tensor_ir/tutorials/sg_execution_times.html | 7 +-
.../tensor_ir/tutorials/tir_creation.html | 21 +-
.../tensor_ir/tutorials/tir_transformation.html | 7 +-
docs/deploy/android.html | 10 -
docs/deploy/arm_compute_lib.html | 10 -
docs/deploy/bnns.html | 10 -
docs/deploy/cpp_deploy.html | 10 -
docs/deploy/hls.html | 10 -
docs/deploy/index.html | 10 -
docs/deploy/integrate.html | 10 -
docs/deploy/tensorrt.html | 10 -
docs/deploy/vitis_ai.html | 10 -
docs/dev/benchmark.html | 10 -
docs/dev/codebase_walkthrough.html | 10 -
docs/dev/convert_layout.html | 10 -
docs/dev/debugger.html | 10 -
docs/dev/device_target_interactions.html | 10 -
docs/dev/frontend/tensorflow.html | 10 -
docs/dev/how_to.html | 10 -
docs/dev/how_to/debugging_tvm.html | 1 -
docs/dev/how_to/how_to.html | 1 -
docs/dev/how_to/relay_add_op.html | 1 -
docs/dev/how_to/relay_add_pass.html | 1 -
docs/dev/how_to/relay_bring_your_own_codegen.html | 1 -
docs/dev/hybrid_script.html | 10 -
docs/dev/index.html | 10 -
docs/dev/inferbound.html | 10 -
docs/dev/introduction_to_module_serialization.html | 10 -
docs/dev/microtvm_design.html | 10 -
docs/dev/model_library_format.html | 10 -
docs/dev/pass_infra.html | 10 -
docs/dev/pytest_target_parametrization.html | 10 -
docs/dev/relay_add_op.html | 10 -
docs/dev/relay_add_pass.html | 10 -
docs/dev/relay_bring_your_own_codegen.html | 10 -
docs/dev/relay_intro.html | 10 -
docs/dev/relay_op_strategy.html | 10 -
docs/dev/runtime.html | 10 -
docs/dev/runtimes/vulkan.html | 10 -
docs/dev/security.html | 10 -
docs/dev/tutorial/codebase_walkthrough.html | 1 -
docs/dev/tutorial/index.html | 5 +-
docs/dev/virtual_machine.html | 10 -
docs/errors.html | 1 -
docs/faq.html | 524 ---------
docs/genindex.html | 1 -
docs/get_started/overview.html | 1 -
docs/get_started/tutorials/index.html | 1 -
docs/get_started/tutorials/ir_module.html | 9 +-
docs/get_started/tutorials/quick_start.html | 5 +-
docs/get_started/tutorials/sg_execution_times.html | 7 +-
docs/how_to/compile_models/from_coreml.html | 2 -
docs/how_to/compile_models/from_darknet.html | 4 +-
docs/how_to/compile_models/from_keras.html | 4 +-
docs/how_to/compile_models/from_oneflow.html | 14 +-
docs/how_to/compile_models/from_onnx.html | 2 -
docs/how_to/compile_models/from_paddle.html | 4 +-
docs/how_to/compile_models/from_pytorch.html | 8 +-
docs/how_to/compile_models/from_tensorflow.html | 4 +-
docs/how_to/compile_models/from_tflite.html | 2 -
docs/how_to/compile_models/index.html | 2 -
docs/how_to/compile_models/sg_execution_times.html | 21 +-
docs/how_to/deploy/adreno.html | 2 -
docs/how_to/deploy/android.html | 2 -
docs/how_to/deploy/arm_compute_lib.html | 2 -
docs/how_to/deploy/bnns.html | 2 -
docs/how_to/deploy/cpp_deploy.html | 2 -
docs/how_to/deploy/hls.html | 2 -
docs/how_to/deploy/index.html | 2 -
docs/how_to/deploy/integrate.html | 2 -
docs/how_to/deploy/mrvl.html | 2 -
docs/how_to/deploy/tensorrt.html | 2 -
docs/how_to/deploy/vitis_ai.html | 2 -
.../deploy_models/deploy_model_on_adreno.html | 6 +-
.../deploy_models/deploy_model_on_adreno_tvmc.html | 15 +-
.../deploy_models/deploy_model_on_android.html | 4 +-
.../how_to/deploy_models/deploy_model_on_nano.html | 2 -
.../how_to/deploy_models/deploy_model_on_rasp.html | 2 -
.../deploy_object_detection_pytorch.html | 22 +-
docs/how_to/deploy_models/deploy_prequantized.html | 10 +-
.../deploy_models/deploy_prequantized_tflite.html | 4 +-
docs/how_to/deploy_models/deploy_sparse.html | 2 -
docs/how_to/deploy_models/index.html | 2 -
docs/how_to/deploy_models/sg_execution_times.html | 23 +-
docs/how_to/dev/index.html | 1 -
docs/how_to/dev/pytest_target_parametrization.html | 1 -
docs/how_to/dev/setup_rpc_system.html | 1 -
docs/how_to/extend_tvm/index.html | 2 -
docs/how_to/extend_tvm/low_level_custom_pass.html | 2 -
docs/how_to/extend_tvm/sg_execution_times.html | 7 +-
docs/how_to/extend_tvm/use_pass_infra.html | 2 -
docs/how_to/extend_tvm/use_pass_instrument.html | 18 +-
docs/how_to/legacy_index.html | 3 -
docs/how_to/optimize_operators/index.html | 2 -
docs/how_to/optimize_operators/opt_conv_cuda.html | 4 +-
.../optimize_operators/opt_conv_tensorcore.html | 4 +-
docs/how_to/optimize_operators/opt_gemm.html | 18 +-
.../optimize_operators/sg_execution_times.html | 9 +-
docs/how_to/profile/index.html | 2 -
docs/how_to/profile/papi.html | 6 +-
docs/how_to/tune_with_autoscheduler/index.html | 2 -
.../sg_execution_times.html | 15 +-
.../tune_conv2d_layer_cuda.html | 2 -
.../tune_with_autoscheduler/tune_network_arm.html | 2 -
.../tune_with_autoscheduler/tune_network_cuda.html | 6 +-
.../tune_with_autoscheduler/tune_network_mali.html | 2 -
.../tune_with_autoscheduler/tune_network_x86.html | 6 +-
.../tune_with_autoscheduler/tune_sparse_x86.html | 2 -
docs/how_to/tune_with_autotvm/index.html | 2 -
.../tune_with_autotvm/sg_execution_times.html | 7 +-
.../how_to/tune_with_autotvm/tune_conv2d_cuda.html | 4 +-
docs/how_to/tune_with_autotvm/tune_relay_arm.html | 2 -
docs/how_to/tune_with_autotvm/tune_relay_cuda.html | 2 -
.../tune_with_autotvm/tune_relay_mobile_gpu.html | 2 -
docs/how_to/tune_with_autotvm/tune_relay_x86.html | 2 -
.../tutorials/cross_compilation_and_rpc.html | 3 +-
docs/how_to/tutorials/customize_opt.html | 5 +-
docs/how_to/tutorials/e2e_opt_model.html | 1 -
docs/how_to/tutorials/index.html | 1 -
docs/how_to/tutorials/optimize_llm.html | 3 +-
docs/how_to/tutorials/sg_execution_times.html | 11 +-
docs/how_to/work_with_microtvm/index.html | 2 -
docs/how_to/work_with_microtvm/micro_aot.html | 2 -
docs/how_to/work_with_microtvm/micro_autotune.html | 20 +-
.../work_with_microtvm/micro_custom_ide.html | 2 -
docs/how_to/work_with_microtvm/micro_ethosu.html | 2 -
.../work_with_microtvm/micro_mlperftiny.html | 2 -
docs/how_to/work_with_microtvm/micro_pytorch.html | 6 +-
docs/how_to/work_with_microtvm/micro_tflite.html | 2 -
docs/how_to/work_with_microtvm/micro_train.html | 18 +-
docs/how_to/work_with_microtvm/micro_tvmc.html | 2 -
.../work_with_microtvm/sg_execution_times.html | 15 +-
docs/how_to/work_with_relay/build_gcn.html | 2 -
docs/how_to/work_with_relay/index.html | 2 -
.../how_to/work_with_relay/sg_execution_times.html | 9 +-
.../how_to/work_with_relay/using_external_lib.html | 2 -
.../work_with_relay/using_pipeline_executor.html | 2 -
docs/how_to/work_with_relay/using_relay_viz.html | 2 -
docs/how_to/work_with_schedules/extern_op.html | 2 -
docs/how_to/work_with_schedules/index.html | 2 -
docs/how_to/work_with_schedules/reduction.html | 2 -
docs/how_to/work_with_schedules/scan.html | 2 -
.../work_with_schedules/schedule_primitives.html | 2 -
.../work_with_schedules/sg_execution_times.html | 17 +-
docs/how_to/work_with_schedules/tedd.html | 2 -
docs/how_to/work_with_schedules/tensorize.html | 4 -
docs/how_to/work_with_schedules/tuple_inputs.html | 2 -
docs/index.html | 2 -
docs/install/docker.html | 1 -
docs/install/from_source.html | 1 -
docs/install/index.html | 1 -
docs/langref/hybrid_script.html | 10 -
docs/langref/index.html | 10 -
docs/langref/relay_adt.html | 10 -
docs/langref/relay_expr.html | 10 -
docs/langref/relay_op.html | 10 -
docs/langref/relay_pattern.html | 10 -
docs/langref/relay_type.html | 10 -
docs/microtvm/index.html | 10 -
docs/objects.inv | Bin 31252 -> 30585 bytes
docs/profiling/index.html | 10 -
docs/profiling/papi.html | 10 -
docs/py-modindex.html | 1 -
docs/reference/api/links.html | 1 -
docs/reference/api/python/auto_scheduler.html | 5 +-
docs/reference/api/python/autotvm.html | 1 -
docs/reference/api/python/contrib.html | 1 -
docs/reference/api/python/dlight.html | 1 -
docs/reference/api/python/driver.html | 1 -
docs/reference/api/python/error.html | 1 -
docs/reference/api/python/graph_executor.html | 1 -
docs/reference/api/python/index.html | 1 -
docs/reference/api/python/instrument.html | 1 -
docs/reference/api/python/ir.html | 1 -
docs/reference/api/python/meta_schedule.html | 1 -
docs/reference/api/python/micro.html | 1 -
docs/reference/api/python/relax/analysis.html | 1 -
docs/reference/api/python/relax/block_builder.html | 1 -
docs/reference/api/python/relax/frontend.html | 1 -
docs/reference/api/python/relax/op.html | 1 -
docs/reference/api/python/relax/relax.html | 1 -
docs/reference/api/python/relax/transform.html | 1 -
docs/reference/api/python/relay/analysis.html | 1 -
docs/reference/api/python/relay/backend.html | 1 -
.../api/python/relay/dataflow_pattern.html | 1 -
docs/reference/api/python/relay/frontend.html | 1 -
docs/reference/api/python/relay/image.html | 1 -
docs/reference/api/python/relay/index.html | 1 -
docs/reference/api/python/relay/nn.html | 1 -
docs/reference/api/python/relay/testing.html | 1 -
docs/reference/api/python/relay/transform.html | 1 -
docs/reference/api/python/relay/vision.html | 1 -
docs/reference/api/python/rpc.html | 1 -
docs/reference/api/python/runtime/disco.html | 1 -
docs/reference/api/python/runtime/ndarray.html | 1 -
docs/reference/api/python/runtime/profiling.html | 1 -
docs/reference/api/python/runtime/relax_vm.html | 1 -
docs/reference/api/python/runtime/runtime.html | 1 -
docs/reference/api/python/target.html | 1 -
docs/reference/api/python/te.html | 1 -
docs/reference/api/python/tir/analysis.html | 1 -
docs/reference/api/python/tir/schedule.html | 1 -
docs/reference/api/python/tir/stmt_functor.html | 1 -
docs/reference/api/python/tir/tir.html | 1 -
docs/reference/api/python/tir/transform.html | 1 -
docs/reference/api/python/topi.html | 283 ++++-
docs/reference/api/python/transform.html | 1 -
.../api/typedoc/classes/ArtifactCache.html | 14 +-
.../typedoc/classes/ArtifactIndexedDBCache.html | 16 +-
docs/reference/api/typedoc/classes/DLDataType.html | 14 +-
docs/reference/api/typedoc/classes/DLDevice.html | 12 +-
docs/reference/api/typedoc/classes/Instance.html | 116 +-
.../classes/LinearCongruentialGenerator.html | 16 +-
docs/reference/api/typedoc/classes/Module.html | 12 +-
docs/reference/api/typedoc/classes/NDArray.html | 30 +-
docs/reference/api/typedoc/classes/RPCServer.html | 16 +-
docs/reference/api/typedoc/classes/Scalar.html | 8 +-
docs/reference/api/typedoc/classes/TVMArray.html | 16 +-
docs/reference/api/typedoc/classes/TVMObject.html | 12 +-
.../api/typedoc/classes/VirtualMachine.html | 10 +-
.../classes/_internal_.AsyncifyHandler.html | 12 +-
.../classes/_internal_.CachedCallStack.html | 36 +-
.../classes/_internal_.CanvasRenderManager.html | 10 +-
.../typedoc/classes/_internal_.Environment.html | 14 +-
.../api/typedoc/classes/_internal_.FFILibrary.html | 22 +-
.../api/typedoc/classes/_internal_.Memory.html | 36 +-
.../typedoc/classes/_internal_.PackedFuncCell.html | 8 +-
.../typedoc/classes/_internal_.RuntimeContext.html | 60 +-
.../api/typedoc/classes/_internal_.TVMString.html | 14 +-
.../typedoc/classes/_internal_.WebGPUContext.html | 28 +-
.../enums/_internal_.AsyncifyStateKind.html | 8 +-
.../typedoc/enums/_internal_.RPCServerState.html | 14 +-
docs/reference/api/typedoc/functions/assert.html | 2 +-
.../api/typedoc/functions/createPolyfillWASI.html | 2 +-
.../api/typedoc/functions/deleteNDArrayCache.html | 2 +-
.../api/typedoc/functions/detectGPUDevice.html | 2 +-
.../api/typedoc/functions/hasNDArrayInCache.html | 2 +-
.../api/typedoc/functions/instantiate.html | 2 +-
docs/reference/api/typedoc/functions/wasmPath.html | 2 +-
.../typedoc/interfaces/ArtifactCacheTemplate.html | 10 +-
.../api/typedoc/interfaces/Disposable.html | 4 +-
.../typedoc/interfaces/GPUDeviceDetectOutput.html | 8 +-
.../api/typedoc/interfaces/InitProgressReport.html | 8 +-
.../api/typedoc/interfaces/LibraryProvider.html | 6 +-
.../interfaces/_internal_.FunctionInfo.html | 8 +-
.../interfaces/_internal_.NDArrayCacheEntry.html | 14 +-
.../interfaces/_internal_.NDArrayShardEntry.html | 10 +-
.../api/typedoc/types/InitProgressCallback.html | 2 +-
docs/reference/api/typedoc/types/PackedFunc.html | 2 +-
.../typedoc/types/_internal_.AsyncPackedFunc.html | 2 +-
.../types/_internal_.FObjectConstructor.html | 2 +-
.../types/_internal_.FTVMWasmAllocSpace.html | 2 +-
.../types/_internal_.FTVMWasmFreeSpace.html | 2 +-
.../types/_internal_.FTVMWasmPackedCFunc.html | 2 +-
.../api/typedoc/types/_internal_.Pointer.html | 2 +-
.../typedoc/types/_internal_.TVMObjectBase.html | 2 +-
docs/reference/langref/hybrid_script.html | 1 -
docs/reference/langref/index.html | 1 -
docs/reference/langref/relay_adt.html | 1 -
docs/reference/langref/relay_expr.html | 1 -
docs/reference/langref/relay_op.html | 1 -
docs/reference/langref/relay_pattern.html | 3 +-
docs/reference/langref/relay_type.html | 1 -
docs/reference/publications.html | 1 -
docs/reference/security.html | 1 -
docs/search.html | 1 -
docs/searchindex.js | 2 +-
docs/topic/microtvm/index.html | 5 +-
docs/topic/vta/dev/config.html | 563 ----------
docs/topic/vta/dev/hardware.html | 739 -------------
docs/topic/vta/dev/index.html | 487 ---------
docs/topic/vta/index.html | 489 ---------
docs/topic/vta/install.html | 875 ---------------
.../vta/tutorials/frontend/deploy_detection.html | 811 --------------
docs/topic/vta/tutorials/frontend/index.html | 483 ---------
.../vta/tutorials/frontend/sg_execution_times.html | 461 --------
docs/topic/vta/tutorials/index.html | 503 ---------
docs/topic/vta/tutorials/matrix_multiply.html | 1037 ------------------
.../vta/tutorials/optimize/convolution_opt.html | 1116 --------------------
docs/topic/vta/tutorials/optimize/index.html | 486 ---------
.../tutorials/optimize/matrix_multiply_opt.html | 994 -----------------
.../vta/tutorials/optimize/sg_execution_times.html | 465 --------
docs/topic/vta/tutorials/sg_execution_times.html | 465 --------
docs/topic/vta/tutorials/vta_get_started.html | 920 ----------------
docs/tutorial/auto_scheduler_matmul_x86.html | 5 +-
docs/tutorial/autotvm_matmul_x86.html | 21 +-
docs/tutorial/autotvm_relay_x86.html | 277 +++--
docs/tutorial/index.html | 1 -
docs/tutorial/intro_topi.html | 3 +-
docs/tutorial/introduction.html | 1 -
docs/tutorial/relay_quick_start.html | 1 -
docs/tutorial/sg_execution_times.html | 25 +-
docs/tutorial/tensor_expr_get_started.html | 44 +-
docs/tutorial/tensor_ir_blitz_course.html | 1 -
docs/tutorial/tvmc_command_line_driver.html | 1 -
docs/tutorial/tvmc_python.html | 1 -
docs/tutorial/uma.html | 1 -
.../auto_scheduler/tune_conv2d_layer_cuda.html | 10 -
.../tutorials/auto_scheduler/tune_network_arm.html | 10 -
.../auto_scheduler/tune_network_cuda.html | 10 -
.../auto_scheduler/tune_network_mali.html | 10 -
.../tutorials/auto_scheduler/tune_network_x86.html | 10 -
docs/tutorials/auto_scheduler/tune_sparse_x86.html | 10 -
docs/tutorials/autotvm/tune_conv2d_cuda.html | 10 -
docs/tutorials/autotvm/tune_relay_arm.html | 10 -
docs/tutorials/autotvm/tune_relay_cuda.html | 10 -
docs/tutorials/autotvm/tune_relay_mobile_gpu.html | 10 -
docs/tutorials/autotvm/tune_relay_x86.html | 10 -
docs/tutorials/dev/bring_your_own_datatypes.html | 10 -
docs/tutorials/dev/low_level_custom_pass.html | 10 -
docs/tutorials/dev/use_pass_infra.html | 10 -
docs/tutorials/dev/use_pass_instrument.html | 10 -
docs/tutorials/frontend/build_gcn.html | 10 -
.../frontend/deploy_model_on_android.html | 10 -
docs/tutorials/frontend/deploy_model_on_rasp.html | 10 -
.../frontend/deploy_object_detection_pytorch.html | 10 -
docs/tutorials/frontend/deploy_prequantized.html | 10 -
.../frontend/deploy_prequantized_tflite.html | 10 -
docs/tutorials/frontend/deploy_quantized.html | 10 -
docs/tutorials/frontend/deploy_sparse.html | 10 -
docs/tutorials/frontend/from_caffe2.html | 10 -
docs/tutorials/frontend/from_coreml.html | 10 -
docs/tutorials/frontend/from_darknet.html | 10 -
docs/tutorials/frontend/from_keras.html | 10 -
docs/tutorials/frontend/from_mxnet.html | 10 -
docs/tutorials/frontend/from_onnx.html | 10 -
docs/tutorials/frontend/from_paddle.html | 10 -
docs/tutorials/frontend/from_pytorch.html | 10 -
docs/tutorials/frontend/from_tensorflow.html | 10 -
docs/tutorials/frontend/from_tflite.html | 10 -
docs/tutorials/frontend/using_external_lib.html | 10 -
.../get_started/auto_scheduler_matmul_x86.html | 10 -
docs/tutorials/get_started/autotvm_matmul_x86.html | 10 -
docs/tutorials/get_started/autotvm_relay_x86.html | 10 -
.../get_started/cross_compilation_and_rpc.html | 10 -
docs/tutorials/get_started/install.html | 10 -
docs/tutorials/get_started/introduction.html | 10 -
docs/tutorials/get_started/relay_quick_start.html | 10 -
.../get_started/tensor_expr_get_started.html | 10 -
.../get_started/tvmc_command_line_driver.html | 10 -
docs/tutorials/get_started/tvmc_python.html | 10 -
docs/tutorials/index.html | 10 -
docs/tutorials/language/extern_op.html | 10 -
docs/tutorials/language/reduction.html | 10 -
docs/tutorials/language/scan.html | 10 -
docs/tutorials/language/schedule_primitives.html | 10 -
docs/tutorials/language/tedd.html | 10 -
docs/tutorials/language/tensorize.html | 10 -
docs/tutorials/language/tuple_inputs.html | 10 -
docs/tutorials/micro/micro_autotune.html | 10 -
docs/tutorials/micro/micro_reference_vm.html | 10 -
docs/tutorials/micro/micro_tflite.html | 10 -
docs/tutorials/optimize/opt_conv_cuda.html | 10 -
docs/tutorials/optimize/opt_conv_tensorcore.html | 10 -
docs/tutorials/optimize/opt_gemm.html | 10 -
docs/tutorials/topi/intro_topi.html | 10 -
docs/vta/dev/config.html | 10 -
docs/vta/dev/hardware.html | 10 -
docs/vta/dev/index.html | 10 -
docs/vta/index.html | 10 -
docs/vta/install.html | 10 -
506 files changed, 1215 insertions(+), 22045 deletions(-)
diff --git a/docs/_downloads/13ef71e33eaef0855c6e883d9ec5d632/convolution_opt.py b/docs/_downloads/13ef71e33eaef0855c6e883d9ec5d632/convolution_opt.py
deleted file mode 100644
index 521a73ab51..0000000000
--- a/docs/_downloads/13ef71e33eaef0855c6e883d9ec5d632/convolution_opt.py
+++ /dev/null
@@ -1,458 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-2D Convolution Optimization
-===========================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This tutorial provides an overview on how to use TVM to map a 2D convolution
-workload efficiently on the VTA design.
-We recommend covering the :ref:`vta-mat-mult-opt` tutorial first.
-
-2D convolution is dominant in most computer vision deep neural networks.
-In this tutorial, we will demonstrate TVM schedule optimizations to map
-2D convolution operators in NCHW layout onto VTA.
-We also introduce the notion of latency hiding, which allows us to
-maximize VTA's compute and memory resource utilization.
-"""
-
-######################################################################
-# RPC Setup
-# ---------
-# We start by programming the Pynq's FPGA and building its RPC runtime.
-
-from __future__ import absolute_import, print_function
-
-import os
-import tvm
-import tvm.testing
-from tvm import te
-import vta
-import numpy as np
-
-from tvm import rpc
-from tvm.contrib import utils
-from vta.testing import simulator
-
-# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
-env = vta.get_env()
-
-# We read the Pynq RPC host IP address and port number from the OS environment
-host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
-port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
-# We configure both the bitstream and the runtime system on the Pynq
-# to match the VTA configuration specified by the vta_config.json file.
-if env.TARGET == "pynq":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
-# In simulation mode, host the RPC server locally.
-elif env.TARGET in ["sim", "tsim"]:
- remote = rpc.LocalSession()
-
-######################################################################
-# Computation Declaration
-# -----------------------
-# As a first step, we need to describe our 2D convolution computation
-# in NCHW format.
-#
-# We define the 2D convolution shape by the batch size,
-# spatial dimensions, input channels, output channels, kernel dimensions,
-# kernel dimensions, padding dimensions, and stride dimensions.
-#
-# We pick the shape of the 9th convolutional layer of the ResNet-18
-# architecture as our convolution workload parameters.
-#
-# We've added extra operators to the 2D convolution that apply
-# shifting and clipping to the output in order to mimic a fixed-point
-# convolution followed by a rectified linear activation.
-# We describe the TVM dataflow graph of the 2D convolution layer below:
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/conv2d_dataflow.png
-# :align: center
-#
-# This computation is intentionally too large to fit onto VTA's on-chip
-# buffers all at once. Therefore in the scheduling phase we'll
-# rely on computation blocking strategies to break the computation down into
-# manageable chunks.
-#
-# .. note::
-#
-# *Spatial padding*
-#
-# Note that we'll need to import the TOPI library to apply spatial padding
-# on the input feature map tensor.
-# Spatial padding facilitates blocking in the context of 2D convolutions
-# due to the fact that the same (x, y) spatial location of the input
-# feature map of any given layer is read more than once if the convolution
-# kernel window size is greater than one.
-# On CPUs, and GPUs, one way to increase efficiency of memory accesses
-# when parallelizing work is spatial packing, which requires data re-layout.
-# VTA load DMA engine can insert padding automatically so that the original
-# input feature map does not have to be re-packed in memory.
-#
-# We show the effect of VTA's on the fly spatial padding when data is being
-# loaded from DRAM into VTA's SRAM, following a 2D strided and padded memory
-# read.
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/padding.png
-# :align: center
-# :width: 480px
-
-from tvm import topi
-
-# 2D convolution layer dimensions taken from ResNet-18 architecture
-# (9th convolutional layer)
-batch_size = 1
-height = 14
-width = 14
-in_channels = 256
-out_channels = 256
-kernel_h = 3
-kernel_w = 3
-pad_h = 1
-pad_w = 1
-stride_h = 1
-stride_w = 1
-assert batch_size % env.BATCH == 0
-assert in_channels % env.BLOCK_IN == 0
-assert out_channels % env.BLOCK_OUT == 0
-
-# Input feature map: (N, IC, H, W, n, ic)
-data_shape = (
- batch_size // env.BATCH,
- in_channels // env.BLOCK_IN,
- height,
- width,
- env.BATCH,
- env.BLOCK_IN,
-)
-# Kernel: (OC, IC, H, W, oc, ic)
-kernel_shape = (
- out_channels // env.BLOCK_OUT,
- in_channels // env.BLOCK_IN,
- kernel_h,
- kernel_w,
- env.BLOCK_OUT,
- env.BLOCK_IN,
-)
-# Derive output feature map dimensions
-fout_height = (height + 2 * pad_h - kernel_h) // stride_h + 1
-fout_width = (width + 2 * pad_w - kernel_w) // stride_w + 1
-# Output feature map: (N, OC, H, W, n, oc)
-output_shape = (
- batch_size // env.BATCH,
- out_channels // env.BLOCK_OUT,
- fout_height,
- fout_width,
- env.BATCH,
- env.BLOCK_OUT,
-)
-
-# Convolution reduction axes
-dy = te.reduce_axis((0, kernel_h), name="dy")
-dx = te.reduce_axis((0, kernel_w), name="dx")
-ic = te.reduce_axis((0, in_channels // env.BLOCK_IN), name="ic")
-ic_tns = te.reduce_axis((0, env.BLOCK_IN), name="ic_tns")
-
-# Input placeholder tensors
-data = te.placeholder(data_shape, name="data", dtype=env.inp_dtype)
-kernel = te.placeholder(kernel_shape, name="kernel", dtype=env.wgt_dtype)
-
-# Copy buffers:
-# Apply spatial padding to input feature map
-data_buf = topi.nn.pad(data, [0, 0, pad_h, pad_w, 0, 0], name="data_buf")
-kernel_buf = te.compute(kernel_shape, lambda *i: kernel(*i), "kernel_buf")
-
-# Declare 2D convolution
-res_conv = te.compute(
- output_shape,
- lambda bo, co, i, j, bi, ci: te.sum(
- data_buf[bo, ic, i * stride_h + dy, j * stride_w + dx, bi, ic_tns].astype(env.acc_dtype)
- * kernel_buf[co, ic, dy, dx, ci, ic_tns].astype(env.acc_dtype),
- axis=[ic, dy, dx, ic_tns],
- ),
- name="res_conv",
-)
-
-# Add shift stage for fix-point normalization
-res_shr = te.compute(output_shape, lambda *i: res_conv(*i) >> 8, name="res_shr")
-
-# Apply clipping between (0, input max value)
-inp_max = (1 << (env.INP_WIDTH - 1)) - 1
-res_max = te.compute(output_shape, lambda *i: tvm.te.max(res_shr(*i), 0), "res_max")
-res_min = te.compute(output_shape, lambda *i: tvm.te.min(res_max(*i), inp_max), "res_min")
-
-# Result Tensor
-res = te.compute(output_shape, lambda *i: res_min(*i).astype(env.inp_dtype), name="res")
-
-
-######################################################################
-# Scheduling the Computation
-# --------------------------
-# We'll look at a set of schedule transformations necessary to map the
-# 2D convolution onto VTA in an efficient fashion.
-# Those include:
-#
-# - Computation blocking
-# - Virtual threading to increase compute utilization
-# - Lowering to VTA hardware intrinsics
-
-# Create TVM schedule
-s = te.create_schedule(res.op)
-# Let's look at the default TVM schedule
-print(tvm.lower(s, [data, kernel, res], simple_mode=True))
-
-######################################################################
-# Blocking the Computation
-# ~~~~~~~~~~~~~~~~~~~~~~~~
-# The 2D convolution is by default too large for activations or kernel weights
-# to fit on VTA's on-chip buffers all at once.
-# We apply blocking along input channels, output channels, and along
-# the height spatial dimensions.
-# We don't apply blocking along the width spatial dimension since it's
-# the innermost dimension in the NCHW layout (and consequently to increase
-# locality, it's best not to block along the innermost dimension).
-
-# Let's define tiling sizes
-b_block = 1 // env.BATCH
-oc_block = 128 // env.BLOCK_OUT
-ic_block = 16 // env.BLOCK_IN
-h_block = 7
-w_block = 14
-
-# Tile the output tensor along the spatial and output channel dimensions
-# (since by default we are doing single batch inference, the split along
-# the batch dimension has no effect)
-b, oc, y, x, b_tns, oc_tns = s[res].op.axis
-b_out, b_inn = s[res].split(b, factor=b_block)
-oc_out, oc_inn = s[res].split(oc, factor=oc_block)
-y_out, y_inn = s[res].split(y, factor=h_block)
-x_out, x_inn = s[res].split(x, factor=w_block)
-s[res].reorder(b_out, oc_out, y_out, x_out, b_inn, oc_inn, y_inn, x_inn, b_tns, oc_tns)
-
-# Move intermediate computation into each output compute tile
-s[res_conv].compute_at(s[res], x_out)
-s[res_shr].compute_at(s[res], x_out)
-s[res_max].compute_at(s[res], x_out)
-s[res_min].compute_at(s[res], x_out)
-
-# Apply additional loop split along reduction axis (input channel)
-b_inn, oc_inn, y_inn, x_inn, b_tns, oc_tns = s[res_conv].op.axis
-ic_out, ic_inn = s[res_conv].split(ic, factor=ic_block)
-
-# Reorder axes.
-# 1) Group the VTA tensor axes in the inner most position: b_tns, oc_tns, ic_tns
-# to allow TVM to tensorize.
-# 2) We move the ic_out axis all the way out of the convolution loop to block
-# along the reduction axis.
-# 3) Now we re-order the block axes: b_inn, oc_inn, y_inn, x_inn, ic_inn, dy, dx.
-# VTA runtime/hardware requires us to write to a different output feature map
-# location for every VTA tensor operation.
-# This restriction requires us to order one of oc_inn, y_inn or x_inn right
-# before b_tns, since they all affect output feature map indexing.
-# Therefore, we choose to bring x_inn inside as shown below.
-s[res_conv].reorder(ic_out, b_inn, oc_inn, y_inn, ic_inn, dy, dx, x_inn, b_tns, oc_tns, ic_tns)
-
-######################################################################
-# Virtual Threading
-# ~~~~~~~~~~~~~~~~~
-# Virtual threading is a mechanism that increases task-level pipeline
-# parallelism in the VTA hardware design.
-# Put it another way, it increases compute resource utilization by hiding
-# memory access latency.
-#
-# In the implementation below, virtual threading distributes work across two
-# threads split along the output channel axis.
-# We show how work is split when computing the 2D convolution in the figure
-# below.
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/virtual_threading.png
-# :align: center
-# :width: 480px
-
-# VTA only supports 2 virtual threads
-v_threads = 2
-
-# Perform virtual thread split along output channel outer axis
-_, tx = s[res].split(oc_out, factor=v_threads)
-s[res].reorder(tx, b_out)
-s[res].bind(tx, te.thread_axis("cthread"))
-
-# Let's look at the current TVM schedule after blocking and virtual threading
-print(tvm.lower(s, [data, kernel, res], simple_mode=True))
-
-######################################################################
-# Lowering Copies to DMA Transfers
-# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-# Next we set the buffer scopes to the corresponding on-chip VTA SRAM buffers.
-# We move the load loops into the 2D convolution computation loop to stage
-# memory loads such that they fit in the on-chip SRAM buffers.
-# Finally we annotate the load/store loop outer axes with the DMA copy pragma
-# to perform bulk memory transfers on VTA.
-
-# Set scope of SRAM buffers
-s[data_buf].set_scope(env.inp_scope)
-s[kernel_buf].set_scope(env.wgt_scope)
-s[res_conv].set_scope(env.acc_scope)
-s[res_shr].set_scope(env.acc_scope)
-s[res_min].set_scope(env.acc_scope)
-s[res_max].set_scope(env.acc_scope)
-
-# Block data and kernel cache reads
-s[data_buf].compute_at(s[res_conv], ic_out)
-s[kernel_buf].compute_at(s[res_conv], ic_out)
-
-# Use DMA copy pragma on DRAM->SRAM operations
-s[data_buf].pragma(s[data_buf].op.axis[0], env.dma_copy)
-s[kernel_buf].pragma(s[kernel_buf].op.axis[0], env.dma_copy)
-
-# Use DMA copy pragma on SRAM->DRAM operation in each result block
-# (this implies that these copies should be performed along b_inn,
-# or result axis 4)
-s[res].pragma(s[res].op.axis[4], env.dma_copy)
-
-######################################################################
-# Lowering Computation to VTA Compute Intrinsics
-# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-# The last phase is to lower the computation loops down to VTA hardware
-# intrinsics by mapping the 2D convolution to tensor intrinsics,
-# and mapping the shift, and clipping computation to the vector ALU.
-
-# Apply tensorization over the batch tensor tile axis
-s[res_conv].tensorize(b_tns, env.gemm)
-
-# Add an ALU pragma over the shift and clipping operations
-s[res_shr].pragma(s[res_shr].op.axis[0], env.alu)
-s[res_min].pragma(s[res_min].op.axis[0], env.alu)
-s[res_max].pragma(s[res_max].op.axis[0], env.alu)
-
-# Let's look at the final lowered TVM schedule after lowering memory
-# loads/stores down to DMA copy intrinsics, and the computation down to
-# VTA compute intrinsics.
-print(vta.lower(s, [data, kernel, res], simple_mode=True))
-
-######################################################################
-# TVM Compilation and Verification
-# --------------------------------
-# After specifying the schedule, we can compile it into a TVM function.
-# We save the module so we can send it over RPC.
-# We run the function and verify it against a numpy implementation to
-# ensure correctness.
-
-# This library facilitates 2D convolution testing
-from tvm.topi.testing import conv2d_nchw_python
-
-# Compile the TVM module
-with vta.build_config(disabled_pass={"tir.CommonSubexprElimTIR"}):
- my_conv = vta.build(
- s, [data, kernel, res], tvm.target.Target("ext_dev", host=env.target_host), name="my_conv"
- )
-temp = utils.tempdir()
-my_conv.save(temp.relpath("conv2d.o"))
-remote.upload(temp.relpath("conv2d.o"))
-f = remote.load_module("conv2d.o")
-
-# Get the remote device context
-ctx = remote.ext_dev(0)
-
-# Initialize the data and kernel arrays randomly in the int range
-# of (-128, 128] in NCHW layout
-data_np = np.random.randint(-128, 128, size=(batch_size, in_channels, height, width)).astype(
- data.dtype
-)
-kernel_np = np.random.randint(
- -128, 128, size=(out_channels, in_channels, kernel_h, kernel_w)
-).astype(kernel.dtype)
-
-# Apply packing to the data and kernel arrays from a 2D NCHW
-# to a 4D NCHWnc packed layout
-data_packed = data_np.reshape(
- batch_size // env.BATCH, env.BATCH, in_channels // env.BLOCK_IN, env.BLOCK_IN, height, width
-).transpose((0, 2, 4, 5, 1, 3))
-
-kernel_packed = kernel_np.reshape(
- out_channels // env.BLOCK_OUT,
- env.BLOCK_OUT,
- in_channels // env.BLOCK_IN,
- env.BLOCK_IN,
- kernel_h,
- kernel_w,
-).transpose((0, 2, 4, 5, 1, 3))
-
-# Format the input/output arrays with tvm.nd.array to the DLPack standard
-data_nd = tvm.nd.array(data_packed, ctx)
-kernel_nd = tvm.nd.array(kernel_packed, ctx)
-res_nd = tvm.nd.array(np.zeros(output_shape).astype(res.dtype), ctx)
-
-# Clear stats
-if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
-
-# Invoke the module to perform the computation
-f(data_nd, kernel_nd, res_nd)
-
-# Verify against numpy implementation
-res_ref = conv2d_nchw_python(
- data_np.astype(env.acc_dtype),
- kernel_np.astype(env.acc_dtype),
- (stride_h, stride_w),
- (pad_h, pad_w),
-).astype(env.acc_dtype)
-res_ref = res_ref >> env.INP_WIDTH
-res_ref = np.clip(res_ref, 0, inp_max)
-res_ref = res_ref.astype(res.dtype)
-res_ref = res_ref.reshape(
- (
- batch_size // env.BATCH,
- env.BATCH,
- out_channels // env.BLOCK_OUT,
- env.BLOCK_OUT,
- fout_height,
- fout_width,
- )
-).transpose((0, 2, 4, 5, 1, 3))
-tvm.testing.assert_allclose(res_ref, res_nd.numpy())
-
-# Print stats
-if env.TARGET in ["sim", "tsim"]:
- sim_stats = simulator.stats()
- print("Execution statistics:")
- for k, v in sim_stats.items():
- print("\t{:<16}: {:>16}".format(k, v))
-
-print("Successful 2D convolution test!")
-
-######################################################################
-# Summary
-# -------
-# This tutorial demonstrates how TVM scheduling primitives can be used to
-# lower 2D convolution onto hardware accelerator intrinsics, making
-# use of hardware specific optimizations, such as latency hiding with
-# virtual threading.
-#
diff --git a/docs/_downloads/1ee0b869c5082223c5dfbb0fe4574252/matrix_multiply.ipynb b/docs/_downloads/1ee0b869c5082223c5dfbb0fe4574252/matrix_multiply.ipynb
deleted file mode 100644
index 48ff0e3b83..0000000000
--- a/docs/_downloads/1ee0b869c5082223c5dfbb0fe4574252/matrix_multiply.ipynb
+++ /dev/null
@@ -1,311 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n\n# Simple Matrix Multiply\n**Author**: [Thierry Moreau](https://homes.cs.washington.edu/~moreau/)\n\nIn this tutorial, we will build on top of the `vta-get-started` tutorial\nand introduce additional concepts required to implement matrix multiplication\non VTA with the TVM workflow.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## RPC Setup\nWe start by programming the Pynq's FPGA and building its RPC runtime\nas we did in the VTA introductory tutorial.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from __future__ import absolute_import, print_function\n\nimport os\nimport tvm\nfrom tvm import te\nimport vta\nimport numpy as np\nfrom tvm import rpc\nfrom tvm.contrib import utils\nfrom vta.testing import simulator\n\n# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file\nenv = vta.get_env()\n\n# We read the Pynq RPC host IP address and port number from the OS environment\nhost = os.environ.get(\"VTA_RPC_HOST\", \"192.168.2.99\")\nport = int(os.environ.g [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Computation Declaration\nIn this example we describe a simple matrix multiplication addition, which\nrequires multiple computation stages, as shown in the dataflow diagram below.\nFirst we describe the input tensors :code:`A` and :code:`B` that are living\nin main memory.\nSecond, we need to declare intermediate tensors :code:`A_buf` and\n:code:`B_buf`, which will live in VTA's on-chip buffers.\nHaving this extra computational stage allows us to explicitly\nstage cached reads [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Data Layout\nWe describe the placeholder tensors :code:`A`, and :code:`B` in a tiled data\nformat to match the data layout requirements imposed by the VTA tensor core.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>**Data Tiling**\n\n One source of complexity when targeting accelerators is to make sure\n that the data layout matches the layout imposed by the accelerator design.\n VTA is designed around a *tensor core* that performs, one matrix-matrix\n operation per cycle between an activation matrix and a weight matrix,\n adding the result matrix to an accumulator matrix, as shown in the\n figure below.\n\n .. image:: https://raw.gith [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>**Data Types**\n\n It's important to not only match the inner-tile\n dimension of VTA's tensor core, but also to match the specific data types\n expected by VTA.\n VTA for now only supports fixed point data types, which integer width is\n specified in the :code:`vta_config.json` file by :code:`INP_WIDTH` and\n :code:`WGT_WIDTH` for the activations and weights data types respectively.\n In addition, the accumulator data type [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Output channel factor m - total 16x16=256 output channels\nm = 16\n# Input channel factor n - total 16x16=256 input channels\nn = 16\n# Batch factor o (we use single batch inference)\no = 1\n# A placeholder tensor in tiled data format\nA = te.placeholder((o, n, env.BATCH, env.BLOCK_IN), name=\"A\", dtype=env.inp_dtype)\n# B placeholder tensor in tiled data format\nB = te.placeholder((m, n, env.BLOCK_OUT, env.BLOCK_IN), name=\"B\", dtype=env.wgt_dtype)\n# A copy buffer\nA_buf = [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Matrix Multiplication\nNow we're ready to describe the matrix multiplication result tensor :code:`C`,\nwith another compute operation.\nThe compute function takes the shape of the tensor, as well as a lambda\nfunction that describes the computation rule for each position of the tensor.\n\nIn order to implement matrix multiplication, the lambda function needs to\ninclude a reduction formula over the input channel dimension axes.\nTo create a reduction formula, we can declare [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Outer input feature reduction axis\nko = te.reduce_axis((0, n), name=\"ko\")\n# Inner input feature reduction axis\nki = te.reduce_axis((0, env.BLOCK_IN), name=\"ki\")\n# Describe the in-VTA matrix multiplication\nC_buf = te.compute(\n (o, m, env.BATCH, env.BLOCK_OUT),\n lambda bo, co, bi, ci: te.sum(\n A_buf[bo, ko, bi, ki].astype(env.acc_dtype) * B_buf[co, ko, ci, ki].astype(env.acc_dtype),\n axis=[ko, ki],\n ),\n name=\"C_buf\",\n)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Casting the Results\nAfter the computation is done, we'll need to send the results computed by VTA\nback to main memory.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>**Memory Store Restrictions**\n\n One specificity of VTA is that it only supports DRAM stores in the narrow\n :code:`env.inp_dtype` data type format.\n This lets us reduce the data footprint for memory transfers, but also lets\n us quantize the wide accumulator data type down to a data format that\n matches the input activation data type.\n This means that in the context of neural network inference, the outputs\n of a given [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Cast to output type, and send to main memory\nC = te.compute(\n (o, m, env.BATCH, env.BLOCK_OUT), lambda *i: C_buf(*i).astype(env.inp_dtype), name=\"C\"\n)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This concludes the computation declaration part of this tutorial.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Scheduling the Computation\nWhile the above lines describes the computation rule, we can obtain\n:code:`C` in many ways.\nTVM asks the user to provide an implementation of the computation called\n*schedule*.\n\nA schedule is a set of transformations to an original computation that\ntransforms the implementation of the computation without affecting\ncorrectness.\nThis simple VTA programming tutorial aims to demonstrate basic schedule\ntransformations that will map the original [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Default Schedule\nAfter we construct the schedule, by default the schedule computes\n:code:`C` in the following way:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Let's take a look at the generated schedule\ns = te.create_schedule(C.op)\nprint(tvm.lower(s, [A, B, C], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Although this schedule makes sense, it won't compile to VTA.\nIn order to obtain correct code generation, we need to apply scheduling\nprimitives and code annotation that will transform the schedule into\none that can be directly lowered onto VTA hardware intrinsics.\nThose include:\n\n - DMA copy operations which will take globally-scoped tensors and copy\n those into locally-scoped tensors.\n - Tensor operations that will perform the matrix multiplication.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Buffer Scopes\nFirst, we set the scope of the buffers to tell TVM that these buffers\nwill be living in the VTA's on-chip SRAM caches.\nBelow, we tell TVM that :code:`A_buf`, :code:`B_buf`, :code:`C_buf`\nwill respectively live in VTA's on-chip input, weight and accumulator\nmemory.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>**VTA's On-Chip SRAMs**\n\n VTA has three different memory scopes, each corresponding to different\n on-chip SRAM buffers.\n\n - :code:`env.inp_scope`: Input buffer, which is a read-only SRAM buffer\n that stores input matrices of shape :code:`(env.BATCH, env.BLOCK_IN)`\n of type :code:`env.inp_dtype`. The input buffer contains\n `2 ^ LOG_INP_BUFF_SIZE` matrix elements (as specified in the\n :code:`vta_config.js [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Set the intermediate tensor's scope to VTA's on-chip buffers\ns[A_buf].set_scope(env.inp_scope)\ns[B_buf].set_scope(env.wgt_scope)\ns[C_buf].set_scope(env.acc_scope)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### DMA Transfers\nWe need to schedule DMA transfers to move data living in DRAM to\nand from the VTA on-chip buffers.\nThis can be achieved using the :code:`compute_at` schedule primitive\nwhich nests the copying of the buffers into the computation loop\nthat performs the matrix multiplication.\n\nWe insert :code:`dma_copy` pragmas to indicate to the compiler\nthat the copy operations will be performed in bulk via DMA,\nwhich is common in hardware accelerators.\nFinally, we pri [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Move buffer copy into matrix multiply loop\ns[A_buf].compute_at(s[C_buf], ko)\ns[B_buf].compute_at(s[C_buf], ko)\n\n# Tag the buffer copies with the DMA pragma to insert a DMA transfer\ns[A_buf].pragma(s[A_buf].op.axis[0], env.dma_copy)\ns[B_buf].pragma(s[B_buf].op.axis[0], env.dma_copy)\ns[C].pragma(s[C].op.axis[0], env.dma_copy)\n\n# Let's take a look at the transformed schedule\nprint(tvm.lower(s, [A, B, C], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Tensorization\nThe last step of the schedule transformation consists in applying\n*tensorization* to our schedule.\nTensorization is analogous to vectorization, but extends the concept\nto a higher-dimensional unit of computation.\nConsequently, tensorization imposes data layout constraints as discussed\nearlier when declaring the data layout input placeholders.\nWe've already arranged our tensors in a tiled format, so the next thing\nwe need to perform is loop reordering to [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "s[C_buf].reorder(\n ko, s[C_buf].op.axis[0], s[C_buf].op.axis[1], s[C_buf].op.axis[2], s[C_buf].op.axis[3], ki\n)\ns[C_buf].tensorize(s[C_buf].op.axis[2], env.gemm)\n\n# Let's take a look at the finalized schedule\nprint(vta.lower(s, [A, B, C], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This concludes the scheduling portion of this tutorial.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## TVM Compilation\nAfter we have finished specifying the schedule, we can compile it\ninto a TVM function.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Build GEMM VTA kernel\nmy_gemm = vta.build(\n s, [A, B, C], tvm.target.Target(\"ext_dev\", host=env.target_host), name=\"my_gemm\"\n)\n\n# Write the compiled module into an object file.\ntemp = utils.tempdir()\nmy_gemm.save(temp.relpath(\"gemm.o\"))\n\n# Send the executable over RPC\nremote.upload(temp.relpath(\"gemm.o\"))\n\n# Load the compiled module\nf = remote.load_module(\"gemm.o\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Running the Function\nThe compiled TVM function uses a concise C API and can be invoked from\ncode language.\n\nTVM provides an array API in python to aid quick testing and prototyping.\nThe array API is based on [DLPack](https://github.com/dmlc/dlpack) standard.\n\n- We first create a remote context (for remote execution on the Pynq).\n- Then :code:`tvm.nd.array` formats the data accordingly.\n- :code:`f()` runs the actual computation.\n- :code:`numpy()` copies the result ar [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Get the remote device context\nctx = remote.ext_dev(0)\n\n# Initialize the A and B arrays randomly in the int range of (-128, 128]\nA_orig = np.random.randint(-128, 128, size=(o * env.BATCH, n * env.BLOCK_IN)).astype(A.dtype)\nB_orig = np.random.randint(-128, 128, size=(m * env.BLOCK_OUT, n * env.BLOCK_IN)).astype(B.dtype)\n\n# Apply packing to the A and B arrays from a 2D to a 4D packed layout\nA_packed = A_orig.reshape(o, env.BATCH, n, env.BLOCK_IN).transpose((0, 2, 1, 3))\n [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Verifying Correctness\nCompute the reference result with numpy and assert that the output of the\nmatrix multiplication indeed is correct\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Compute reference result with numpy\nC_ref = np.dot(A_orig.astype(env.acc_dtype), B_orig.T.astype(env.acc_dtype)).astype(C.dtype)\nC_ref = C_ref.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))\nnp.testing.assert_equal(C_ref, C_nd.numpy())\n\n# Print stats\nif env.TARGET in [\"sim\", \"tsim\"]:\n sim_stats = simulator.stats()\n print(\"Execution statistics:\")\n for k, v in sim_stats.items():\n print(\"\\t{:<16}: {:>16}\".format(k, v))\n\nprint(\ [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\nThis tutorial showcases the TVM workflow to implement a simple matrix\nmultiplication example on VTA.\nThe general workflow includes:\n\n- Programming the FPGA with the VTA bitstream over RPC.\n- Describing matrix multiplication via a series of computations.\n- Describing how we want to perform the computation using schedule primitives.\n- Compiling the function to the VTA target.\n- Running the compiled module and verifying it against a numpy implementation.\n\n\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.20"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/3b5e41b16a898b72d18127ebe2182c66/tensorize.ipynb b/docs/_downloads/3b5e41b16a898b72d18127ebe2182c66/tensorize.ipynb
index 6170ad8281..3c8efe013f 100644
--- a/docs/_downloads/3b5e41b16a898b72d18127ebe2182c66/tensorize.ipynb
+++ b/docs/_downloads/3b5e41b16a898b72d18127ebe2182c66/tensorize.ipynb
@@ -213,7 +213,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "## Summary\nThis tutorial demonstrates the usage of tensorize intrinsic in TVM.\nTensorize provides a way for users to get fully optimized schedule via micro-kernels.\nFor example, INT8 quantization on Intel CPUs uses tensorization\nto invoke AVX instruction directly.\nIt also enables TVM to compile to ASICs -\ncheckout `vta-index` for details.\nWe also demonstrates how to use inline assembly importing,\nwhich helps users inject asm easily into the schedule.\n\n\n"
+ "## Summary\nThis tutorial demonstrates the usage of tensorize intrinsic in TVM.\nTensorize provides a way for users to get fully optimized schedule via micro-kernels.\nFor example, INT8 quantization on Intel CPUs uses tensorization\nto invoke AVX instruction directly.\nWe also demonstrates how to use inline assembly importing,\nwhich helps users inject asm easily into the schedule.\n\n\n"
]
}
],
diff --git a/docs/_downloads/428c6201e29ce74e73c6b41eee589f62/tensorize.py b/docs/_downloads/428c6201e29ce74e73c6b41eee589f62/tensorize.py
index 63ba829903..8f7035511e 100644
--- a/docs/_downloads/428c6201e29ce74e73c6b41eee589f62/tensorize.py
+++ b/docs/_downloads/428c6201e29ce74e73c6b41eee589f62/tensorize.py
@@ -312,8 +312,6 @@ tvm.testing.assert_allclose(c.numpy(), np.dot(a, b.T), rtol=1e-3)
# Tensorize provides a way for users to get fully optimized schedule via micro-kernels.
# For example, INT8 quantization on Intel CPUs uses tensorization
# to invoke AVX instruction directly.
-# It also enables TVM to compile to ASICs -
-# checkout :ref:`vta-index` for details.
# We also demonstrates how to use inline assembly importing,
# which helps users inject asm easily into the schedule.
#
diff --git a/docs/_downloads/4d3f955a709b320db0d42740fead8ac1/matrix_multiply_opt.ipynb b/docs/_downloads/4d3f955a709b320db0d42740fead8ac1/matrix_multiply_opt.ipynb
deleted file mode 100644
index b5446a9e90..0000000000
--- a/docs/_downloads/4d3f955a709b320db0d42740fead8ac1/matrix_multiply_opt.ipynb
+++ /dev/null
@@ -1,176 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n\n# Matrix Multiply Blocking\n**Author**: [Thierry Moreau](https://homes.cs.washington.edu/~moreau/)\n\nThis tutorial provides an overview on how to use TVM to map matrix\nmultiplication efficiently on the VTA design.\nWe recommend covering the `basic-mat-mult` tutorial first.\n\nIn this tutorial, we will demonstrate TVM schedule optimizations to break large\nneural network operators down onto smaller blocks to achieve computation within\nlimited hardware accelerator resources.\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## RPC Setup\nWe start by programming the Pynq's FPGA and building its RPC runtime.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from __future__ import absolute_import, print_function\n\nimport os\nimport tvm\nfrom tvm import te\nimport vta\nimport numpy as np\nfrom tvm import rpc\nfrom tvm.contrib import utils\nfrom vta.testing import simulator\n\n# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file\nenv = vta.get_env()\n\n# We read the Pynq RPC host IP address and port number from the OS environment\nhost = os.environ.get(\"VTA_RPC_HOST\", \"192.168.2.99\")\nport = int(os.environ.g [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Computation Declaration\nAs a first step, we need to describe our matrix multiplication computation.\nWe define the matrix multiplication as the computation one would find in a\nfully connected layer, defined by its batch size, input channels, and output\nchannels.\nThese have to be integer multiples of the VTA tensor shape:\n:code:`BATCH`, :code:`BLOCK_IN`, and :code:`BLOCK_OUT` respectively.\n\nWe've added extra operators to the matrix multiplication that apply\nshifting an [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Fully connected layer dimensions: 1024 x 1024\nbatch_size = 1\nin_channels = 1024\nout_channels = 1024\nassert batch_size % env.BATCH == 0\nassert in_channels % env.BLOCK_IN == 0\nassert out_channels % env.BLOCK_OUT == 0\n\n# Let's derive the tiled input tensor shapes\ndata_shape = (batch_size // env.BATCH, in_channels // env.BLOCK_IN, env.BATCH, env.BLOCK_IN)\nweight_shape = (\n out_channels // env.BLOCK_OUT,\n in_channels // env.BLOCK_IN,\n env.BLOCK_OUT,\n env.B [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Scheduling the Computation\nWe'll look at a set of schedule transformations necessary to map the\nmatrix multiplications onto VTA in an efficient fashion.\nThose include:\n\n- Computation blocking\n- Lowering to VTA hardware intrinsics\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Create TVM schedule\ns = te.create_schedule(res.op)\n# Let's look at the default TVM schedule\nprint(tvm.lower(s, [data, weight, res], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Blocking the Computation\nThe matrix multiplication is by default too large for activations or weights\nto fit on VTA's on-chip buffers all at once.\nWe block the (1, 1024) by (1024, 1024) matrix multiplication into\nsmaller (1, 256) by (256, 256) matrix multiplications so the intermediate\ntensors can fit on the accelerator's on-chip SRAM.\nThis approach is similar to blocking techniques applied to CPUs and GPUs in\norder to increase cache hit rate.\n\nWe perform blocking a [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Let's define tiling sizes (expressed in multiples of VTA tensor shape size)\nb_block = 1 // env.BATCH\ni_block = 256 // env.BLOCK_IN\no_block = 256 // env.BLOCK_OUT\n\n# Tile the output tensor along the batch and output channel dimensions\n# (since by default we are doing single batch inference, the split along\n# the batch dimension has no effect)\nb, oc, b_tns, oc_tns = s[res].op.axis\nb_out, b_inn = s[res].split(b, b_block)\noc_out, oc_inn = s[res].split(oc, o_block)\ns[re [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Lowering Copies to DMA Transfers\nNext we set the buffer scopes to the corresponding on-chip VTA SRAM buffers.\nWe move the load loops into the matrix multiply computation loop to stage\nmemory loads such that they fit in the on-chip SRAM buffers.\nFinally we annotate the load/store loop outer axes with the DMA copy pragma\nto perform bulk memory transfers on VTA.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Set scope of SRAM buffers\ns[data_buf].set_scope(env.inp_scope)\ns[weight_buf].set_scope(env.wgt_scope)\ns[res_gemm].set_scope(env.acc_scope)\ns[res_shr].set_scope(env.acc_scope)\ns[res_min].set_scope(env.acc_scope)\ns[res_max].set_scope(env.acc_scope)\n\n# Block data and weight cache reads\ns[data_buf].compute_at(s[res_gemm], ic_out)\ns[weight_buf].compute_at(s[res_gemm], ic_out)\n\n# Use DMA copy pragma on DRAM->SRAM operations\ns[data_buf].pragma(s[data_buf].op.axis[0], env [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Lowering Computation to VTA Compute Intrinsics\nThe last phase is to lower the computation loops down to VTA hardware\nintrinsics by mapping the matrix multiplication to tensor intrinsics,\nand mapping the shift, and clipping computation to the vector ALU.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Apply tensorization over the batch tensor tile axis\ns[res_gemm].tensorize(b_tns, env.gemm)\n\n# Add an ALU pragma over the shift and clipping operations\ns[res_shr].pragma(s[res_shr].op.axis[0], env.alu)\ns[res_min].pragma(s[res_min].op.axis[0], env.alu)\ns[res_max].pragma(s[res_max].op.axis[0], env.alu)\n\n# Let's look at the final lowered TVM schedule after lowering memory\n# loads/stores down to DMA copy intrinsics, and the computation down to\n# VTA compute intrinsics.\np [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## TVM Compilation and Verification\nAfter specifying the schedule, we can compile it into a TVM function.\nWe save the module so we can send it over RPC.\nWe run the function and verify it against a numpy implementation to\nensure correctness.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Compile the TVM module\nmy_gemm = vta.build(\n s, [data, weight, res], tvm.target.Target(\"ext_dev\", host=env.target_host), name=\"my_gemm\"\n)\ntemp = utils.tempdir()\nmy_gemm.save(temp.relpath(\"gemm.o\"))\nremote.upload(temp.relpath(\"gemm.o\"))\nf = remote.load_module(\"gemm.o\")\n\n# Get the remote device context\nctx = remote.ext_dev(0)\n\n# Initialize the data and weight arrays randomly in the int range of (-128, 128]\ndata_np = np.random.randint(-128, 128, size=(ba [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\nThis tutorial demonstrates how TVM scheduling primitives can achieve\ncomputation blocking for a matrix multiplication example.\nThis allows us to map arbitrarily large computation onto limited\nhardware accelerator resources.\n\n\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.20"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/65b9451c8de050d7cd9da2fe5a49acc6/deploy_detection.py b/docs/_downloads/65b9451c8de050d7cd9da2fe5a49acc6/deploy_detection.py
deleted file mode 100644
index 0c430e8f9c..0000000000
--- a/docs/_downloads/65b9451c8de050d7cd9da2fe5a49acc6/deploy_detection.py
+++ /dev/null
@@ -1,322 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-Deploy Pretrained Vision Detection Model from Darknet on VTA
-============================================================
-**Author**: `Hua Jiang <https://github.com/huajsj>`_
-
-This tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny
-inference onto the VTA accelerator design to perform Image detection tasks.
-It showcases Relay as a front end compiler that can perform quantization (VTA
-only supports int8/32 inference) as well as graph packing (in order to enable
-tensorization in the core) to massage the compute graph for the hardware target.
-"""
-
-######################################################################
-# Install dependencies
-# --------------------
-# To use the autotvm package in tvm, we need to install some extra dependencies.
-# (change "3" to "2" if you use python2):
-#
-# .. code-block:: bash
-#
-# pip3 install "Pillow<7"
-#
-# YOLO-V3-tiny Model with Darknet parsing have dependancy with CFFI and CV2 library,
-# we need to install CFFI and CV2 before executing this script.
-#
-# .. code-block:: bash
-#
-# pip3 install cffi
-# pip3 install opencv-python
-#
-# Now return to the python code. Import packages.
-
-from __future__ import absolute_import, print_function
-
-import sys
-import os
-import time
-import matplotlib.pyplot as plt
-import numpy as np
-import tvm
-import vta
-from tvm import rpc, autotvm, relay
-from tvm.relay.testing import yolo_detection, darknet
-from tvm.relay.testing.darknet import __darknetffi__
-from tvm.contrib import graph_executor, utils
-from tvm.contrib.download import download_testdata
-from vta.testing import simulator
-from vta.top import graph_pack
-
-# Make sure that TVM was compiled with RPC=1
-assert tvm.runtime.enabled("rpc")
-
-##############################################################################
-# Download yolo net configure file, weight file, darknet library file based on
-# Model Name
-# ----------------------------------------------------------------------------
-MODEL_NAME = "yolov3-tiny"
-REPO_URL = "https://github.com/dmlc/web-data/blob/main/darknet/"
-
-cfg_path = download_testdata(
- "https://github.com/pjreddie/darknet/blob/master/cfg/" + MODEL_NAME + ".cfg" + "?raw=true",
- MODEL_NAME + ".cfg",
- module="darknet",
-)
-weights_path = download_testdata(
- "https://pjreddie.com/media/files/" + MODEL_NAME + ".weights" + "?raw=true",
- MODEL_NAME + ".weights",
- module="darknet",
-)
-
-if sys.platform in ["linux", "linux2"]:
- darknet_lib_path = download_testdata(
- REPO_URL + "lib/" + "libdarknet2.0.so" + "?raw=true", "libdarknet2.0.so", module="darknet"
- )
-elif sys.platform == "darwin":
- darknet_lib_path = download_testdata(
- REPO_URL + "lib_osx/" + "libdarknet_mac2.0.so" + "?raw=true",
- "libdarknet_mac2.0.so",
- module="darknet",
- )
-else:
- raise NotImplementedError("Darknet lib is not supported on {} platform".format(sys.platform))
-
-##################################################
-# Download yolo categories and illustration front.
-# ------------------------------------------------
-coco_path = download_testdata(
- REPO_URL + "data/" + "coco.names" + "?raw=true", "coco.names", module="data"
-)
-font_path = download_testdata(
- REPO_URL + "data/" + "arial.ttf" + "?raw=true", "arial.ttf", module="data"
-)
-with open(coco_path) as f:
- content = f.readlines()
-names = [x.strip() for x in content]
-
-########################################
-# Define the platform and model targets.
-# --------------------------------------
-# Execute on CPU vs. VTA, and define the model.
-
-# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
-env = vta.get_env()
-# Set ``device=arm_cpu`` to run inference on the CPU
-# or ``device=vta`` to run inference on the FPGA.
-device = "vta"
-target = env.target if device == "vta" else env.target_vta_cpu
-
-pack_dict = {
- "yolov3-tiny": ["nn.max_pool2d", "cast", 4, 186],
-}
-
-# Name of Darknet model to compile
-# The ``start_pack`` and ``stop_pack`` labels indicate where
-# to start and end the graph packing relay pass: in other words
-# where to start and finish offloading to VTA.
-# the number 4 indicate the ``start_pack`` index is 4, the
-# number 186 indicate the ``stop_pack index`` is 186, by using
-# name and index number, here we can located to correct place
-# where to start/end when there are multiple ``nn.max_pool2d``
-# or ``cast``, print(mod.astext(show_meta_data=False)) can help
-# to find operator name and index information.
-assert MODEL_NAME in pack_dict
-
-#############################
-# Obtain an execution remote.
-# ---------------------------
-# When target is 'pynq' or other FPGA backend, reconfigure FPGA and runtime.
-# Otherwise, if target is 'sim', execute locally.
-
-if env.TARGET not in ["sim", "tsim"]:
- # Get remote from tracker node if environment variable is set.
- # To set up the tracker, you'll need to follow the "Auto-tuning
- # a convolutional network for VTA" tutorial.
- tracker_host = os.environ.get("TVM_TRACKER_HOST", None)
- tracker_port = os.environ.get("TVM_TRACKER_PORT", None)
- # Otherwise if you have a device you want to program directly from
- # the host, make sure you've set the variables below to the IP of
- # your board.
- device_host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- device_port = os.environ.get("VTA_RPC_PORT", "9091")
- if not tracker_host or not tracker_port:
- remote = rpc.connect(device_host, int(device_port))
- else:
- remote = autotvm.measure.request_remote(
- env.TARGET, tracker_host, int(tracker_port), timeout=10000
- )
- # Reconfigure the JIT runtime and FPGA.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- reconfig_start = time.time()
- vta.reconfig_runtime(remote)
- vta.program_fpga(remote, bitstream=None)
- reconfig_time = time.time() - reconfig_start
- print("Reconfigured FPGA and RPC runtime in {0:.2f}s!".format(reconfig_time))
-
-# In simulation mode, host the RPC server locally.
-else:
- remote = rpc.LocalSession()
-
-# Get execution context from remote
-ctx = remote.ext_dev(0) if device == "vta" else remote.cpu(0)
-
-#####################################
-# Build the inference graph executor.
-# -----------------------------------
-# Using Darknet library load downloaded vision model and compile with Relay.
-# The compilation steps are:
-#
-# 1. Front end translation from Darknet into Relay module.
-# 2. Apply 8-bit quantization: here we skip the first conv layer,
-# and dense layer which will both be executed in fp32 on the CPU.
-# 3. Perform graph packing to alter the data layout for tensorization.
-# 4. Perform constant folding to reduce number of operators (e.g. eliminate batch norm multiply).
-# 5. Perform relay build to object file.
-# 6. Load the object file onto remote (FPGA device).
-# 7. Generate graph executor, `m`.
-
-# Load pre-configured AutoTVM schedules
-with autotvm.tophub.context(target):
- net = __darknetffi__.dlopen(darknet_lib_path).load_network(
- cfg_path.encode("utf-8"), weights_path.encode("utf-8"), 0
- )
- dshape = (env.BATCH, net.c, net.h, net.w)
- dtype = "float32"
-
- # Measure build start time
- build_start = time.time()
-
- # Start front end compilation
- mod, params = relay.frontend.from_darknet(net, dtype=dtype, shape=dshape)
-
- if target.device_name == "vta":
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with tvm.transform.PassContext(opt_level=3):
- with relay.quantize.qconfig(
- global_scale=23.0,
- skip_conv_layers=[0],
- store_lowbit_output=True,
- round_for_shift=True,
- ):
- mod = relay.quantize.quantize(mod, params=params)
- # Perform graph packing and constant folding for VTA target
- mod = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=pack_dict[MODEL_NAME][0],
- stop_name=pack_dict[MODEL_NAME][1],
- start_name_idx=pack_dict[MODEL_NAME][2],
- stop_name_idx=pack_dict[MODEL_NAME][3],
- )
- else:
- mod = mod["main"]
-
- # Compile Relay program with AlterOpLayout disabled
- with vta.build_config(disabled_pass={"AlterOpLayout", "tir.CommonSubexprElimTIR"}):
- lib = relay.build(
- mod, target=tvm.target.Target(target, host=env.target_host), params=params
- )
-
- # Measure Relay build time
- build_time = time.time() - build_start
- print(MODEL_NAME + " inference graph built in {0:.2f}s!".format(build_time))
-
- # Send the inference library over to the remote RPC server
- temp = utils.tempdir()
- lib.export_library(temp.relpath("graphlib.tar"))
- remote.upload(temp.relpath("graphlib.tar"))
- lib = remote.load_module("graphlib.tar")
-
- # Graph executor
- m = graph_executor.GraphModule(lib["default"](ctx))
-
-####################################
-# Perform image detection inference.
-# ----------------------------------
-# We run detect on an downloaded image
-# Download test image
-[neth, netw] = dshape[2:]
-test_image = "person.jpg"
-img_url = REPO_URL + "data/" + test_image + "?raw=true"
-img_path = download_testdata(img_url, test_image, "data")
-data = darknet.load_image(img_path, neth, netw).transpose(1, 2, 0)
-
-# Prepare test image for inference
-plt.imshow(data)
-plt.show()
-data = data.transpose((2, 0, 1))
-data = data[np.newaxis, :]
-data = np.repeat(data, env.BATCH, axis=0)
-
-# Set the network parameters and inputs
-m.set_input("data", data)
-
-# Perform inference and gather execution statistics
-# More on: :py:method:`tvm.runtime.Module.time_evaluator`
-num = 4 # number of times we run module for a single measurement
-rep = 3 # number of measurements (we derive std dev from this)
-timer = m.module.time_evaluator("run", ctx, number=num, repeat=rep)
-
-if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
- timer()
- sim_stats = simulator.stats()
- print("\nExecution statistics:")
- for k, v in sim_stats.items():
- # Since we execute the workload many times, we need to normalize stats
- # Note that there is always one warm up run
- # Therefore we divide the overall stats by (num * rep + 1)
- print("\t{:<16}: {:>16}".format(k, v // (num * rep + 1)))
-else:
- tcost = timer()
- std = np.std(tcost.results) * 1000
- mean = tcost.mean * 1000
- print("\nPerformed inference in %.2fms (std = %.2f) for %d samples" % (mean, std, env.BATCH))
- print("Average per sample inference time: %.2fms" % (mean / env.BATCH))
-
-# Get detection results from out
-thresh = 0.5
-nms_thresh = 0.45
-tvm_out = []
-for i in range(2):
- layer_out = {}
- layer_out["type"] = "Yolo"
- # Get the yolo layer attributes (n, out_c, out_h, out_w, classes, total)
- layer_attr = m.get_output(i * 4 + 3).numpy()
- layer_out["biases"] = m.get_output(i * 4 + 2).numpy()
- layer_out["mask"] = m.get_output(i * 4 + 1).numpy()
- out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
- layer_out["output"] = m.get_output(i * 4).numpy().reshape(out_shape)
- layer_out["classes"] = layer_attr[4]
- tvm_out.append(layer_out)
- thresh = 0.560
-
-# Show detection results
-img = darknet.load_image_color(img_path)
-_, im_h, im_w = img.shape
-dets = yolo_detection.fill_network_boxes((netw, neth), (im_w, im_h), thresh, 1, tvm_out)
-last_layer = net.layers[net.n - 1]
-yolo_detection.do_nms_sort(dets, last_layer.classes, nms_thresh)
-yolo_detection.draw_detections(font_path, img, dets, thresh, names, last_layer.classes)
-plt.imshow(img.transpose(1, 2, 0))
-plt.show()
diff --git a/docs/_downloads/66e1a42229aae7ed49ac268f520e6727/deploy_detection.ipynb b/docs/_downloads/66e1a42229aae7ed49ac268f520e6727/deploy_detection.ipynb
deleted file mode 100644
index c35072c265..0000000000
--- a/docs/_downloads/66e1a42229aae7ed49ac268f520e6727/deploy_detection.ipynb
+++ /dev/null
@@ -1,169 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# Deploy Pretrained Vision Detection Model from Darknet on VTA\n**Author**: [Hua Jiang](https://github.com/huajsj)\n\nThis tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny\ninference onto the VTA accelerator design to perform Image detection tasks.\nIt showcases Relay as a front end compiler that can perform quantization (VTA\nonly supports int8/32 inference) as well as graph packing (in order to enable\ntensorization in the core) to massage the compute [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Install dependencies\nTo use the autotvm package in tvm, we need to install some extra dependencies.\n(change \"3\" to \"2\" if you use python2):\n\n```bash\npip3 install \"Pillow<7\"\n```\nYOLO-V3-tiny Model with Darknet parsing have dependancy with CFFI and CV2 library,\nwe need to install CFFI and CV2 before executing this script.\n\n```bash\npip3 install cffi\npip3 install opencv-python\n```\nNow return to the python code. Import packages.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from __future__ import absolute_import, print_function\n\nimport sys\nimport os\nimport time\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport tvm\nimport vta\nfrom tvm import rpc, autotvm, relay\nfrom tvm.relay.testing import yolo_detection, darknet\nfrom tvm.relay.testing.darknet import __darknetffi__\nfrom tvm.contrib import graph_executor, utils\nfrom tvm.contrib.download import download_testdata\nfrom vta.testing import simulator\nfrom vta.top import graph_pack\n [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Download yolo net configure file, weight file, darknet library file based on\nModel Name\n----------------------------------------------------------------------------\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "MODEL_NAME = \"yolov3-tiny\"\nREPO_URL = \"https://github.com/dmlc/web-data/blob/main/darknet/\"\n\ncfg_path = download_testdata(\n \"https://github.com/pjreddie/darknet/blob/master/cfg/\" + MODEL_NAME + \".cfg\" + \"?raw=true\",\n MODEL_NAME + \".cfg\",\n module=\"darknet\",\n)\nweights_path = download_testdata(\n \"https://pjreddie.com/media/files/\" + MODEL_NAME + \".weights\" + \"?raw=true\",\n MODEL_NAME + \".weights\",\n module=\"darknet\",\n)\n\nif sys.p [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Download yolo categories and illustration front.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "coco_path = download_testdata(\n REPO_URL + \"data/\" + \"coco.names\" + \"?raw=true\", \"coco.names\", module=\"data\"\n)\nfont_path = download_testdata(\n REPO_URL + \"data/\" + \"arial.ttf\" + \"?raw=true\", \"arial.ttf\", module=\"data\"\n)\nwith open(coco_path) as f:\n content = f.readlines()\nnames = [x.strip() for x in content]"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Define the platform and model targets.\nExecute on CPU vs. VTA, and define the model.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file\nenv = vta.get_env()\n# Set ``device=arm_cpu`` to run inference on the CPU\n# or ``device=vta`` to run inference on the FPGA.\ndevice = \"vta\"\ntarget = env.target if device == \"vta\" else env.target_vta_cpu\n\npack_dict = {\n \"yolov3-tiny\": [\"nn.max_pool2d\", \"cast\", 4, 186],\n}\n\n# Name of Darknet model to compile\n# The ``start_pack`` and ``stop_pack`` labels indicate where\n# to start and e [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Obtain an execution remote.\nWhen target is 'pynq' or other FPGA backend, reconfigure FPGA and runtime.\nOtherwise, if target is 'sim', execute locally.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "if env.TARGET not in [\"sim\", \"tsim\"]:\n # Get remote from tracker node if environment variable is set.\n # To set up the tracker, you'll need to follow the \"Auto-tuning\n # a convolutional network for VTA\" tutorial.\n tracker_host = os.environ.get(\"TVM_TRACKER_HOST\", None)\n tracker_port = os.environ.get(\"TVM_TRACKER_PORT\", None)\n # Otherwise if you have a device you want to program directly from\n # the host, make sure you've set the variables be [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Build the inference graph executor.\nUsing Darknet library load downloaded vision model and compile with Relay.\nThe compilation steps are:\n\n1. Front end translation from Darknet into Relay module.\n2. Apply 8-bit quantization: here we skip the first conv layer,\n and dense layer which will both be executed in fp32 on the CPU.\n3. Perform graph packing to alter the data layout for tensorization.\n4. Perform constant folding to reduce number of operators (e.g. eliminate ba [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Load pre-configured AutoTVM schedules\nwith autotvm.tophub.context(target):\n net = __darknetffi__.dlopen(darknet_lib_path).load_network(\n cfg_path.encode(\"utf-8\"), weights_path.encode(\"utf-8\"), 0\n )\n dshape = (env.BATCH, net.c, net.h, net.w)\n dtype = \"float32\"\n\n # Measure build start time\n build_start = time.time()\n\n # Start front end compilation\n mod, params = relay.frontend.from_darknet(net, dtype=dtype, shape=dshape)\n\n if [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Perform image detection inference.\nWe run detect on an downloaded image\nDownload test image\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "[neth, netw] = dshape[2:]\ntest_image = \"person.jpg\"\nimg_url = REPO_URL + \"data/\" + test_image + \"?raw=true\"\nimg_path = download_testdata(img_url, test_image, \"data\")\ndata = darknet.load_image(img_path, neth, netw).transpose(1, 2, 0)\n\n# Prepare test image for inference\nplt.imshow(data)\nplt.show()\ndata = data.transpose((2, 0, 1))\ndata = data[np.newaxis, :]\ndata = np.repeat(data, env.BATCH, axis=0)\n\n# Set the network parameters and inputs\nm.set_input(\"data\", [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.20"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/822e9d945c0bbf1cf23fc4f53c1b7906/matrix_multiply_opt.py b/docs/_downloads/822e9d945c0bbf1cf23fc4f53c1b7906/matrix_multiply_opt.py
deleted file mode 100644
index b470475b16..0000000000
--- a/docs/_downloads/822e9d945c0bbf1cf23fc4f53c1b7906/matrix_multiply_opt.py
+++ /dev/null
@@ -1,374 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-.. _vta-mat-mult-opt:
-
-Matrix Multiply Blocking
-========================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This tutorial provides an overview on how to use TVM to map matrix
-multiplication efficiently on the VTA design.
-We recommend covering the :ref:`basic-mat-mult` tutorial first.
-
-In this tutorial, we will demonstrate TVM schedule optimizations to break large
-neural network operators down onto smaller blocks to achieve computation within
-limited hardware accelerator resources.
-"""
-
-######################################################################
-# RPC Setup
-# ---------
-# We start by programming the Pynq's FPGA and building its RPC runtime.
-
-from __future__ import absolute_import, print_function
-
-import os
-import tvm
-from tvm import te
-import vta
-import numpy as np
-from tvm import rpc
-from tvm.contrib import utils
-from vta.testing import simulator
-
-# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
-env = vta.get_env()
-
-# We read the Pynq RPC host IP address and port number from the OS environment
-host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
-port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
-# We configure both the bitstream and the runtime system on the Pynq
-# to match the VTA configuration specified by the vta_config.json file.
-if env.TARGET == "pynq":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
-# In simulation mode, host the RPC server locally.
-elif env.TARGET in ["sim", "tsim"]:
- remote = rpc.LocalSession()
-
-######################################################################
-# Computation Declaration
-# -----------------------
-# As a first step, we need to describe our matrix multiplication computation.
-# We define the matrix multiplication as the computation one would find in a
-# fully connected layer, defined by its batch size, input channels, and output
-# channels.
-# These have to be integer multiples of the VTA tensor shape:
-# :code:`BATCH`, :code:`BLOCK_IN`, and :code:`BLOCK_OUT` respectively.
-#
-# We've added extra operators to the matrix multiplication that apply
-# shifting and clipping to the output in order to mimic a fixed-point
-# matrix multiplication followed by a rectified linear activation.
-# We describe the TVM dataflow graph of the fully connected layer below:
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/fc_dataflow.png
-# :align: center
-#
-# This computation is intentionally too large to fit onto VTA's on-chip
-# buffers all at once. Therefore in the scheduling phase we'll
-# rely on computation blocking strategies to break the computation down into
-# manageable chunks.
-
-# Fully connected layer dimensions: 1024 x 1024
-batch_size = 1
-in_channels = 1024
-out_channels = 1024
-assert batch_size % env.BATCH == 0
-assert in_channels % env.BLOCK_IN == 0
-assert out_channels % env.BLOCK_OUT == 0
-
-# Let's derive the tiled input tensor shapes
-data_shape = (batch_size // env.BATCH, in_channels // env.BLOCK_IN, env.BATCH, env.BLOCK_IN)
-weight_shape = (
- out_channels // env.BLOCK_OUT,
- in_channels // env.BLOCK_IN,
- env.BLOCK_OUT,
- env.BLOCK_IN,
-)
-output_shape = (batch_size // env.BATCH, out_channels // env.BLOCK_OUT, env.BATCH, env.BLOCK_OUT)
-num_ops = in_channels * out_channels * batch_size * 2
-
-# Reduction axes
-ic = te.reduce_axis((0, in_channels // env.BLOCK_IN), name="ic")
-ic_tns = te.reduce_axis((0, env.BLOCK_IN), name="ic_tns")
-
-# Input placeholder tensors
-data = te.placeholder(data_shape, name="data", dtype=env.inp_dtype)
-weight = te.placeholder(weight_shape, name="weight", dtype=env.wgt_dtype)
-
-# Copy buffers
-data_buf = te.compute(data_shape, lambda *i: data(*i), "data_buf")
-weight_buf = te.compute(weight_shape, lambda *i: weight(*i), "weight_buf")
-
-# Declare matrix multiply computation
-res_gemm = te.compute(
- output_shape,
- lambda bo, co, bi, ci: te.sum(
- data_buf[bo, ic, bi, ic_tns].astype(env.acc_dtype)
- * weight_buf[co, ic, ci, ic_tns].astype(env.acc_dtype),
- axis=[ic, ic_tns],
- ),
- name="res_gem",
-)
-
-# Add shift stage for fix-point normalization
-res_shr = te.compute(output_shape, lambda *i: res_gemm(*i) >> env.INP_WIDTH, name="res_shr")
-
-# Apply clipping between (0, input max value)
-inp_max = (1 << (env.INP_WIDTH - 1)) - 1
-res_max = te.compute(output_shape, lambda *i: tvm.te.max(res_shr(*i), 0), "res_max")
-res_min = te.compute(output_shape, lambda *i: tvm.te.min(res_max(*i), inp_max), "res_min")
-
-# Apply typecast to input data type before sending results back
-res = te.compute(output_shape, lambda *i: res_min(*i).astype(env.inp_dtype), name="res")
-
-######################################################################
-# Scheduling the Computation
-# --------------------------
-# We'll look at a set of schedule transformations necessary to map the
-# matrix multiplications onto VTA in an efficient fashion.
-# Those include:
-#
-# - Computation blocking
-# - Lowering to VTA hardware intrinsics
-
-
-# Create TVM schedule
-s = te.create_schedule(res.op)
-# Let's look at the default TVM schedule
-print(tvm.lower(s, [data, weight, res], simple_mode=True))
-
-######################################################################
-# Blocking the Computation
-# ~~~~~~~~~~~~~~~~~~~~~~~~
-# The matrix multiplication is by default too large for activations or weights
-# to fit on VTA's on-chip buffers all at once.
-# We block the (1, 1024) by (1024, 1024) matrix multiplication into
-# smaller (1, 256) by (256, 256) matrix multiplications so the intermediate
-# tensors can fit on the accelerator's on-chip SRAM.
-# This approach is similar to blocking techniques applied to CPUs and GPUs in
-# order to increase cache hit rate.
-#
-# We perform blocking along each axes (the batch axis being untouched since
-# we are performing singe-batch inference).
-# We also leave the inner-most tensorization axes as-is in order to allow
-# TVM to pattern-match tensorization.
-# We show the outcome of blocking on the computation schedule in the diagram
-# below:
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/blocking.png
-# :align: center
-# :width: 480px
-#
-# .. note::
-#
-# The code after loop splitting and reordering is equivalent to the following
-# pseudo-code. We ignore the batch axis since we are only performing single-batch
-# inference in this example:
-#
-# .. code-block:: c
-#
-# for (int oc_out = 0; oc_out < 4; ++oc_out) {
-# // Initialization loop
-# for (int oc_inn = 0; oc_inn < 16; ++oc_inn) {
-# for (int oc_tns = 0; oc_tns < 16; ++oc_tns) {
-# int j = (oc_out * 16 + oc_inn) * 16 + oc_tns;
-# C[0][j] = 0;
-# }
-# }
-# for (int ic_out = 0; ic_out < 4; ++ic_out) {
-# // Block loop
-# for (int oc_inn = 0; oc_inn < 16; ++oc_inn) {
-# for (int ic_inn = 0; ic_inn < 16; ++ic_inn) {
-# // Tensorization loop
-# for (int oc_tns = 0; oc_tns < 16; ++oc_tns) {
-# for (int ic_tns = 0; ic_tns < 16; ++ic_tns) {
-# int i = (ic_out * 16 + ic_inn) * 16 + ic_tns;
-# int j = (oc_out * 16 + oc_inn) * 16 + oc_tns;
-# C[0][i] = C[0][i] + A[0][i] * B[j][i];
-# }
-# }
-# }
-# }
-# }
-# }
-# }
-
-# Let's define tiling sizes (expressed in multiples of VTA tensor shape size)
-b_block = 1 // env.BATCH
-i_block = 256 // env.BLOCK_IN
-o_block = 256 // env.BLOCK_OUT
-
-# Tile the output tensor along the batch and output channel dimensions
-# (since by default we are doing single batch inference, the split along
-# the batch dimension has no effect)
-b, oc, b_tns, oc_tns = s[res].op.axis
-b_out, b_inn = s[res].split(b, b_block)
-oc_out, oc_inn = s[res].split(oc, o_block)
-s[res].reorder(b_out, oc_out, b_inn, oc_inn)
-
-# Move intermediate computation into each output compute tile
-s[res_gemm].compute_at(s[res], oc_out)
-s[res_shr].compute_at(s[res], oc_out)
-s[res_max].compute_at(s[res], oc_out)
-s[res_min].compute_at(s[res], oc_out)
-
-# Apply additional loop split along reduction axis (input channel)
-b_inn, oc_inn, b_tns, oc_tns = s[res_gemm].op.axis
-ic_out, ic_inn = s[res_gemm].split(ic, i_block)
-
-# Reorder axes. We move the ic_out axis all the way out of the GEMM
-# loop to block along the reduction axis
-s[res_gemm].reorder(ic_out, b_inn, oc_inn, ic_inn, b_tns, oc_tns, ic_tns)
-
-# Let's look at the current TVM schedule after blocking
-print(tvm.lower(s, [data, weight, res], simple_mode=True))
-
-######################################################################
-# Lowering Copies to DMA Transfers
-# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-# Next we set the buffer scopes to the corresponding on-chip VTA SRAM buffers.
-# We move the load loops into the matrix multiply computation loop to stage
-# memory loads such that they fit in the on-chip SRAM buffers.
-# Finally we annotate the load/store loop outer axes with the DMA copy pragma
-# to perform bulk memory transfers on VTA.
-
-# Set scope of SRAM buffers
-s[data_buf].set_scope(env.inp_scope)
-s[weight_buf].set_scope(env.wgt_scope)
-s[res_gemm].set_scope(env.acc_scope)
-s[res_shr].set_scope(env.acc_scope)
-s[res_min].set_scope(env.acc_scope)
-s[res_max].set_scope(env.acc_scope)
-
-# Block data and weight cache reads
-s[data_buf].compute_at(s[res_gemm], ic_out)
-s[weight_buf].compute_at(s[res_gemm], ic_out)
-
-# Use DMA copy pragma on DRAM->SRAM operations
-s[data_buf].pragma(s[data_buf].op.axis[0], env.dma_copy)
-s[weight_buf].pragma(s[weight_buf].op.axis[0], env.dma_copy)
-
-# Use DMA copy pragma on SRAM->DRAM operation
-# (this implies that these copies should be performed along b_inn,
-# or result axis 2)
-s[res].pragma(s[res].op.axis[2], env.dma_copy)
-
-######################################################################
-# Lowering Computation to VTA Compute Intrinsics
-# ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-# The last phase is to lower the computation loops down to VTA hardware
-# intrinsics by mapping the matrix multiplication to tensor intrinsics,
-# and mapping the shift, and clipping computation to the vector ALU.
-
-# Apply tensorization over the batch tensor tile axis
-s[res_gemm].tensorize(b_tns, env.gemm)
-
-# Add an ALU pragma over the shift and clipping operations
-s[res_shr].pragma(s[res_shr].op.axis[0], env.alu)
-s[res_min].pragma(s[res_min].op.axis[0], env.alu)
-s[res_max].pragma(s[res_max].op.axis[0], env.alu)
-
-# Let's look at the final lowered TVM schedule after lowering memory
-# loads/stores down to DMA copy intrinsics, and the computation down to
-# VTA compute intrinsics.
-print(vta.lower(s, [data, weight, res], simple_mode=True))
-
-######################################################################
-# TVM Compilation and Verification
-# --------------------------------
-# After specifying the schedule, we can compile it into a TVM function.
-# We save the module so we can send it over RPC.
-# We run the function and verify it against a numpy implementation to
-# ensure correctness.
-
-# Compile the TVM module
-my_gemm = vta.build(
- s, [data, weight, res], tvm.target.Target("ext_dev", host=env.target_host), name="my_gemm"
-)
-temp = utils.tempdir()
-my_gemm.save(temp.relpath("gemm.o"))
-remote.upload(temp.relpath("gemm.o"))
-f = remote.load_module("gemm.o")
-
-# Get the remote device context
-ctx = remote.ext_dev(0)
-
-# Initialize the data and weight arrays randomly in the int range of (-128, 128]
-data_np = np.random.randint(-128, 128, size=(batch_size, in_channels)).astype(data.dtype)
-weight_np = np.random.randint(-128, 128, size=(out_channels, in_channels)).astype(weight.dtype)
-
-# Apply packing to the data and weight arrays from a 2D to a 4D packed layout
-data_packed = data_np.reshape(
- batch_size // env.BATCH, env.BATCH, in_channels // env.BLOCK_IN, env.BLOCK_IN
-).transpose((0, 2, 1, 3))
-weight_packed = weight_np.reshape(
- out_channels // env.BLOCK_OUT, env.BLOCK_OUT, in_channels // env.BLOCK_IN, env.BLOCK_IN
-).transpose((0, 2, 1, 3))
-
-# Format the input/output arrays with tvm.nd.array to the DLPack standard
-data_nd = tvm.nd.array(data_packed, ctx)
-weight_nd = tvm.nd.array(weight_packed, ctx)
-res_nd = tvm.nd.array(np.zeros(output_shape).astype(res.dtype), ctx)
-
-# Clear stats
-if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
-
-# Invoke the module to perform the computation
-f(data_nd, weight_nd, res_nd)
-
-# Verify against numpy implementation
-res_ref = np.dot(data_np.astype(env.acc_dtype), weight_np.T.astype(env.acc_dtype))
-res_ref = res_ref >> env.INP_WIDTH
-res_ref = np.clip(res_ref, 0, inp_max)
-res_ref = res_ref.astype(res.dtype)
-res_ref = res_ref.reshape(
- batch_size // env.BATCH, env.BATCH, out_channels // env.BLOCK_OUT, env.BLOCK_OUT
-).transpose((0, 2, 1, 3))
-np.testing.assert_equal(res_ref, res_nd.numpy())
-
-# Print stats
-if env.TARGET in ["sim", "tsim"]:
- sim_stats = simulator.stats()
- print("Execution statistics:")
- for k, v in sim_stats.items():
- print("\t{:<16}: {:>16}".format(k, v))
-
-print("Successful blocked matrix multiply test!")
-
-######################################################################
-# Summary
-# -------
-# This tutorial demonstrates how TVM scheduling primitives can achieve
-# computation blocking for a matrix multiplication example.
-# This allows us to map arbitrarily large computation onto limited
-# hardware accelerator resources.
-#
diff --git a/docs/_downloads/83b9961c758069912464db3443fffc06/vta_get_started.ipynb b/docs/_downloads/83b9961c758069912464db3443fffc06/vta_get_started.ipynb
deleted file mode 100644
index 7797483fc2..0000000000
--- a/docs/_downloads/83b9961c758069912464db3443fffc06/vta_get_started.ipynb
+++ /dev/null
@@ -1,373 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n\n# Get Started with VTA\n**Author**: [Thierry Moreau](https://homes.cs.washington.edu/~moreau/)\n\nThis is an introduction tutorial on how to use TVM to program the VTA design.\n\nIn this tutorial, we will demonstrate the basic TVM workflow to implement\na vector addition on the VTA design's vector ALU.\nThis process includes specific scheduling transformations necessary to lower\ncomputation down to low-level accelerator operations.\n\nTo begin, we need to import TVM which i [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from __future__ import absolute_import, print_function\n\nimport os\nimport tvm\nfrom tvm import te\nimport vta\nimport numpy as np"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Loading in VTA Parameters\nVTA is a modular and customizable design. Consequently, the user\nis free to modify high-level hardware parameters that affect\nthe hardware design layout.\nThese parameters are specified in the :code:`vta_config.json` file by their\n:code:`log2` values.\nThese VTA parameters can be loaded with the :code:`vta.get_env`\nfunction.\n\nFinally, the TVM target is also specified in the :code:`vta_config.json` file.\nWhen set to *sim*, execution will take [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "env = vta.get_env()"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### FPGA Programming\nWhen targeting the Pynq FPGA development board, we need to configure\nthe board with a VTA bitstream.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# We'll need the TVM RPC module and the VTA simulator module\nfrom tvm import rpc\nfrom tvm.contrib import utils\nfrom vta.testing import simulator\n\n# We read the Pynq RPC host IP address and port number from the OS environment\nhost = os.environ.get(\"VTA_RPC_HOST\", \"192.168.2.99\")\nport = int(os.environ.get(\"VTA_RPC_PORT\", \"9091\"))\n\n# We configure both the bitstream and the runtime system on the Pynq\n# to match the VTA configuration specified by the vta_config.json [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Computation Declaration\nAs a first step, we need to describe our computation.\nTVM adopts tensor semantics, with each intermediate result\nrepresented as multi-dimensional array. The user needs to describe\nthe computation rule that generates the output tensors.\n\nIn this example we describe a vector addition, which requires multiple\ncomputation stages, as shown in the dataflow diagram below.\nFirst we describe the input tensors :code:`A` and :code:`B` that are living\nin [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Input Placeholders\nWe describe the placeholder tensors :code:`A`, and :code:`B` in a tiled data\nformat to match the data layout requirements imposed by the VTA vector ALU.\n\nFor VTA's general purpose operations such as vector adds, the tile size is\n:code:`(env.BATCH, env.BLOCK_OUT)`.\nThe dimensions are specified in\nthe :code:`vta_config.json` configuration file and are set by default to\na (1, 16) vector.\n\nIn addition, A and B's data types also needs to match the :cod [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Output channel factor m - total 64 x 16 = 1024 output channels\nm = 64\n# Batch factor o - total 1 x 1 = 1\no = 1\n# A placeholder tensor in tiled data format\nA = te.placeholder((o, m, env.BATCH, env.BLOCK_OUT), name=\"A\", dtype=env.acc_dtype)\n# B placeholder tensor in tiled data format\nB = te.placeholder((o, m, env.BATCH, env.BLOCK_OUT), name=\"B\", dtype=env.acc_dtype)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Copy Buffers\nOne specificity of hardware accelerators, is that on-chip memory has to be\nexplicitly managed.\nThis means that we'll need to describe intermediate tensors :code:`A_buf`\nand :code:`B_buf` that can have a different memory scope than the original\nplaceholder tensors :code:`A` and :code:`B`.\n\nLater in the scheduling phase, we can tell the compiler that :code:`A_buf`\nand :code:`B_buf` will live in the VTA's on-chip buffers (SRAM), while\n:code:`A` and :code:`B [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# A copy buffer\nA_buf = te.compute((o, m, env.BATCH, env.BLOCK_OUT), lambda *i: A(*i), \"A_buf\")\n# B copy buffer\nB_buf = te.compute((o, m, env.BATCH, env.BLOCK_OUT), lambda *i: B(*i), \"B_buf\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Vector Addition\nNow we're ready to describe the vector addition result tensor :code:`C`,\nwith another compute operation.\nThe compute function takes the shape of the tensor, as well as a lambda\nfunction that describes the computation rule for each position of the tensor.\n\nNo computation happens during this phase, as we are only declaring how\nthe computation should be done.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Describe the in-VTA vector addition\nC_buf = te.compute(\n (o, m, env.BATCH, env.BLOCK_OUT),\n lambda *i: A_buf(*i).astype(env.acc_dtype) + B_buf(*i).astype(env.acc_dtype),\n name=\"C_buf\",\n)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Casting the Results\nAfter the computation is done, we'll need to send the results computed by VTA\nback to main memory.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "<div class=\"alert alert-info\"><h4>Note</h4><p>**Memory Store Restrictions**\n\n One specificity of VTA is that it only supports DRAM stores in the narrow\n :code:`env.inp_dtype` data type format.\n This lets us reduce the data footprint for memory transfers (more on this\n in the basic matrix multiply example).</p></div>\n\nWe perform one last typecast operation to the narrow\ninput activation data format.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Cast to output type, and send to main memory\nC = te.compute(\n (o, m, env.BATCH, env.BLOCK_OUT), lambda *i: C_buf(*i).astype(env.inp_dtype), name=\"C\"\n)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This concludes the computation declaration part of this tutorial.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Scheduling the Computation\nWhile the above lines describes the computation rule, we can obtain\n:code:`C` in many ways.\nTVM asks the user to provide an implementation of the computation called\n*schedule*.\n\nA schedule is a set of transformations to an original computation that\ntransforms the implementation of the computation without affecting\ncorrectness.\nThis simple VTA programming tutorial aims to demonstrate basic schedule\ntransformations that will map the origina [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Default Schedule\nAfter we construct the schedule, by default the schedule computes\n:code:`C` in the following way:\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Let's take a look at the generated schedule\ns = te.create_schedule(C.op)\n\nprint(tvm.lower(s, [A, B, C], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "Although this schedule makes sense, it won't compile to VTA.\nIn order to obtain correct code generation, we need to apply scheduling\nprimitives and code annotation that will transform the schedule into\none that can be directly lowered onto VTA hardware intrinsics.\nThose include:\n\n - DMA copy operations which will take globally-scoped tensors and copy\n those into locally-scoped tensors.\n - Vector ALU operations that will perform the vector add.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Buffer Scopes\nFirst, we set the scope of the copy buffers to indicate to TVM that these\nintermediate tensors will be stored in the VTA's on-chip SRAM buffers.\nBelow, we tell TVM that :code:`A_buf`, :code:`B_buf`, :code:`C_buf`\nwill live in VTA's on-chip *accumulator buffer* which serves as\nVTA's general purpose register file.\n\nSet the intermediate tensors' scope to VTA's on-chip accumulator buffer\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "s[A_buf].set_scope(env.acc_scope)\ns[B_buf].set_scope(env.acc_scope)\ns[C_buf].set_scope(env.acc_scope)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## DMA Transfers\nWe need to schedule DMA transfers to move data living in DRAM to\nand from the VTA on-chip buffers.\nWe insert :code:`dma_copy` pragmas to indicate to the compiler\nthat the copy operations will be performed in bulk via DMA,\nwhich is common in hardware accelerators.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Tag the buffer copies with the DMA pragma to map a copy loop to a\n# DMA transfer operation\ns[A_buf].pragma(s[A_buf].op.axis[0], env.dma_copy)\ns[B_buf].pragma(s[B_buf].op.axis[0], env.dma_copy)\ns[C].pragma(s[C].op.axis[0], env.dma_copy)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## ALU Operations\nVTA has a vector ALU that can perform vector operations on tensors\nin the accumulator buffer.\nIn order to tell TVM that a given operation needs to be mapped to the\nVTA's vector ALU, we need to explicitly tag the vector addition loop\nwith an :code:`env.alu` pragma.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Tell TVM that the computation needs to be performed\n# on VTA's vector ALU\ns[C_buf].pragma(C_buf.op.axis[0], env.alu)\n\n# Let's take a look at the finalized schedule\nprint(vta.lower(s, [A, B, C], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "This concludes the scheduling portion of this tutorial.\n\n"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### TVM Compilation\nAfter we have finished specifying the schedule, we can compile it\ninto a TVM function. By default TVM compiles into a type-erased\nfunction that can be directly called from python side.\n\nIn the following line, we use :code:`tvm.build` to create a function.\nThe build function takes the schedule, the desired signature of the\nfunction(including the inputs and outputs) as well as target language\nwe want to compile to.\n\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "my_vadd = vta.build(\n s, [A, B, C], tvm.target.Target(\"ext_dev\", host=env.target_host), name=\"my_vadd\"\n)"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Saving the Module\nTVM lets us save our module into a file so it can loaded back later. This\nis called ahead-of-time compilation and allows us to save some compilation\ntime.\nMore importantly, this allows us to cross-compile the executable on our\ndevelopment machine and send it over to the Pynq FPGA board over RPC for\nexecution.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Write the compiled module into an object file.\ntemp = utils.tempdir()\nmy_vadd.save(temp.relpath(\"vadd.o\"))\n\n# Send the executable over RPC\nremote.upload(temp.relpath(\"vadd.o\"))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Loading the Module\nWe can load the compiled module from the file system to run the code.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "f = remote.load_module(\"vadd.o\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Running the Function\nThe compiled TVM function uses a concise C API and can be invoked from\nany language.\n\nTVM provides an array API in python to aid quick testing and prototyping.\nThe array API is based on [DLPack](https://github.com/dmlc/dlpack) standard.\n\n- We first create a remote context (for remote execution on the Pynq).\n- Then :code:`tvm.nd.array` formats the data accordingly.\n- :code:`f()` runs the actual computation.\n- :code:`numpy()` copies the result ar [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Get the remote device context\nctx = remote.ext_dev(0)\n\n# Initialize the A and B arrays randomly in the int range of (-128, 128]\nA_orig = np.random.randint(-128, 128, size=(o * env.BATCH, m * env.BLOCK_OUT)).astype(A.dtype)\nB_orig = np.random.randint(-128, 128, size=(o * env.BATCH, m * env.BLOCK_OUT)).astype(B.dtype)\n\n# Apply packing to the A and B arrays from a 2D to a 4D packed layout\nA_packed = A_orig.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))\nB [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Verifying Correctness\nCompute the reference result with numpy and assert that the output of the\nmatrix multiplication indeed is correct\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Compute reference result with numpy\nC_ref = (A_orig.astype(env.acc_dtype) + B_orig.astype(env.acc_dtype)).astype(C.dtype)\nC_ref = C_ref.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))\nnp.testing.assert_equal(C_ref, C_nd.numpy())\nprint(\"Successful vector add test!\")"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Summary\nThis tutorial provides a walk-through of TVM for programming the\ndeep learning accelerator VTA with a simple vector addition example.\nThe general workflow includes:\n\n- Programming the FPGA with the VTA bitstream over RPC.\n- Describing the vector add computation via a series of computations.\n- Describing how we want to perform the computation using schedule primitives.\n- Compiling the function to the VTA target.\n- Running the compiled module and verifying it [...]
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.20"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/b3f997c945cc7de3e03a1e0c4c73fabd/convolution_opt.ipynb b/docs/_downloads/b3f997c945cc7de3e03a1e0c4c73fabd/convolution_opt.ipynb
deleted file mode 100644
index 624f36720c..0000000000
--- a/docs/_downloads/b3f997c945cc7de3e03a1e0c4c73fabd/convolution_opt.ipynb
+++ /dev/null
@@ -1,194 +0,0 @@
-{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "%%shell\n# Installs the latest dev build of TVM from PyPI. If you wish to build\n# from source, see https://tvm.apache.org/docs/install/from_source.html\npip install apache-tvm --pre"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "\n# 2D Convolution Optimization\n**Author**: [Thierry Moreau](https://homes.cs.washington.edu/~moreau/)\n\nThis tutorial provides an overview on how to use TVM to map a 2D convolution\nworkload efficiently on the VTA design.\nWe recommend covering the `vta-mat-mult-opt` tutorial first.\n\n2D convolution is dominant in most computer vision deep neural networks.\nIn this tutorial, we will demonstrate TVM schedule optimizations to map\n2D convolution operators in NCHW layout onto V [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## RPC Setup\nWe start by programming the Pynq's FPGA and building its RPC runtime.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from __future__ import absolute_import, print_function\n\nimport os\nimport tvm\nimport tvm.testing\nfrom tvm import te\nimport vta\nimport numpy as np\n\nfrom tvm import rpc\nfrom tvm.contrib import utils\nfrom vta.testing import simulator\n\n# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file\nenv = vta.get_env()\n\n# We read the Pynq RPC host IP address and port number from the OS environment\nhost = os.environ.get(\"VTA_RPC_HOST\", \"192.168.2.99\")\np [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Computation Declaration\nAs a first step, we need to describe our 2D convolution computation\nin NCHW format.\n\nWe define the 2D convolution shape by the batch size,\nspatial dimensions, input channels, output channels, kernel dimensions,\nkernel dimensions, padding dimensions, and stride dimensions.\n\nWe pick the shape of the 9th convolutional layer of the ResNet-18\narchitecture as our convolution workload parameters.\n\nWe've added extra operators to the 2D convolution t [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "from tvm import topi\n\n# 2D convolution layer dimensions taken from ResNet-18 architecture\n# (9th convolutional layer)\nbatch_size = 1\nheight = 14\nwidth = 14\nin_channels = 256\nout_channels = 256\nkernel_h = 3\nkernel_w = 3\npad_h = 1\npad_w = 1\nstride_h = 1\nstride_w = 1\nassert batch_size % env.BATCH == 0\nassert in_channels % env.BLOCK_IN == 0\nassert out_channels % env.BLOCK_OUT == 0\n\n# Input feature map: (N, IC, H, W, n, ic)\ndata_shape = (\n batch_size // env.BA [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Scheduling the Computation\nWe'll look at a set of schedule transformations necessary to map the\n2D convolution onto VTA in an efficient fashion.\nThose include:\n\n- Computation blocking\n- Virtual threading to increase compute utilization\n- Lowering to VTA hardware intrinsics\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Create TVM schedule\ns = te.create_schedule(res.op)\n# Let's look at the default TVM schedule\nprint(tvm.lower(s, [data, kernel, res], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Blocking the Computation\nThe 2D convolution is by default too large for activations or kernel weights\nto fit on VTA's on-chip buffers all at once.\nWe apply blocking along input channels, output channels, and along\nthe height spatial dimensions.\nWe don't apply blocking along the width spatial dimension since it's\nthe innermost dimension in the NCHW layout (and consequently to increase\nlocality, it's best not to block along the innermost dimension).\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Let's define tiling sizes\nb_block = 1 // env.BATCH\noc_block = 128 // env.BLOCK_OUT\nic_block = 16 // env.BLOCK_IN\nh_block = 7\nw_block = 14\n\n# Tile the output tensor along the spatial and output channel dimensions\n# (since by default we are doing single batch inference, the split along\n# the batch dimension has no effect)\nb, oc, y, x, b_tns, oc_tns = s[res].op.axis\nb_out, b_inn = s[res].split(b, factor=b_block)\noc_out, oc_inn = s[res].split(oc, factor=oc_block)\ny_o [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Virtual Threading\nVirtual threading is a mechanism that increases task-level pipeline\nparallelism in the VTA hardware design.\nPut it another way, it increases compute resource utilization by hiding\nmemory access latency.\n\nIn the implementation below, virtual threading distributes work across two\nthreads split along the output channel axis.\nWe show how work is split when computing the 2D convolution in the figure\nbelow.\n\n<img src=\"https://raw.githubusercontent.com [...]
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# VTA only supports 2 virtual threads\nv_threads = 2\n\n# Perform virtual thread split along output channel outer axis\n_, tx = s[res].split(oc_out, factor=v_threads)\ns[res].reorder(tx, b_out)\ns[res].bind(tx, te.thread_axis(\"cthread\"))\n\n# Let's look at the current TVM schedule after blocking and virtual threading\nprint(tvm.lower(s, [data, kernel, res], simple_mode=True))"
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Lowering Copies to DMA Transfers\nNext we set the buffer scopes to the corresponding on-chip VTA SRAM buffers.\nWe move the load loops into the 2D convolution computation loop to stage\nmemory loads such that they fit in the on-chip SRAM buffers.\nFinally we annotate the load/store loop outer axes with the DMA copy pragma\nto perform bulk memory transfers on VTA.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Set scope of SRAM buffers\ns[data_buf].set_scope(env.inp_scope)\ns[kernel_buf].set_scope(env.wgt_scope)\ns[res_conv].set_scope(env.acc_scope)\ns[res_shr].set_scope(env.acc_scope)\ns[res_min].set_scope(env.acc_scope)\ns[res_max].set_scope(env.acc_scope)\n\n# Block data and kernel cache reads\ns[data_buf].compute_at(s[res_conv], ic_out)\ns[kernel_buf].compute_at(s[res_conv], ic_out)\n\n# Use DMA copy pragma on DRAM->SRAM operations\ns[data_buf].pragma(s[data_buf].op.axis[0], env [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "### Lowering Computation to VTA Compute Intrinsics\nThe last phase is to lower the computation loops down to VTA hardware\nintrinsics by mapping the 2D convolution to tensor intrinsics,\nand mapping the shift, and clipping computation to the vector ALU.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# Apply tensorization over the batch tensor tile axis\ns[res_conv].tensorize(b_tns, env.gemm)\n\n# Add an ALU pragma over the shift and clipping operations\ns[res_shr].pragma(s[res_shr].op.axis[0], env.alu)\ns[res_min].pragma(s[res_min].op.axis[0], env.alu)\ns[res_max].pragma(s[res_max].op.axis[0], env.alu)\n\n# Let's look at the final lowered TVM schedule after lowering memory\n# loads/stores down to DMA copy intrinsics, and the computation down to\n# VTA compute intrinsics.\np [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## TVM Compilation and Verification\nAfter specifying the schedule, we can compile it into a TVM function.\nWe save the module so we can send it over RPC.\nWe run the function and verify it against a numpy implementation to\nensure correctness.\n\n"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "metadata": {
- "collapsed": false
- },
- "outputs": [],
- "source": [
- "# This library facilitates 2D convolution testing\nfrom tvm.topi.testing import conv2d_nchw_python\n\n# Compile the TVM module\nwith vta.build_config(disabled_pass={\"tir.CommonSubexprElimTIR\"}):\n my_conv = vta.build(\n s, [data, kernel, res], tvm.target.Target(\"ext_dev\", host=env.target_host), name=\"my_conv\"\n )\ntemp = utils.tempdir()\nmy_conv.save(temp.relpath(\"conv2d.o\"))\nremote.upload(temp.relpath(\"conv2d.o\"))\nf = remote.load_module(\"conv2d.o\")\n\ [...]
- ]
- },
- {
- "cell_type": "markdown",
- "metadata": {},
- "source": [
- "## Summary\nThis tutorial demonstrates how TVM scheduling primitives can be used to\nlower 2D convolution onto hardware accelerator intrinsics, making\nuse of hardware specific optimizations, such as latency hiding with\nvirtual threading.\n\n\n"
- ]
- }
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.8.20"
- }
- },
- "nbformat": 4,
- "nbformat_minor": 0
-}
\ No newline at end of file
diff --git a/docs/_downloads/d2434fbd36b5bd5a93a69ca80465d5b6/vta_get_started.py b/docs/_downloads/d2434fbd36b5bd5a93a69ca80465d5b6/vta_get_started.py
deleted file mode 100644
index 3482258dec..0000000000
--- a/docs/_downloads/d2434fbd36b5bd5a93a69ca80465d5b6/vta_get_started.py
+++ /dev/null
@@ -1,405 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-.. _vta-get-started:
-
-Get Started with VTA
-====================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This is an introduction tutorial on how to use TVM to program the VTA design.
-
-In this tutorial, we will demonstrate the basic TVM workflow to implement
-a vector addition on the VTA design's vector ALU.
-This process includes specific scheduling transformations necessary to lower
-computation down to low-level accelerator operations.
-
-To begin, we need to import TVM which is our deep learning optimizing compiler.
-We also need to import the VTA python package which contains VTA specific
-extensions for TVM to target the VTA design.
-"""
-from __future__ import absolute_import, print_function
-
-import os
-import tvm
-from tvm import te
-import vta
-import numpy as np
-
-######################################################################
-# Loading in VTA Parameters
-# ~~~~~~~~~~~~~~~~~~~~~~~~~
-# VTA is a modular and customizable design. Consequently, the user
-# is free to modify high-level hardware parameters that affect
-# the hardware design layout.
-# These parameters are specified in the :code:`vta_config.json` file by their
-# :code:`log2` values.
-# These VTA parameters can be loaded with the :code:`vta.get_env`
-# function.
-#
-# Finally, the TVM target is also specified in the :code:`vta_config.json` file.
-# When set to *sim*, execution will take place inside of a behavioral
-# VTA simulator.
-# If you want to run this tutorial on the Pynq FPGA development platform,
-# follow the *VTA Pynq-Based Testing Setup* guide.
-
-env = vta.get_env()
-
-######################################################################
-# FPGA Programming
-# ----------------
-# When targeting the Pynq FPGA development board, we need to configure
-# the board with a VTA bitstream.
-
-# We'll need the TVM RPC module and the VTA simulator module
-from tvm import rpc
-from tvm.contrib import utils
-from vta.testing import simulator
-
-# We read the Pynq RPC host IP address and port number from the OS environment
-host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
-port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
-# We configure both the bitstream and the runtime system on the Pynq
-# to match the VTA configuration specified by the vta_config.json file.
-if env.TARGET == "pynq" or env.TARGET == "de10nano":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
-# In simulation mode, host the RPC server locally.
-elif env.TARGET in ("sim", "tsim", "intelfocl"):
- remote = rpc.LocalSession()
-
- if env.TARGET in ["intelfocl"]:
- # program intelfocl aocx
- vta.program_fpga(remote, bitstream="vta.bitstream")
-
-######################################################################
-# Computation Declaration
-# -----------------------
-# As a first step, we need to describe our computation.
-# TVM adopts tensor semantics, with each intermediate result
-# represented as multi-dimensional array. The user needs to describe
-# the computation rule that generates the output tensors.
-#
-# In this example we describe a vector addition, which requires multiple
-# computation stages, as shown in the dataflow diagram below.
-# First we describe the input tensors :code:`A` and :code:`B` that are living
-# in main memory.
-# Second, we need to declare intermediate tensors :code:`A_buf` and
-# :code:`B_buf`, which will live in VTA's on-chip buffers.
-# Having this extra computational stage allows us to explicitly
-# stage cached reads and writes.
-# Third, we describe the vector addition computation which will
-# add :code:`A_buf` to :code:`B_buf` to produce :code:`C_buf`.
-# The last operation is a cast and copy back to DRAM, into results tensor
-# :code:`C`.
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/vadd_dataflow.png
-# :align: center
-
-######################################################################
-# Input Placeholders
-# ~~~~~~~~~~~~~~~~~~
-# We describe the placeholder tensors :code:`A`, and :code:`B` in a tiled data
-# format to match the data layout requirements imposed by the VTA vector ALU.
-#
-# For VTA's general purpose operations such as vector adds, the tile size is
-# :code:`(env.BATCH, env.BLOCK_OUT)`.
-# The dimensions are specified in
-# the :code:`vta_config.json` configuration file and are set by default to
-# a (1, 16) vector.
-#
-# In addition, A and B's data types also needs to match the :code:`env.acc_dtype`
-# which is set by the :code:`vta_config.json` file to be a 32-bit integer.
-
-# Output channel factor m - total 64 x 16 = 1024 output channels
-m = 64
-# Batch factor o - total 1 x 1 = 1
-o = 1
-# A placeholder tensor in tiled data format
-A = te.placeholder((o, m, env.BATCH, env.BLOCK_OUT), name="A", dtype=env.acc_dtype)
-# B placeholder tensor in tiled data format
-B = te.placeholder((o, m, env.BATCH, env.BLOCK_OUT), name="B", dtype=env.acc_dtype)
-
-######################################################################
-# Copy Buffers
-# ~~~~~~~~~~~~
-# One specificity of hardware accelerators, is that on-chip memory has to be
-# explicitly managed.
-# This means that we'll need to describe intermediate tensors :code:`A_buf`
-# and :code:`B_buf` that can have a different memory scope than the original
-# placeholder tensors :code:`A` and :code:`B`.
-#
-# Later in the scheduling phase, we can tell the compiler that :code:`A_buf`
-# and :code:`B_buf` will live in the VTA's on-chip buffers (SRAM), while
-# :code:`A` and :code:`B` will live in main memory (DRAM).
-# We describe A_buf and B_buf as the result of a compute
-# operation that is the identity function.
-# This can later be interpreted by the compiler as a cached read operation.
-
-# A copy buffer
-A_buf = te.compute((o, m, env.BATCH, env.BLOCK_OUT), lambda *i: A(*i), "A_buf")
-# B copy buffer
-B_buf = te.compute((o, m, env.BATCH, env.BLOCK_OUT), lambda *i: B(*i), "B_buf")
-
-######################################################################
-# Vector Addition
-# ~~~~~~~~~~~~~~~
-# Now we're ready to describe the vector addition result tensor :code:`C`,
-# with another compute operation.
-# The compute function takes the shape of the tensor, as well as a lambda
-# function that describes the computation rule for each position of the tensor.
-#
-# No computation happens during this phase, as we are only declaring how
-# the computation should be done.
-
-# Describe the in-VTA vector addition
-C_buf = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT),
- lambda *i: A_buf(*i).astype(env.acc_dtype) + B_buf(*i).astype(env.acc_dtype),
- name="C_buf",
-)
-
-######################################################################
-# Casting the Results
-# ~~~~~~~~~~~~~~~~~~~
-# After the computation is done, we'll need to send the results computed by VTA
-# back to main memory.
-
-######################################################################
-# .. note::
-#
-# **Memory Store Restrictions**
-#
-# One specificity of VTA is that it only supports DRAM stores in the narrow
-# :code:`env.inp_dtype` data type format.
-# This lets us reduce the data footprint for memory transfers (more on this
-# in the basic matrix multiply example).
-#
-# We perform one last typecast operation to the narrow
-# input activation data format.
-
-# Cast to output type, and send to main memory
-C = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT), lambda *i: C_buf(*i).astype(env.inp_dtype), name="C"
-)
-
-######################################################################
-# This concludes the computation declaration part of this tutorial.
-
-
-######################################################################
-# Scheduling the Computation
-# --------------------------
-# While the above lines describes the computation rule, we can obtain
-# :code:`C` in many ways.
-# TVM asks the user to provide an implementation of the computation called
-# *schedule*.
-#
-# A schedule is a set of transformations to an original computation that
-# transforms the implementation of the computation without affecting
-# correctness.
-# This simple VTA programming tutorial aims to demonstrate basic schedule
-# transformations that will map the original schedule down to VTA hardware
-# primitives.
-
-
-######################################################################
-# Default Schedule
-# ~~~~~~~~~~~~~~~~
-# After we construct the schedule, by default the schedule computes
-# :code:`C` in the following way:
-
-# Let's take a look at the generated schedule
-s = te.create_schedule(C.op)
-
-print(tvm.lower(s, [A, B, C], simple_mode=True))
-
-######################################################################
-# Although this schedule makes sense, it won't compile to VTA.
-# In order to obtain correct code generation, we need to apply scheduling
-# primitives and code annotation that will transform the schedule into
-# one that can be directly lowered onto VTA hardware intrinsics.
-# Those include:
-#
-# - DMA copy operations which will take globally-scoped tensors and copy
-# those into locally-scoped tensors.
-# - Vector ALU operations that will perform the vector add.
-
-######################################################################
-# Buffer Scopes
-# ~~~~~~~~~~~~~
-# First, we set the scope of the copy buffers to indicate to TVM that these
-# intermediate tensors will be stored in the VTA's on-chip SRAM buffers.
-# Below, we tell TVM that :code:`A_buf`, :code:`B_buf`, :code:`C_buf`
-# will live in VTA's on-chip *accumulator buffer* which serves as
-# VTA's general purpose register file.
-#
-# Set the intermediate tensors' scope to VTA's on-chip accumulator buffer
-s[A_buf].set_scope(env.acc_scope)
-s[B_buf].set_scope(env.acc_scope)
-s[C_buf].set_scope(env.acc_scope)
-
-######################################################################
-# DMA Transfers
-# ~~~~~~~~~~~~~
-# We need to schedule DMA transfers to move data living in DRAM to
-# and from the VTA on-chip buffers.
-# We insert :code:`dma_copy` pragmas to indicate to the compiler
-# that the copy operations will be performed in bulk via DMA,
-# which is common in hardware accelerators.
-
-# Tag the buffer copies with the DMA pragma to map a copy loop to a
-# DMA transfer operation
-s[A_buf].pragma(s[A_buf].op.axis[0], env.dma_copy)
-s[B_buf].pragma(s[B_buf].op.axis[0], env.dma_copy)
-s[C].pragma(s[C].op.axis[0], env.dma_copy)
-
-######################################################################
-# ALU Operations
-# ~~~~~~~~~~~~~~
-# VTA has a vector ALU that can perform vector operations on tensors
-# in the accumulator buffer.
-# In order to tell TVM that a given operation needs to be mapped to the
-# VTA's vector ALU, we need to explicitly tag the vector addition loop
-# with an :code:`env.alu` pragma.
-
-# Tell TVM that the computation needs to be performed
-# on VTA's vector ALU
-s[C_buf].pragma(C_buf.op.axis[0], env.alu)
-
-# Let's take a look at the finalized schedule
-print(vta.lower(s, [A, B, C], simple_mode=True))
-
-######################################################################
-# This concludes the scheduling portion of this tutorial.
-
-######################################################################
-# TVM Compilation
-# ---------------
-# After we have finished specifying the schedule, we can compile it
-# into a TVM function. By default TVM compiles into a type-erased
-# function that can be directly called from python side.
-#
-# In the following line, we use :code:`tvm.build` to create a function.
-# The build function takes the schedule, the desired signature of the
-# function(including the inputs and outputs) as well as target language
-# we want to compile to.
-#
-my_vadd = vta.build(
- s, [A, B, C], tvm.target.Target("ext_dev", host=env.target_host), name="my_vadd"
-)
-
-######################################################################
-# Saving the Module
-# ~~~~~~~~~~~~~~~~~
-# TVM lets us save our module into a file so it can loaded back later. This
-# is called ahead-of-time compilation and allows us to save some compilation
-# time.
-# More importantly, this allows us to cross-compile the executable on our
-# development machine and send it over to the Pynq FPGA board over RPC for
-# execution.
-
-# Write the compiled module into an object file.
-temp = utils.tempdir()
-my_vadd.save(temp.relpath("vadd.o"))
-
-# Send the executable over RPC
-remote.upload(temp.relpath("vadd.o"))
-
-######################################################################
-# Loading the Module
-# ~~~~~~~~~~~~~~~~~~
-# We can load the compiled module from the file system to run the code.
-
-f = remote.load_module("vadd.o")
-
-######################################################################
-# Running the Function
-# --------------------
-# The compiled TVM function uses a concise C API and can be invoked from
-# any language.
-#
-# TVM provides an array API in python to aid quick testing and prototyping.
-# The array API is based on `DLPack <https://github.com/dmlc/dlpack>`_ standard.
-#
-# - We first create a remote context (for remote execution on the Pynq).
-# - Then :code:`tvm.nd.array` formats the data accordingly.
-# - :code:`f()` runs the actual computation.
-# - :code:`numpy()` copies the result array back in a format that can be
-# interpreted.
-#
-
-# Get the remote device context
-ctx = remote.ext_dev(0)
-
-# Initialize the A and B arrays randomly in the int range of (-128, 128]
-A_orig = np.random.randint(-128, 128, size=(o * env.BATCH, m * env.BLOCK_OUT)).astype(A.dtype)
-B_orig = np.random.randint(-128, 128, size=(o * env.BATCH, m * env.BLOCK_OUT)).astype(B.dtype)
-
-# Apply packing to the A and B arrays from a 2D to a 4D packed layout
-A_packed = A_orig.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
-B_packed = B_orig.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
-
-# Format the input/output arrays with tvm.nd.array to the DLPack standard
-A_nd = tvm.nd.array(A_packed, ctx)
-B_nd = tvm.nd.array(B_packed, ctx)
-C_nd = tvm.nd.array(np.zeros((o, m, env.BATCH, env.BLOCK_OUT)).astype(C.dtype), ctx)
-
-# Invoke the module to perform the computation
-f(A_nd, B_nd, C_nd)
-
-######################################################################
-# Verifying Correctness
-# ---------------------
-# Compute the reference result with numpy and assert that the output of the
-# matrix multiplication indeed is correct
-
-# Compute reference result with numpy
-C_ref = (A_orig.astype(env.acc_dtype) + B_orig.astype(env.acc_dtype)).astype(C.dtype)
-C_ref = C_ref.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
-np.testing.assert_equal(C_ref, C_nd.numpy())
-print("Successful vector add test!")
-
-######################################################################
-# Summary
-# -------
-# This tutorial provides a walk-through of TVM for programming the
-# deep learning accelerator VTA with a simple vector addition example.
-# The general workflow includes:
-#
-# - Programming the FPGA with the VTA bitstream over RPC.
-# - Describing the vector add computation via a series of computations.
-# - Describing how we want to perform the computation using schedule primitives.
-# - Compiling the function to the VTA target.
-# - Running the compiled module and verifying it against a numpy implementation.
-#
-# You are more than welcome to check other examples out and tutorials
-# to learn more about the supported operations, schedule primitives
-# and other features supported by TVM to program VTA.
-#
diff --git a/docs/_downloads/de1c160863e8a3826753e987a4138298/matrix_multiply.py b/docs/_downloads/de1c160863e8a3826753e987a4138298/matrix_multiply.py
deleted file mode 100644
index 0d11678544..0000000000
--- a/docs/_downloads/de1c160863e8a3826753e987a4138298/matrix_multiply.py
+++ /dev/null
@@ -1,474 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-"""
-.. _basic-mat-mult:
-
-Simple Matrix Multiply
-======================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-In this tutorial, we will build on top of the :ref:`vta-get-started` tutorial
-and introduce additional concepts required to implement matrix multiplication
-on VTA with the TVM workflow.
-"""
-
-######################################################################
-# RPC Setup
-# ---------
-# We start by programming the Pynq's FPGA and building its RPC runtime
-# as we did in the VTA introductory tutorial.
-
-from __future__ import absolute_import, print_function
-
-import os
-import tvm
-from tvm import te
-import vta
-import numpy as np
-from tvm import rpc
-from tvm.contrib import utils
-from vta.testing import simulator
-
-# Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
-env = vta.get_env()
-
-# We read the Pynq RPC host IP address and port number from the OS environment
-host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
-port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
-# We configure both the bitstream and the runtime system on the Pynq
-# to match the VTA configuration specified by the vta_config.json file.
-if env.TARGET == "pynq" or env.TARGET == "de10nano":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
-# In simulation mode, host the RPC server locally.
-elif env.TARGET in ["sim", "tsim"]:
- remote = rpc.LocalSession()
-
-######################################################################
-# Computation Declaration
-# -----------------------
-# In this example we describe a simple matrix multiplication addition, which
-# requires multiple computation stages, as shown in the dataflow diagram below.
-# First we describe the input tensors :code:`A` and :code:`B` that are living
-# in main memory.
-# Second, we need to declare intermediate tensors :code:`A_buf` and
-# :code:`B_buf`, which will live in VTA's on-chip buffers.
-# Having this extra computational stage allows us to explicitly
-# stage cached reads and writes.
-# Third, we describe the matrix multiplication computation over
-# :code:`A_buf` and :code:`B_buf` to produce the product matrix :code:`C_buf`.
-# The last operation is a cast and copy back to DRAM, into results tensor
-# :code:`C`.
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/gemm_dataflow.png
-# :align: center
-
-######################################################################
-# Data Layout
-# ~~~~~~~~~~~
-# We describe the placeholder tensors :code:`A`, and :code:`B` in a tiled data
-# format to match the data layout requirements imposed by the VTA tensor core.
-
-######################################################################
-# .. note::
-#
-# **Data Tiling**
-#
-# One source of complexity when targeting accelerators is to make sure
-# that the data layout matches the layout imposed by the accelerator design.
-# VTA is designed around a *tensor core* that performs, one matrix-matrix
-# operation per cycle between an activation matrix and a weight matrix,
-# adding the result matrix to an accumulator matrix, as shown in the
-# figure below.
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/tensor_core.png
-# :align: center
-# :width: 480px
-#
-# The dimensions of that matrix-matrix multiplication are specified in
-# the :code:`vta_config.json` configuration file.
-# The activation matrix has a :code:`(BATCH, BLOCK_IN)` shape
-# and the transposed weight matrix has a :code:`(BLOCK_OUT, BLOCK_IN)` shape,
-# thus inferring that the resulting output matrix has a
-# :code:`(BATCH, BLOCK_OUT)` shape.
-# Consequently input and output tensors processed by VTA need to be
-# tiled according to these aforementioned dimension.
-#
-# The diagram below shows the impact of data tiling on a matrix that is
-# originally of shape (4, 8).
-# Tiling by a (2, 2) tile shape ensures that data within each tile is
-# contiguous.
-# The resulting tiled tensor has a shape of (2, 4, 2, 2).
-#
-# .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/data_tiling.png
-# :align: center
-# :width: 480px
-#
-# We first define the variables :code:`m`, :code:`n`, :code:`o` to represent
-# the shape of the matrix multiplication. These variables are multiplicative
-# factors over the :code:`BLOCK_OUT`, :code:`BLOCK_IN`, and :code:`BATCH`
-# tensor dimensions respectively.
-# By default, the configuration file sets :code:`BATCH`, :code:`BLOCK_IN`, and
-# :code:`BLOCK_OUT` to be 1, 16 and 16 respectively (:code:`BATCH` being set to
-# 1 implies that our compute building block is vector-matrix multiply).
-#
-
-######################################################################
-# .. note::
-#
-# **Data Types**
-#
-# It's important to not only match the inner-tile
-# dimension of VTA's tensor core, but also to match the specific data types
-# expected by VTA.
-# VTA for now only supports fixed point data types, which integer width is
-# specified in the :code:`vta_config.json` file by :code:`INP_WIDTH` and
-# :code:`WGT_WIDTH` for the activations and weights data types respectively.
-# In addition, the accumulator data type integer width is specified by
-# :code:`ACC_WIDTH`.
-#
-# By default, the configuration file sets :code:`INP_WIDTH`
-# and :code:`WGT_WIDTH` to 8.
-# The accumulator width :code:`ACC_WIDTH` is set to 32, in order to avoid
-# overflow during accumulation.
-# As a result, :code:`env.inp_dtype` and :code:`env.wgt_dtype` are all
-# narrow 8-bit integers, while :code:`env.acc_dtype` is a standard 32-bit
-# integer.
-
-# Output channel factor m - total 16x16=256 output channels
-m = 16
-# Input channel factor n - total 16x16=256 input channels
-n = 16
-# Batch factor o (we use single batch inference)
-o = 1
-# A placeholder tensor in tiled data format
-A = te.placeholder((o, n, env.BATCH, env.BLOCK_IN), name="A", dtype=env.inp_dtype)
-# B placeholder tensor in tiled data format
-B = te.placeholder((m, n, env.BLOCK_OUT, env.BLOCK_IN), name="B", dtype=env.wgt_dtype)
-# A copy buffer
-A_buf = te.compute((o, n, env.BATCH, env.BLOCK_IN), lambda *i: A(*i), "A_buf")
-# B copy buffer
-B_buf = te.compute((m, n, env.BLOCK_OUT, env.BLOCK_IN), lambda *i: B(*i), "B_buf")
-
-######################################################################
-# Matrix Multiplication
-# ~~~~~~~~~~~~~~~~~~~~~
-# Now we're ready to describe the matrix multiplication result tensor :code:`C`,
-# with another compute operation.
-# The compute function takes the shape of the tensor, as well as a lambda
-# function that describes the computation rule for each position of the tensor.
-#
-# In order to implement matrix multiplication, the lambda function needs to
-# include a reduction formula over the input channel dimension axes.
-# To create a reduction formula, we can declare a reduction axis using
-# :code:`te.reduce_axis`, which takes in the range of reductions.
-# :code:`te.sum` takes in the expression to be reduced as well as
-# the reduction axes to compute the sum of value over all k in the declared
-# ranges.
-#
-# Note that the reduction needs to be performed over 32-bit :code:`env.acc_dtype`
-# accumulator data types.
-#
-# No computation happens during this phase, as we are only declaring how
-# the computation should be done.
-
-# Outer input feature reduction axis
-ko = te.reduce_axis((0, n), name="ko")
-# Inner input feature reduction axis
-ki = te.reduce_axis((0, env.BLOCK_IN), name="ki")
-# Describe the in-VTA matrix multiplication
-C_buf = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT),
- lambda bo, co, bi, ci: te.sum(
- A_buf[bo, ko, bi, ki].astype(env.acc_dtype) * B_buf[co, ko, ci, ki].astype(env.acc_dtype),
- axis=[ko, ki],
- ),
- name="C_buf",
-)
-
-######################################################################
-# Casting the Results
-# ~~~~~~~~~~~~~~~~~~~
-# After the computation is done, we'll need to send the results computed by VTA
-# back to main memory.
-
-######################################################################
-# .. note::
-#
-# **Memory Store Restrictions**
-#
-# One specificity of VTA is that it only supports DRAM stores in the narrow
-# :code:`env.inp_dtype` data type format.
-# This lets us reduce the data footprint for memory transfers, but also lets
-# us quantize the wide accumulator data type down to a data format that
-# matches the input activation data type.
-# This means that in the context of neural network inference, the outputs
-# of a given layer after activation can be consumed directly by the next
-# layer.
-#
-# We perform one last typecast operation to the narrow
-# input activation data format.
-
-# Cast to output type, and send to main memory
-C = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT), lambda *i: C_buf(*i).astype(env.inp_dtype), name="C"
-)
-
-######################################################################
-# This concludes the computation declaration part of this tutorial.
-
-######################################################################
-# Scheduling the Computation
-# --------------------------
-# While the above lines describes the computation rule, we can obtain
-# :code:`C` in many ways.
-# TVM asks the user to provide an implementation of the computation called
-# *schedule*.
-#
-# A schedule is a set of transformations to an original computation that
-# transforms the implementation of the computation without affecting
-# correctness.
-# This simple VTA programming tutorial aims to demonstrate basic schedule
-# transformations that will map the original schedule down to VTA hardware
-# primitives.
-
-
-######################################################################
-# Default Schedule
-# ~~~~~~~~~~~~~~~~
-# After we construct the schedule, by default the schedule computes
-# :code:`C` in the following way:
-
-# Let's take a look at the generated schedule
-s = te.create_schedule(C.op)
-print(tvm.lower(s, [A, B, C], simple_mode=True))
-
-######################################################################
-# Although this schedule makes sense, it won't compile to VTA.
-# In order to obtain correct code generation, we need to apply scheduling
-# primitives and code annotation that will transform the schedule into
-# one that can be directly lowered onto VTA hardware intrinsics.
-# Those include:
-#
-# - DMA copy operations which will take globally-scoped tensors and copy
-# those into locally-scoped tensors.
-# - Tensor operations that will perform the matrix multiplication.
-
-######################################################################
-# Buffer Scopes
-# ~~~~~~~~~~~~~
-# First, we set the scope of the buffers to tell TVM that these buffers
-# will be living in the VTA's on-chip SRAM caches.
-# Below, we tell TVM that :code:`A_buf`, :code:`B_buf`, :code:`C_buf`
-# will respectively live in VTA's on-chip input, weight and accumulator
-# memory.
-
-######################################################################
-# .. note::
-#
-# **VTA's On-Chip SRAMs**
-#
-# VTA has three different memory scopes, each corresponding to different
-# on-chip SRAM buffers.
-#
-# - :code:`env.inp_scope`: Input buffer, which is a read-only SRAM buffer
-# that stores input matrices of shape :code:`(env.BATCH, env.BLOCK_IN)`
-# of type :code:`env.inp_dtype`. The input buffer contains
-# `2 ^ LOG_INP_BUFF_SIZE` matrix elements (as specified in the
-# :code:`vta_config.json` file).
-# - :code:`env.wgt_scope`: Weight buffer, which is a read-only SRAM buffer
-# that stores weight matrices of shape :code:`(env.BLOCK_OUT, env.BLOCK_IN)`
-# of type :code:`env.wgt_dtype`. The weight buffer contains
-# `2 ^ LOG_WGT_BUFF_SIZE` matrix elements.
-# - :code:`env.acc_scope`: Accumulator buffer, which is a read/write SRAM
-# buffer that stores accumulator matrices of shape
-# :code:`(env.BATCH, env.BLOCK_OUT)` of type :code:`env.acc_dtype`.
-# The accumulator buffer is VTA's general purpose register file: it holds
-# both intermediate results of convolutions and matrix multiplications
-# as well as intermediate results of pooling, batch normalization, and
-# activation layers. The accumulator buffer contains
-# `2 ^ LOG_ACC_BUFF_SIZE` matrix elements.
-
-# Set the intermediate tensor's scope to VTA's on-chip buffers
-s[A_buf].set_scope(env.inp_scope)
-s[B_buf].set_scope(env.wgt_scope)
-s[C_buf].set_scope(env.acc_scope)
-
-######################################################################
-# DMA Transfers
-# ~~~~~~~~~~~~~
-# We need to schedule DMA transfers to move data living in DRAM to
-# and from the VTA on-chip buffers.
-# This can be achieved using the :code:`compute_at` schedule primitive
-# which nests the copying of the buffers into the computation loop
-# that performs the matrix multiplication.
-#
-# We insert :code:`dma_copy` pragmas to indicate to the compiler
-# that the copy operations will be performed in bulk via DMA,
-# which is common in hardware accelerators.
-# Finally, we print the temporary schedule to observe the effects of
-# moving the copy operations into the matrix multiplication loop.
-
-# Move buffer copy into matrix multiply loop
-s[A_buf].compute_at(s[C_buf], ko)
-s[B_buf].compute_at(s[C_buf], ko)
-
-# Tag the buffer copies with the DMA pragma to insert a DMA transfer
-s[A_buf].pragma(s[A_buf].op.axis[0], env.dma_copy)
-s[B_buf].pragma(s[B_buf].op.axis[0], env.dma_copy)
-s[C].pragma(s[C].op.axis[0], env.dma_copy)
-
-# Let's take a look at the transformed schedule
-print(tvm.lower(s, [A, B, C], simple_mode=True))
-
-######################################################################
-# Tensorization
-# ~~~~~~~~~~~~~
-# The last step of the schedule transformation consists in applying
-# *tensorization* to our schedule.
-# Tensorization is analogous to vectorization, but extends the concept
-# to a higher-dimensional unit of computation.
-# Consequently, tensorization imposes data layout constraints as discussed
-# earlier when declaring the data layout input placeholders.
-# We've already arranged our tensors in a tiled format, so the next thing
-# we need to perform is loop reordering to accommodate for tensorization.
-#
-# Here we choose to move the outermost reduction axis all the way out.
-# This dictates that we first iterate over input channels, then batch
-# dimensions, and finally output channels.
-# Lastly, we apply the tensorization scheduling primitive :code:`tensorize`
-# along the outer axis of the inner-most matrix matrix multiplication tensor
-# block.
-# We print the finalized schedule that is ready for code-generation
-# by the VTA runtime JIT compiler.
-
-s[C_buf].reorder(
- ko, s[C_buf].op.axis[0], s[C_buf].op.axis[1], s[C_buf].op.axis[2], s[C_buf].op.axis[3], ki
-)
-s[C_buf].tensorize(s[C_buf].op.axis[2], env.gemm)
-
-# Let's take a look at the finalized schedule
-print(vta.lower(s, [A, B, C], simple_mode=True))
-
-######################################################################
-# This concludes the scheduling portion of this tutorial.
-
-######################################################################
-# TVM Compilation
-# ---------------
-# After we have finished specifying the schedule, we can compile it
-# into a TVM function.
-
-# Build GEMM VTA kernel
-my_gemm = vta.build(
- s, [A, B, C], tvm.target.Target("ext_dev", host=env.target_host), name="my_gemm"
-)
-
-# Write the compiled module into an object file.
-temp = utils.tempdir()
-my_gemm.save(temp.relpath("gemm.o"))
-
-# Send the executable over RPC
-remote.upload(temp.relpath("gemm.o"))
-
-# Load the compiled module
-f = remote.load_module("gemm.o")
-
-######################################################################
-# Running the Function
-# --------------------
-# The compiled TVM function uses a concise C API and can be invoked from
-# code language.
-#
-# TVM provides an array API in python to aid quick testing and prototyping.
-# The array API is based on `DLPack <https://github.com/dmlc/dlpack>`_ standard.
-#
-# - We first create a remote context (for remote execution on the Pynq).
-# - Then :code:`tvm.nd.array` formats the data accordingly.
-# - :code:`f()` runs the actual computation.
-# - :code:`numpy()` copies the result array back in a format that can be
-# interpreted.
-#
-
-# Get the remote device context
-ctx = remote.ext_dev(0)
-
-# Initialize the A and B arrays randomly in the int range of (-128, 128]
-A_orig = np.random.randint(-128, 128, size=(o * env.BATCH, n * env.BLOCK_IN)).astype(A.dtype)
-B_orig = np.random.randint(-128, 128, size=(m * env.BLOCK_OUT, n * env.BLOCK_IN)).astype(B.dtype)
-
-# Apply packing to the A and B arrays from a 2D to a 4D packed layout
-A_packed = A_orig.reshape(o, env.BATCH, n, env.BLOCK_IN).transpose((0, 2, 1, 3))
-B_packed = B_orig.reshape(m, env.BLOCK_OUT, n, env.BLOCK_IN).transpose((0, 2, 1, 3))
-
-# Format the input/output arrays with tvm.nd.array to the DLPack standard
-A_nd = tvm.nd.array(A_packed, ctx)
-B_nd = tvm.nd.array(B_packed, ctx)
-C_nd = tvm.nd.array(np.zeros((o, m, env.BATCH, env.BLOCK_OUT)).astype(C.dtype), ctx)
-
-# Clear stats
-if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
-
-# Invoke the module to perform the computation
-f(A_nd, B_nd, C_nd)
-
-######################################################################
-# Verifying Correctness
-# ---------------------
-# Compute the reference result with numpy and assert that the output of the
-# matrix multiplication indeed is correct
-
-# Compute reference result with numpy
-C_ref = np.dot(A_orig.astype(env.acc_dtype), B_orig.T.astype(env.acc_dtype)).astype(C.dtype)
-C_ref = C_ref.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
-np.testing.assert_equal(C_ref, C_nd.numpy())
-
-# Print stats
-if env.TARGET in ["sim", "tsim"]:
- sim_stats = simulator.stats()
- print("Execution statistics:")
- for k, v in sim_stats.items():
- print("\t{:<16}: {:>16}".format(k, v))
-
-print("Successful matrix multiply test!")
-
-######################################################################
-# Summary
-# -------
-# This tutorial showcases the TVM workflow to implement a simple matrix
-# multiplication example on VTA.
-# The general workflow includes:
-#
-# - Programming the FPGA with the VTA bitstream over RPC.
-# - Describing matrix multiplication via a series of computations.
-# - Describing how we want to perform the computation using schedule primitives.
-# - Compiling the function to the VTA target.
-# - Running the compiled module and verifying it against a numpy implementation.
-#
diff --git a/docs/_images/sphx_glr_convolution_opt_thumb.png b/docs/_images/sphx_glr_convolution_opt_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_convolution_opt_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_deploy_detection_001.png b/docs/_images/sphx_glr_deploy_detection_001.png
deleted file mode 100644
index 1fe07a6fa9..0000000000
Binary files a/docs/_images/sphx_glr_deploy_detection_001.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_deploy_detection_thumb.png b/docs/_images/sphx_glr_deploy_detection_thumb.png
deleted file mode 100644
index ae4a7c458d..0000000000
Binary files a/docs/_images/sphx_glr_deploy_detection_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_matrix_multiply_opt_thumb.png b/docs/_images/sphx_glr_matrix_multiply_opt_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_matrix_multiply_opt_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_matrix_multiply_thumb.png b/docs/_images/sphx_glr_matrix_multiply_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_matrix_multiply_thumb.png and /dev/null differ
diff --git a/docs/_images/sphx_glr_vta_get_started_thumb.png b/docs/_images/sphx_glr_vta_get_started_thumb.png
deleted file mode 100644
index 8a5fed589d..0000000000
Binary files a/docs/_images/sphx_glr_vta_get_started_thumb.png and /dev/null differ
diff --git a/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt b/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt
index 293a45ef25..db0ee069aa 100644
--- a/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/deep_dive/relax/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:00.279** total execution time for **deep_dive_relax_tutorials** files:
+**00:00.263** total execution time for **deep_dive_relax_tutorials** files:
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_creation.py` (``relax_creation.py``) | 00:00.190 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_creation.py` (``relax_creation.py``) | 00:00.177 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_transformation.py` (``relax_transformation.py``) | 00:00.088 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_relax_tutorials_relax_transformation.py` (``relax_transformation.py``) | 00:00.086 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt b/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt
index 964f4f8bf3..97badbd93e 100644
--- a/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/deep_dive/tensor_ir/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:00.599** total execution time for **deep_dive_tensor_ir_tutorials** files:
+**00:00.578** total execution time for **deep_dive_tensor_ir_tutorials** files:
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_transformation.py` (``tir_transformation.py``) | 00:00.343 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_transformation.py` (``tir_transformation.py``) | 00:00.339 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_creation.py` (``tir_creation.py``) | 00:00.257 | 0.0 MB |
+| :ref:`sphx_glr_deep_dive_tensor_ir_tutorials_tir_creation.py` (``tir_creation.py``) | 00:00.239 | 0.0 MB |
+-------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/deep_dive/tensor_ir/tutorials/tir_creation.rst.txt b/docs/_sources/deep_dive/tensor_ir/tutorials/tir_creation.rst.txt
index cc5e7d87cd..ab935c8922 100644
--- a/docs/_sources/deep_dive/tensor_ir/tutorials/tir_creation.rst.txt
+++ b/docs/_sources/deep_dive/tensor_ir/tutorials/tir_creation.rst.txt
@@ -325,17 +325,17 @@ Now let's check the runtime dynamic shape inference:
.. code-block:: none
- [[1.0349991 0.46769714 0.64449865 0.7286963 ]
- [1.1301504 0.52516913 0.6791077 0.85188246]
- [0.7237143 0.578327 0.75730014 0.6265181 ]
- [0.9829985 0.6091558 0.49355108 0.6870811 ]]
- [[31.650864 33.627274 31.264925 ... 33.159992 31.85989 30.697412]
- [31.032137 30.348467 29.811312 ... 28.875015 28.613274 28.820261]
- [31.732956 34.129776 33.237835 ... 33.480133 33.00921 32.201576]
+ [[1.0507458 0.83323884 1.1242714 0.9468925 ]
+ [0.7240881 0.7887001 0.97964585 0.7109102 ]
+ [1.1448604 1.1497723 1.9596794 0.9711095 ]
+ [1.6936905 1.7802111 2.3159087 1.5361466 ]]
+ [[32.428276 34.412544 33.17177 ... 31.82429 34.071648 33.186745]
+ [34.203777 34.515644 31.37471 ... 30.777018 32.987213 33.015873]
+ [31.574286 34.46622 32.09642 ... 30.701006 33.221638 32.793884]
...
- [33.6534 34.314987 33.06307 ... 32.318672 31.819223 32.726852]
- [28.399832 30.069996 29.883163 ... 27.921192 28.124653 29.471693]
- [31.149609 32.04736 33.430996 ... 31.032625 29.738329 30.313547]]
+ [32.298668 33.092064 29.534029 ... 29.707777 31.85951 32.498753]
+ [28.152506 30.74788 29.596851 ... 27.94949 31.377981 29.030062]
+ [29.837336 32.96691 32.739925 ... 29.112307 31.560297 30.418003]]
diff --git a/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt b/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt
index fb645fddfb..a5e65457d6 100644
--- a/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt
+++ b/docs/_sources/deep_dive/tensor_ir/tutorials/tir_transformation.rst.txt
@@ -123,7 +123,7 @@ original implementation.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 2.4145 2.4145 2.4145 2.4145 0.0000
+ 2.3107 2.3107 2.3107 2.3107 0.0000
@@ -295,7 +295,7 @@ action involves reordering these two loops.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 0.9504 0.9504 0.9504 0.9504 0.0000
+ 0.9621 0.9621 0.9621 0.9621 0.0000
@@ -423,7 +423,7 @@ from the reduction update via the **decompose_reduction** primitive.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 0.3361 0.3361 0.3361 0.3361 0.0000
+ 0.3358 0.3358 0.3358 0.3358 0.0000
diff --git a/docs/_sources/faq.rst.txt b/docs/_sources/faq.rst.txt
deleted file mode 100644
index 4104c82a12..0000000000
--- a/docs/_sources/faq.rst.txt
+++ /dev/null
@@ -1,69 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-
-Frequently Asked Questions
-==========================
-
-
-How to Install
---------------
-See :ref:`installation`.
-
-
-How to add a new Hardware Backend
----------------------------------
-
-- If the hardware backend has LLVM support,
- then we can directly generate the code by setting the correct target triple as in :py:mod:`~tvm.target`.
-- If the target hardware is a GPU, try to use the cuda, opencl or vulkan backend.
-- If the target hardware is a special accelerator,
- checkout :ref:`vta-index` and :ref:`relay-bring-your-own-codegen`.
-- For all of the above cases, You may want to add target specific
- optimization templates using AutoTVM, see :ref:`tutorials-autotvm-sec`.
-- Besides using LLVM's vectorization, we can also embed micro-kernels to leverage hardware intrinsics,
- see :ref:`tutorials-tensorize`.
-
-
-TVM's relation to Other IR/DSL Projects
----------------------------------------
-There are usually two levels of abstractions of IR in the deep learning systems.
-TensorFlow's XLA and Intel's ngraph both use a computation graph representation.
-This representation is high level, and can be helpful to perform generic optimizations
-such as memory reuse, layout transformation and automatic differentiation.
-
-TVM adopts a low-level representation, that explicitly express the choice of memory
-layout, parallelization pattern, locality and hardware primitives etc.
-This level of IR is closer to directly target hardwares.
-The low-level IR adopts ideas from existing image processing languages like Halide, darkroom
-and loop transformation tools like loopy and polyhedra-based analysis.
-We specifically focus on expressing deep learning workloads (e.g. recurrence),
-optimization for different hardware backends and embedding with frameworks to provide
-end-to-end compilation stack.
-
-
-TVM's relation to libDNN, cuDNN
--------------------------------
-TVM can incorporate these libraries as external calls. One goal of TVM is to be able to
-generate high-performing kernels. We will evolve TVM an incremental manner as
-we learn from the techniques of manual kernel crafting and add these as primitives in DSL.
-See also top for recipes of operators in TVM.
-
-
-Security
---------
-See :ref:`dev-security`
diff --git a/docs/_sources/get_started/tutorials/ir_module.rst.txt b/docs/_sources/get_started/tutorials/ir_module.rst.txt
index 199ef51895..23a54ff618 100644
--- a/docs/_sources/get_started/tutorials/ir_module.rst.txt
+++ b/docs/_sources/get_started/tutorials/ir_module.rst.txt
@@ -698,8 +698,8 @@ We can deploy the IRModule on CPU by specifying the target as ``llvm``.
.. code-block:: none
- [[ 0.2722235 0.3597607 0.05561743 -0.29528138 -0.13545823 -0.11075222
- -0.1564103 0.05459877 -0.1148698 0.04708464]]
+ [[ 0.27592248 0.28434217 0.16162783 -0.14820412 -0.14907646 -0.15991737
+ -0.09785634 0.03884996 -0.15732142 0.04104152]]
@@ -765,8 +765,8 @@ Now we can compile the IRModule on GPU, the similar way as we did on CPU.
.. code-block:: none
- [[ 0.27222347 0.3597607 0.05561745 -0.29528147 -0.13545828 -0.11075222
- -0.15641028 0.0545988 -0.11486976 0.04708462]]
+ [[ 0.27592242 0.28434223 0.16162789 -0.14820413 -0.14907649 -0.15991738
+ -0.09785635 0.03884993 -0.15732145 0.04104149]]
diff --git a/docs/_sources/get_started/tutorials/quick_start.rst.txt b/docs/_sources/get_started/tutorials/quick_start.rst.txt
index 9012817f19..692a4d457b 100644
--- a/docs/_sources/get_started/tutorials/quick_start.rst.txt
+++ b/docs/_sources/get_started/tutorials/quick_start.rst.txt
@@ -230,8 +230,8 @@ different devices.
.. code-block:: none
- [[26167.342 26965.111 24802.63 25524.426 25869.35 24266.36 23954.615
- 24636.719 26092.74 23605.797]]
+ [[23906.58 24324.775 23631.066 24541.99 24140.402 24413.68 23987.445
+ 23273.367 24473.607 23897.77 ]]
diff --git a/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt b/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt
index 7c2a2040ae..335b375c33 100644
--- a/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/get_started/tutorials/sg_execution_times.rst.txt
@@ -5,10 +5,10 @@
Computation times
=================
-**00:02.369** total execution time for **get_started_tutorials** files:
+**00:02.775** total execution time for **get_started_tutorials** files:
+---------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_get_started_tutorials_ir_module.py` (``ir_module.py``) | 00:02.161 | 0.0 MB |
+| :ref:`sphx_glr_get_started_tutorials_ir_module.py` (``ir_module.py``) | 00:02.562 | 0.0 MB |
+---------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_get_started_tutorials_quick_start.py` (``quick_start.py``) | 00:00.208 | 0.0 MB |
+| :ref:`sphx_glr_get_started_tutorials_quick_start.py` (``quick_start.py``) | 00:00.214 | 0.0 MB |
+---------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/compile_models/from_darknet.rst.txt b/docs/_sources/how_to/compile_models/from_darknet.rst.txt
index dbb0eb775d..b0a5ef5b64 100644
--- a/docs/_sources/how_to/compile_models/from_darknet.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_darknet.rst.txt
@@ -318,7 +318,7 @@ The process is no different from other examples.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 39.781 seconds)
+ **Total running time of the script:** ( 1 minutes 39.308 seconds)
.. _sphx_glr_download_how_to_compile_models_from_darknet.py:
diff --git a/docs/_sources/how_to/compile_models/from_keras.rst.txt b/docs/_sources/how_to/compile_models/from_keras.rst.txt
index 6f33beb78b..081816dd0d 100644
--- a/docs/_sources/how_to/compile_models/from_keras.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_keras.rst.txt
@@ -232,7 +232,7 @@ Look up prediction top 1 index in 1000 class synset.
.. code-block:: none
Relay top-1 id: 285, class name: Egyptian cat
- 1/1 [==============================] - ETA: 0s 1/1 [==============================] - 1s 930ms/step
+ 1/1 [==============================] - ETA: 0s 1/1 [==============================] - 1s 923ms/step
Keras top-1 id: 285, class name: Egyptian cat
diff --git a/docs/_sources/how_to/compile_models/from_oneflow.rst.txt b/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
index 3baca86a1a..b20bf82833 100644
--- a/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_oneflow.rst.txt
@@ -121,7 +121,7 @@ Load a pretrained OneFlow model and save model
.. code-block:: none
Downloading: "https://oneflow-public.oss-cn-beijing.aliyuncs.com/model_zoo/flowvision/classification/ResNet/resnet18.zip" to /workspace/.oneflow/flowvision_cache/resnet18.zip
- 0%| | 0.00/41.5M [00:00<?, ?B/s] 24%|##4 | 10.1M/41.5M [00:00<00:00, 106MB/s] 49%|####8 | 20.2M/41.5M [00:00<00:00, 69.5MB/s] 66%|######6 | 27.5M/41.5M [00:00<00:00, 54.2MB/s] 82%|########2 | 34.1M/41.5M [00:00<00:00, 55.6MB/s] 96%|#########5| 39.8M/41.5M [00:00<00:00, 47.2MB/s] 100%|##########| 41.5M/41.5M [00:00<00:00, 52.2MB/s]
+ 0%| | 0.00/41.5M [00:00<?, ?B/s] 25%|##4 | 10.3M/41.5M [00:00<00:00, 108MB/s] 49%|####9 | 20.5M/41.5M [00:00<00:00, 68.9MB/s] 67%|######7 | 27.9M/41.5M [00:00<00:00, 43.5MB/s] 82%|########1 | 33.8M/41.5M [00:00<00:00, 48.0MB/s] 95%|#########4| 39.3M/41.5M [00:01<00:00, 20.3MB/s] 100%|##########| 41.5M/41.5M [00:01<00:00, 29.1MB/s]
diff --git a/docs/_sources/how_to/compile_models/from_paddle.rst.txt b/docs/_sources/how_to/compile_models/from_paddle.rst.txt
index 974641dd2b..42572ea663 100644
--- a/docs/_sources/how_to/compile_models/from_paddle.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_paddle.rst.txt
@@ -209,7 +209,7 @@ Look up prediction top 1 index in 1000 class synset.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 6.022 seconds)
+ **Total running time of the script:** ( 1 minutes 7.950 seconds)
.. _sphx_glr_download_how_to_compile_models_from_paddle.py:
diff --git a/docs/_sources/how_to/compile_models/from_pytorch.rst.txt b/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
index 652420d130..a0a1f36638 100644
--- a/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_pytorch.rst.txt
@@ -101,7 +101,7 @@ Load a pretrained PyTorch model
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet18_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet18_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/resnet18-f37072fd.pth" to /workspace/.cache/torch/hub/checkpoints/resnet18-f37072fd.pth
- 0%| | 0.00/44.7M [00:00<?, ?B/s] 23%|##2 | 10.2M/44.7M [00:00<00:00, 107MB/s] 63%|######2 | 28.0M/44.7M [00:00<00:00, 153MB/s] 100%|##########| 44.7M/44.7M [00:00<00:00, 159MB/s]
+ 0%| | 0.00/44.7M [00:00<?, ?B/s] 38%|###7 | 16.8M/44.7M [00:00<00:00, 175MB/s] 88%|########8 | 39.4M/44.7M [00:00<00:00, 211MB/s] 100%|##########| 44.7M/44.7M [00:00<00:00, 208MB/s]
diff --git a/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt b/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
index 6ea9cb24b5..ee0ec437e0 100644
--- a/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
+++ b/docs/_sources/how_to/compile_models/from_tensorflow.rst.txt
@@ -424,7 +424,7 @@ Run the corresponding model on tensorflow
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 33.423 seconds)
+ **Total running time of the script:** ( 1 minutes 34.595 seconds)
.. _sphx_glr_download_how_to_compile_models_from_tensorflow.py:
diff --git a/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt b/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
index e62e418e7f..566d775ca2 100644
--- a/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/compile_models/sg_execution_times.rst.txt
@@ -5,24 +5,24 @@
Computation times
=================
-**06:57.198** total execution time for **how_to_compile_models** files:
+**07:02.734** total execution time for **how_to_compile_models** files:
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_darknet.py` (``from_darknet.py``) | 01:39.781 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_darknet.py` (``from_darknet.py``) | 01:39.308 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_tensorflow.py` (``from_tensorflow.py``) | 01:33.423 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_tensorflow.py` (``from_tensorflow.py``) | 01:34.595 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_paddle.py` (``from_paddle.py``) | 01:06.022 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_paddle.py` (``from_paddle.py``) | 01:07.950 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_oneflow.py` (``from_oneflow.py``) | 00:45.472 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_oneflow.py` (``from_oneflow.py``) | 00:47.147 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_coreml.py` (``from_coreml.py``) | 00:42.222 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_coreml.py` (``from_coreml.py``) | 00:41.724 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_pytorch.py` (``from_pytorch.py``) | 00:28.456 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_pytorch.py` (``from_pytorch.py``) | 00:29.926 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_keras.py` (``from_keras.py``) | 00:26.135 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_keras.py` (``from_keras.py``) | 00:27.079 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_tflite.py` (``from_tflite.py``) | 00:12.724 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_tflite.py` (``from_tflite.py``) | 00:12.036 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_compile_models_from_onnx.py` (``from_onnx.py``) | 00:02.963 | 0.0 MB |
+| :ref:`sphx_glr_how_to_compile_models_from_onnx.py` (``from_onnx.py``) | 00:02.968 | 0.0 MB |
+-----------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
index 9cc27d4cd7..48b8047cc1 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno.rst.txt
@@ -664,7 +664,7 @@ well as provides information about the model's performance
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 4124.1030 4123.4906 4131.1127 4122.2636 2.4116
+ 4119.6973 4119.4336 4126.6300 4116.2098 2.8146
@@ -672,7 +672,7 @@ well as provides information about the model's performance
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 22.714 seconds)
+ **Total running time of the script:** ( 1 minutes 21.962 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_model_on_adreno.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
index c6bef2afe4..6443f59ee1 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_adreno_tvmc.rst.txt
@@ -142,7 +142,7 @@ Make a PyTorch Resnet50 Model
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet50_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet50_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/resnet50-0676ba61.pth" to /workspace/.cache/torch/hub/checkpoints/resnet50-0676ba61.pth
- 0%| | 0.00/97.8M [00:00<?, ?B/s] 16%|#6 | 15.8M/97.8M [00:00<00:00, 165MB/s] 38%|###8 | 37.5M/97.8M [00:00<00:00, 201MB/s] 61%|###### | 59.5M/97.8M [00:00<00:00, 215MB/s] 84%|########3 | 81.9M/97.8M [00:00<00:00, 222MB/s] 100%|##########| 97.8M/97.8M [00:00<00:00, 217MB/s]
+ 0%| | 0.00/97.8M [00:00<?, ?B/s] 14%|#4 | 13.9M/97.8M [00:00<00:00, 145MB/s] 35%|###4 | 33.8M/97.8M [00:00<00:00, 182MB/s] 55%|#####4 | 53.4M/97.8M [00:00<00:00, 193MB/s] 75%|#######4 | 73.1M/97.8M [00:00<00:00, 199MB/s] 95%|#########4| 92.9M/97.8M [00:00<00:00, 201MB/s] 100%|##########| 97.8M/97.8M [00:00<00:00, 195MB/s]
@@ -677,7 +677,7 @@ Let tvmc fill inputs using random
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 6.369 seconds)
+ **Total running time of the script:** ( 1 minutes 6.421 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_model_on_adreno_tvmc.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt b/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
index 5f4a874b74..b5a7f217ba 100644
--- a/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_model_on_android.rst.txt
@@ -437,7 +437,7 @@ Execute on TVM
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 15.3608 15.3703 15.7223 14.9183 0.2214
+ 14.9539 14.9309 15.1575 14.8359 0.0980
diff --git a/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt b/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
index 927c172a52..327bff2b04 100644
--- a/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_object_detection_pytorch.rst.txt
@@ -130,7 +130,7 @@ Load pre-trained maskrcnn from torchvision and do tracing
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MaskRCNN_ResNet50_FPN_Weights.COCO_V1`. You can also use `weights=MaskRCNN_ResNet50_FPN_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth" to /workspace/.cache/torch/hub/checkpoints/maskrcnn_resnet50_fpn_coco-bf2d0c1e.pth
- 0%| | 0.00/170M [00:00<?, ?B/s] 9%|8 | 14.9M/170M [00:00<00:01, 155MB/s] 22%|##1 | 36.6M/170M [00:00<00:00, 198MB/s] 34%|###4 | 58.2M/170M [00:00<00:00, 211MB/s] 47%|####7 | 80.0M/170M [00:00<00:00, 217MB/s] 60%|#####9 | 102M/170M [00:00<00:00, 220MB/s] 72%|#######2 | 123M/170M [00:00<00:00, 222MB/s] 85%|########5 | 144M/170M [00:00<00:00, 222MB/s] 98%|#########7| 166M/170M [00:00<00:00, 223MB/s] 100%|##########| 170M/170M [00:00<00:00, 217MB/s]
+ 0%| | 0.00/170M [00:00<?, ?B/s] 9%|8 | 14.5M/170M [00:00<00:01, 151MB/s] 20%|## | 34.2M/170M [00:00<00:00, 184MB/s] 32%|###1 | 54.0M/170M [00:00<00:00, 194MB/s] 43%|####3 | 73.2M/170M [00:00<00:00, 197MB/s] 55%|#####4 | 92.6M/170M [00:00<00:00, 199MB/s] 66%|######6 | 112M/170M [00:00<00:00, 201MB/s] 78%|#######7 | 132M/170M [00:00<00:00, 202MB/s] 89%|########9 | 152M/170M [00:00<00:00, 203MB/s] 100%|##########| 170M/170M [00:00<00:00, 199MB/s]
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/detection/generalized_rcnn.py:75: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
for img in images:
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/detection/transform.py:122: TracerWarning: Iterating over a tensor might cause the trace to be incorrect. Passing a tensor of different shape won't change the number of iterations executed (and might lead to errors or silently give incorrect results).
@@ -330,7 +330,7 @@ Get boxes with score larger than 0.9
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 3 minutes 49.344 seconds)
+ **Total running time of the script:** ( 3 minutes 47.735 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_object_detection_pytorch.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt b/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
index a27ad5db0a..5d227a10f0 100644
--- a/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_prequantized.rst.txt
@@ -227,7 +227,7 @@ training. Other models require a full post training calibration.
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=MobileNet_V2_Weights.IMAGENET1K_V1`. You can also use `weights=MobileNet_V2_Weights.DEFAULT` to get the most up-to-date weights.
warnings.warn(msg)
Downloading: "https://download.pytorch.org/models/mobilenet_v2-b0353104.pth" to /workspace/.cache/torch/hub/checkpoints/mobilenet_v2-b0353104.pth
- 0%| | 0.00/13.6M [00:00<?, ?B/s] 75%|#######4 | 10.1M/13.6M [00:00<00:00, 106MB/s] 100%|##########| 13.6M/13.6M [00:00<00:00, 119MB/s]
+ 0%| | 0.00/13.6M [00:00<?, ?B/s] 90%|######### | 12.2M/13.6M [00:00<00:00, 128MB/s] 100%|##########| 13.6M/13.6M [00:00<00:00, 133MB/s]
@@ -404,7 +404,7 @@ Here we give an example of how to measure performance of TVM compiled models.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 89.2456 89.2001 90.0784 88.5770 0.3112
+ 89.3390 89.3392 90.6846 88.8456 0.3079
@@ -452,7 +452,7 @@ TODO
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 36.254 seconds)
+ **Total running time of the script:** ( 1 minutes 36.195 seconds)
.. _sphx_glr_download_how_to_deploy_models_deploy_prequantized.py:
diff --git a/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt b/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
index dd250cabd4..2948f61c58 100644
--- a/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
+++ b/docs/_sources/how_to/deploy_models/deploy_prequantized_tflite.rst.txt
@@ -423,7 +423,7 @@ Here we give an example of how to measure performance of TVM compiled models.
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 110.8661 110.8157 115.1778 110.1944 0.5290
+ 111.7438 111.7453 115.6274 110.9298 0.5324
diff --git a/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt b/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
index f42fd7ab7c..fa737566ef 100644
--- a/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/deploy_models/sg_execution_times.rst.txt
@@ -5,24 +5,24 @@
Computation times
=================
-**10:49.155** total execution time for **how_to_deploy_models** files:
+**10:46.290** total execution time for **how_to_deploy_models** files:
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_object_detection_pytorch.py` (``deploy_object_detection_pytorch.py``) | 03:49.344 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_object_detection_pytorch.py` (``deploy_object_detection_pytorch.py``) | 03:47.735 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized.py` (``deploy_prequantized.py``) | 01:36.254 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized.py` (``deploy_prequantized.py``) | 01:36.195 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno.py` (``deploy_model_on_adreno.py``) | 01:22.714 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno.py` (``deploy_model_on_adreno.py``) | 01:21.962 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno_tvmc.py` (``deploy_model_on_adreno_tvmc.py``) | 01:06.369 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_adreno_tvmc.py` (``deploy_model_on_adreno_tvmc.py``) | 01:06.421 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_android.py` (``deploy_model_on_android.py``) | 00:56.708 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_android.py` (``deploy_model_on_android.py``) | 00:55.657 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized_tflite.py` (``deploy_prequantized_tflite.py``) | 00:53.727 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_prequantized_tflite.py` (``deploy_prequantized_tflite.py``) | 00:54.192 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_rasp.py` (``deploy_model_on_rasp.py``) | 00:32.063 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_nano.py` (``deploy_model_on_nano.py``) | 00:32.238 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_nano.py` (``deploy_model_on_nano.py``) | 00:31.969 | 0.0 MB |
+| :ref:`sphx_glr_how_to_deploy_models_deploy_model_on_rasp.py` (``deploy_model_on_rasp.py``) | 00:31.882 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_deploy_models_deploy_sparse.py` (``deploy_sparse.py``) | 00:00.008 | 0.0 MB |
+------------------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt b/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
index ad3543bde0..f26105dfbf 100644
--- a/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:04.335** total execution time for **how_to_extend_tvm** files:
+**00:04.237** total execution time for **how_to_extend_tvm** files:
+-------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_use_pass_instrument.py` (``use_pass_instrument.py``) | 00:03.004 | 0.0 MB |
+| :ref:`sphx_glr_how_to_extend_tvm_use_pass_instrument.py` (``use_pass_instrument.py``) | 00:02.934 | 0.0 MB |
+-------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_extend_tvm_use_pass_infra.py` (``use_pass_infra.py``) | 00:01.323 | 0.0 MB |
+| :ref:`sphx_glr_how_to_extend_tvm_use_pass_infra.py` (``use_pass_infra.py``) | 00:01.295 | 0.0 MB |
+-------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_extend_tvm_low_level_custom_pass.py` (``low_level_custom_pass.py``) | 00:00.008 | 0.0 MB |
+-------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt b/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
index 83243b2671..81383539f8 100644
--- a/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
+++ b/docs/_sources/how_to/extend_tvm/use_pass_instrument.rst.txt
@@ -220,10 +220,10 @@ profile the execution time of each passes.
.. code-block:: none
Printing results of timing profile...
- InferType: 25366us [25366us] (48.26%; 48.26%)
- FoldScaleAxis: 27191us [11us] (51.74%; 51.74%)
- FoldConstant: 27179us [1856us] (51.71%; 99.96%)
- InferType: 25324us [25324us] (48.18%; 93.17%)
+ InferType: 25041us [25041us] (48.57%; 48.57%)
+ FoldScaleAxis: 26514us [10us] (51.43%; 51.43%)
+ FoldConstant: 26504us [1840us] (51.41%; 99.96%)
+ InferType: 24664us [24664us] (47.84%; 93.06%)
@@ -262,10 +262,10 @@ Refer to following sections and :py:func:`tvm.instrument.pass_instrument` for th
.. code-block:: none
Printing results of timing profile...
- InferType: 25571us [25571us] (48.60%; 48.60%)
- FoldScaleAxis: 27048us [11us] (51.40%; 51.40%)
- FoldConstant: 27037us [1931us] (51.38%; 99.96%)
- InferType: 25106us [25106us] (47.71%; 92.86%)
+ InferType: 24517us [24517us] (48.31%; 48.31%)
+ FoldScaleAxis: 26236us [8us] (51.69%; 51.69%)
+ FoldConstant: 26228us [1805us] (51.68%; 99.97%)
+ InferType: 24422us [24422us] (48.12%; 93.12%)
diff --git a/docs/_sources/how_to/legacy_index.rst.txt b/docs/_sources/how_to/legacy_index.rst.txt
index a98e04c969..d675adbee2 100644
--- a/docs/_sources/how_to/legacy_index.rst.txt
+++ b/docs/_sources/how_to/legacy_index.rst.txt
@@ -35,4 +35,3 @@ schedule with tesor expressions?"
work_with_microtvm/index
extend_tvm/index
profile/index
- ../faq
diff --git a/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt b/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
index a1a9dd448f..1d09c5a842 100644
--- a/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_conv_cuda.rst.txt
@@ -331,7 +331,7 @@ latency of convolution.
.. code-block:: none
- Convolution: 41.744705 ms
+ Convolution: 43.502113 ms
diff --git a/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt b/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
index ce5787ef18..a4ef8ab79d 100644
--- a/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_conv_tensorcore.rst.txt
@@ -598,7 +598,7 @@ be able to run on our build server
.. code-block:: none
- conv2d with tensor core: 12.219805 ms
+ conv2d with tensor core: 12.221417 ms
diff --git a/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt b/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
index 39ba7de3de..36bac00dc5 100644
--- a/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/opt_gemm.rst.txt
@@ -134,8 +134,8 @@ Then we write a baseline implementation, the simplest way to write a matrix mult
.. code-block:: none
- Numpy running time: 0.018924
- Baseline: 3.411686
+ Numpy running time: 0.018746
+ Baseline: 3.306999
@@ -227,7 +227,7 @@ fill 32 * 32 * sizeof(float) which is 4KB in the cache whose total size is 32KB
.. code-block:: none
- Opt1: 0.311584
+ Opt1: 0.308203
@@ -318,7 +318,7 @@ In this tutorial, we chose to vectorize the inner loop row data since it is cach
.. code-block:: none
- Opt2: 0.297810
+ Opt2: 0.278769
@@ -406,7 +406,7 @@ the access pattern for A matrix is more cache friendly.
.. code-block:: none
- Opt3: 0.119894
+ Opt3: 0.117252
@@ -523,7 +523,7 @@ flattening.
.. code-block:: none
- Opt4: 0.107438
+ Opt4: 0.108845
@@ -635,7 +635,7 @@ write to C when all the block results are ready.
.. code-block:: none
- Opt5: 0.111990
+ Opt5: 0.113520
@@ -748,7 +748,7 @@ Furthermore, we can also utilize multi-core processors to do the thread-level pa
.. code-block:: none
- Opt6: 0.134311
+ Opt6: 0.134244
diff --git a/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt b/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
index bf6e59e51a..cf394d18de 100644
--- a/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/optimize_operators/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:34.878** total execution time for **how_to_optimize_operators** files:
+**00:34.234** total execution time for **how_to_optimize_operators** files:
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_gemm.py` (``opt_gemm.py``) | 00:31.567 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_gemm.py` (``opt_gemm.py``) | 00:30.893 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_tensorcore.py` (``opt_conv_tensorcore.py``) | 00:02.092 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_tensorcore.py` (``opt_conv_tensorcore.py``) | 00:02.113 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_cuda.py` (``opt_conv_cuda.py``) | 00:01.219 | 0.0 MB |
+| :ref:`sphx_glr_how_to_optimize_operators_opt_conv_cuda.py` (``opt_conv_cuda.py``) | 00:01.228 | 0.0 MB |
+-----------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
index 21cc353f88..e4d7d54f49 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/sg_execution_times.rst.txt
@@ -5,18 +5,18 @@
Computation times
=================
-**03:52.229** total execution time for **how_to_tune_with_autoscheduler** files:
+**03:50.268** total execution time for **how_to_tune_with_autoscheduler** files:
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_x86.py` (``tune_network_x86.py``) | 01:41.147 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_x86.py` (``tune_network_x86.py``) | 01:40.437 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_cuda.py` (``tune_network_cuda.py``) | 01:15.920 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_cuda.py` (``tune_network_cuda.py``) | 01:14.990 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_arm.py` (``tune_network_arm.py``) | 00:21.398 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_arm.py` (``tune_network_arm.py``) | 00:21.049 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_mali.py` (``tune_network_mali.py``) | 00:17.968 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_network_mali.py` (``tune_network_mali.py``) | 00:17.894 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_conv2d_layer_cuda.py` (``tune_conv2d_layer_cuda.py``) | 00:15.692 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_conv2d_layer_cuda.py` (``tune_conv2d_layer_cuda.py``) | 00:15.794 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_sparse_x86.py` (``tune_sparse_x86.py``) | 00:00.103 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autoscheduler_tune_sparse_x86.py` (``tune_sparse_x86.py``) | 00:00.104 | 0.0 MB |
+----------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
index 2aec905206..a1bc1c6851 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_cuda.rst.txt
@@ -632,7 +632,7 @@ so we can read the log file and load the best schedules.
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 3.1975 3.1974 3.1979 3.1972 0.0003
+ 3.2062 3.2071 3.2084 3.2031 0.0023
@@ -659,7 +659,7 @@ Other Tips
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 15.920 seconds)
+ **Total running time of the script:** ( 1 minutes 14.990 seconds)
.. _sphx_glr_download_how_to_tune_with_autoscheduler_tune_network_cuda.py:
diff --git a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
index 4c0a266cf4..fbf89fed15 100644
--- a/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
+++ b/docs/_sources/how_to/tune_with_autoscheduler/tune_network_x86.rst.txt
@@ -654,7 +654,7 @@ so we can read the log file and load the best schedules.
Evaluate inference time cost...
Execution time summary:
mean (ms) median (ms) max (ms) min (ms) std (ms)
- 761.1314 760.8683 762.3203 760.2056 0.8832
+ 761.1109 761.3337 761.4163 760.5828 0.3750
@@ -681,7 +681,7 @@ Other Tips
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 41.147 seconds)
+ **Total running time of the script:** ( 1 minutes 40.437 seconds)
.. _sphx_glr_download_how_to_tune_with_autoscheduler_tune_network_x86.py:
diff --git a/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt b/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
index 2ea49316c9..46aae6485e 100644
--- a/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
Computation times
=================
-**00:23.847** total execution time for **how_to_tune_with_autotvm** files:
+**00:23.864** total execution time for **how_to_tune_with_autotvm** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_conv2d_cuda.py` (``tune_conv2d_cuda.py``) | 00:23.810 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_conv2d_cuda.py` (``tune_conv2d_cuda.py``) | 00:23.826 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_x86.py` (``tune_relay_x86.py``) | 00:00.020 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_x86.py` (``tune_relay_x86.py``) | 00:00.021 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_tune_with_autotvm_tune_relay_mobile_gpu.py` (``tune_relay_mobile_gpu.py``) | 00:00.006 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt b/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
index bdc36a0c64..7b6aa25ba7 100644
--- a/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
+++ b/docs/_sources/how_to/tune_with_autotvm/tune_conv2d_cuda.rst.txt
@@ -326,7 +326,7 @@ and measure running time.
Best config:
,None
- Time cost of this operator: 0.037111
+ Time cost of this operator: 0.037197
diff --git a/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt b/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt
index 9cca55f9c4..449d656c62 100644
--- a/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt
+++ b/docs/_sources/how_to/tutorials/cross_compilation_and_rpc.rst.txt
@@ -274,7 +274,7 @@ device and returns the measured cost. Network overhead is excluded.
.. code-block:: none
- 1.219e-07 secs/op
+ 1.293e-07 secs/op
diff --git a/docs/_sources/how_to/tutorials/customize_opt.rst.txt b/docs/_sources/how_to/tutorials/customize_opt.rst.txt
index 740f428343..7527ccd15e 100644
--- a/docs/_sources/how_to/tutorials/customize_opt.rst.txt
+++ b/docs/_sources/how_to/tutorials/customize_opt.rst.txt
@@ -420,8 +420,8 @@ We can build and deploy the optimized model to the TVM runtime.
.. code-block:: none
- [[26195.57 25223.076 24700.54 25951.34 24422.746 25216.812 24859.023
- 24812.828 25638.484 24069.904]]
+ [[24775.57 26398.43 24994.54 24746.398 25497.479 25316.045 26465.264
+ 23891.504 26020.29 25203.572]]
diff --git a/docs/_sources/how_to/tutorials/optimize_llm.rst.txt b/docs/_sources/how_to/tutorials/optimize_llm.rst.txt
index 15a0291b03..72ab791fb5 100644
--- a/docs/_sources/how_to/tutorials/optimize_llm.rst.txt
+++ b/docs/_sources/how_to/tutorials/optimize_llm.rst.txt
@@ -858,7 +858,7 @@ IRModule to generate the token.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 12.283 seconds)
+ **Total running time of the script:** ( 1 minutes 12.653 seconds)
.. _sphx_glr_download_how_to_tutorials_optimize_llm.py:
diff --git a/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt b/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt
index dc5deb01dd..91ba29fdd3 100644
--- a/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/tutorials/sg_execution_times.rst.txt
@@ -5,14 +5,14 @@
Computation times
=================
-**01:16.580** total execution time for **how_to_tutorials** files:
+**01:16.942** total execution time for **how_to_tutorials** files:
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_optimize_llm.py` (``optimize_llm.py``) | 01:12.283 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_optimize_llm.py` (``optimize_llm.py``) | 01:12.653 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_e2e_opt_model.py` (``e2e_opt_model.py``) | 00:02.626 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_e2e_opt_model.py` (``e2e_opt_model.py``) | 00:02.657 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_customize_opt.py` (``customize_opt.py``) | 00:00.848 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_customize_opt.py` (``customize_opt.py``) | 00:00.841 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_tutorials_cross_compilation_and_rpc.py` (``cross_compilation_and_rpc.py``) | 00:00.822 | 0.0 MB |
+| :ref:`sphx_glr_how_to_tutorials_cross_compilation_and_rpc.py` (``cross_compilation_and_rpc.py``) | 00:00.791 | 0.0 MB |
+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
index 6537e3fbf3..fd2d87d1b8 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_autotune.rst.txt
@@ -360,10 +360,10 @@ Timing the untuned program
########## Build without Autotuning ##########
Node Name Ops Time(us) Time(%) Shape Inputs Outputs Measurements(us)
--------- --- -------- ------- ----- ------ ------- ----------------
- tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 304.2 98.744 (1, 2, 10, 10, 3) 2 1 [304.2]
- tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 2.896 0.94 (1, 6, 10, 10) 1 1 [2.896]
- tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.972 0.316 (1, 1, 10, 10, 3) 1 1 [0.972]
- Total_time - 308.068 - - - - -
+ tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 306.6 98.639 (1, 2, 10, 10, 3) 2 1 [306.6]
+ tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 2.905 0.935 (1, 6, 10, 10) 1 1 [2.905]
+ tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 1.327 0.427 (1, 1, 10, 10, 3) 1 1 [1.327]
+ Total_time - 310.832 - - - - -
@@ -428,10 +428,10 @@ Timing the tuned program
########## Build with Autotuning ##########
Node Name Ops Time(us) Time(%) Shape Inputs Outputs Measurements(us)
--------- --- -------- ------- ----- ------ ------- ----------------
- tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 103.0 97.522 (1, 6, 10, 10, 1) 2 1 [103.0]
- tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 1.773 1.679 (1, 6, 10, 10) 1 1 [1.773]
- tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.844 0.799 (1, 3, 10, 10, 1) 1 1 [0.844]
- Total_time - 105.617 - - - - -
+ tvmgen_default_fused_nn_contrib_conv2d_NCHWc tvmgen_default_fused_nn_contrib_conv2d_NCHWc 102.1 97.492 (1, 6, 10, 10, 1) 2 1 [102.1]
+ tvmgen_default_fused_layout_transform_1 tvmgen_default_fused_layout_transform_1 1.781 1.701 (1, 6, 10, 10) 1 1 [1.781]
+ tvmgen_default_fused_layout_transform tvmgen_default_fused_layout_transform 0.845 0.807 (1, 3, 10, 10, 1) 1 1 [0.845]
+ Total_time - 104.726 - - - - -
@@ -439,7 +439,7 @@ Timing the tuned program
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 28.325 seconds)
+ **Total running time of the script:** ( 1 minutes 26.537 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_autotune.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
index 47c5ee2c3a..12aa11a10d 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_pytorch.rst.txt
@@ -118,7 +118,7 @@ download a cat image and preprocess it to use as the model input.
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/quantization/utils.py:376: UserWarning: must run observer before calling calculate_qparams. Returning default values.
warnings.warn(
Downloading: "https://download.pytorch.org/models/quantized/mobilenet_v2_qnnpack_37f702c5.pth" to /workspace/.cache/torch/hub/checkpoints/mobilenet_v2_qnnpack_37f702c5.pth
- 0%| | 0.00/3.42M [00:00<?, ?B/s] 100%|##########| 3.42M/3.42M [00:00<00:00, 81.5MB/s]
+ 0%| | 0.00/3.42M [00:00<?, ?B/s] 100%|##########| 3.42M/3.42M [00:00<00:00, 81.4MB/s]
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/nn/quantized/modules/__init__.py:98: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
return torch.quantize_per_tensor(X, float(self.scale),
/venv/apache-tvm-py3.8/lib/python3.8/site-packages/torch/ao/nn/quantized/modules/__init__.py:99: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
@@ -324,7 +324,7 @@ Look up prediction top 1 index in 1000 class synset.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 34.307 seconds)
+ **Total running time of the script:** ( 1 minutes 34.454 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_pytorch.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt b/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
index 2fa28793a0..8578da2da6 100644
--- a/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/micro_train.rst.txt
@@ -218,7 +218,7 @@ take about **2 minutes** to download the Stanford Cars, while COCO 2017 validati
.. code-block:: none
- '/tmp/tmpkgxwqq69/images/random'
+ '/tmp/tmpa8cpm8lt/images/random'
@@ -318,8 +318,8 @@ objects to other stuff? We can display some examples from our datasets using ``m
.. code-block:: none
- /tmp/tmpkgxwqq69/images/target contains 8144 images
- /tmp/tmpkgxwqq69/images/random contains 5000 images
+ /tmp/tmpa8cpm8lt/images/target contains 8144 images
+ /tmp/tmpa8cpm8lt/images/random contains 5000 images
@@ -494,13 +494,13 @@ the time on our validation set).
.. code-block:: none
Epoch 1/3
- 328/328 - 47s - loss: 0.2237 - accuracy: 0.9226 - val_loss: 0.1605 - val_accuracy: 0.9471 - 47s/epoch - 144ms/step
+ 328/328 - 47s - loss: 0.2161 - accuracy: 0.9280 - val_loss: 0.1428 - val_accuracy: 0.9494 - 47s/epoch - 144ms/step
Epoch 2/3
- 328/328 - 44s - loss: 0.1044 - accuracy: 0.9614 - val_loss: 0.1071 - val_accuracy: 0.9615 - 44s/epoch - 135ms/step
+ 328/328 - 44s - loss: 0.0989 - accuracy: 0.9635 - val_loss: 0.1099 - val_accuracy: 0.9653 - 44s/epoch - 134ms/step
Epoch 3/3
- 328/328 - 44s - loss: 0.0687 - accuracy: 0.9748 - val_loss: 0.0962 - val_accuracy: 0.9702 - 44s/epoch - 134ms/step
+ 328/328 - 44s - loss: 0.0721 - accuracy: 0.9727 - val_loss: 0.1520 - val_accuracy: 0.9517 - 44s/epoch - 133ms/step
- <keras.callbacks.History object at 0x7f812d0d9850>
+ <keras.callbacks.History object at 0x7f8f03fd6e80>
@@ -861,7 +861,7 @@ Arduino tutorial for how to do that `on GitHub <https://github.com/guberti/tvm-a
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 4 minutes 48.029 seconds)
+ **Total running time of the script:** ( 4 minutes 45.338 seconds)
.. _sphx_glr_download_how_to_work_with_microtvm_micro_train.py:
diff --git a/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
index d6b3ae357b..f2f9939349 100644
--- a/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_microtvm/sg_execution_times.rst.txt
@@ -5,20 +5,20 @@
Computation times
=================
-**08:22.603** total execution time for **how_to_work_with_microtvm** files:
+**08:17.857** total execution time for **how_to_work_with_microtvm** files:
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_train.py` (``micro_train.py``) | 04:48.029 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_train.py` (``micro_train.py``) | 04:45.338 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_pytorch.py` (``micro_pytorch.py``) | 01:34.307 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_pytorch.py` (``micro_pytorch.py``) | 01:34.454 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_autotune.py` (``micro_autotune.py``) | 01:28.325 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_autotune.py` (``micro_autotune.py``) | 01:26.537 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_aot.py` (``micro_aot.py``) | 00:12.474 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_aot.py` (``micro_aot.py``) | 00:12.416 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_custom_ide.py` (``micro_custom_ide.py``) | 00:10.975 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_custom_ide.py` (``micro_custom_ide.py``) | 00:10.802 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_microtvm_micro_tflite.py` (``micro_tflite.py``) | 00:08.493 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_microtvm_micro_tflite.py` (``micro_tflite.py``) | 00:08.309 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_work_with_microtvm_micro_ethosu.py` (``micro_ethosu.py``) | 00:00.000 | 0.0 MB |
+-----------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
index 64569f73fa..61e0605000 100644
--- a/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_relay/sg_execution_times.rst.txt
@@ -5,14 +5,14 @@
Computation times
=================
-**00:41.582** total execution time for **how_to_work_with_relay** files:
+**00:40.890** total execution time for **how_to_work_with_relay** files:
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_pipeline_executor.py` (``using_pipeline_executor.py``) | 00:35.573 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_pipeline_executor.py` (``using_pipeline_executor.py``) | 00:34.973 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_using_external_lib.py` (``using_external_lib.py``) | 00:04.015 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_using_external_lib.py` (``using_external_lib.py``) | 00:03.873 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_relay_build_gcn.py` (``build_gcn.py``) | 00:01.987 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_relay_build_gcn.py` (``build_gcn.py``) | 00:02.038 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_how_to_work_with_relay_using_relay_viz.py` (``using_relay_viz.py``) | 00:00.006 | 0.0 MB |
+----------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt b/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
index a97711fa75..fde9b2be61 100644
--- a/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
+++ b/docs/_sources/how_to/work_with_schedules/sg_execution_times.rst.txt
@@ -5,20 +5,20 @@
Computation times
=================
-**00:03.285** total execution time for **how_to_work_with_schedules** files:
+**00:03.031** total execution time for **how_to_work_with_schedules** files:
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tensorize.py` (``tensorize.py``) | 00:01.544 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tensorize.py` (``tensorize.py``) | 00:01.326 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_reduction.py` (``reduction.py``) | 00:00.717 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_reduction.py` (``reduction.py``) | 00:00.704 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_scan.py` (``scan.py``) | 00:00.715 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_scan.py` (``scan.py``) | 00:00.696 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_extern_op.py` (``extern_op.py``) | 00:00.137 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_extern_op.py` (``extern_op.py``) | 00:00.132 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_schedule_primitives.py` (``schedule_primitives.py``) | 00:00.072 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_schedule_primitives.py` (``schedule_primitives.py``) | 00:00.071 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tedd.py` (``tedd.py``) | 00:00.067 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tedd.py` (``tedd.py``) | 00:00.068 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_how_to_work_with_schedules_tuple_inputs.py` (``tuple_inputs.py``) | 00:00.033 | 0.0 MB |
+| :ref:`sphx_glr_how_to_work_with_schedules_tuple_inputs.py` (``tuple_inputs.py``) | 00:00.034 | 0.0 MB |
+------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/how_to/work_with_schedules/tensorize.rst.txt b/docs/_sources/how_to/work_with_schedules/tensorize.rst.txt
index e6f2f8d05c..212c3f5603 100644
--- a/docs/_sources/how_to/work_with_schedules/tensorize.rst.txt
+++ b/docs/_sources/how_to/work_with_schedules/tensorize.rst.txt
@@ -528,7 +528,7 @@ Tensorize for squared GEMV, build and check the results,
-.. GENERATED FROM PYTHON SOURCE LINES 309-320
+.. GENERATED FROM PYTHON SOURCE LINES 309-318
Summary
-------
@@ -536,8 +536,6 @@ This tutorial demonstrates the usage of tensorize intrinsic in TVM.
Tensorize provides a way for users to get fully optimized schedule via micro-kernels.
For example, INT8 quantization on Intel CPUs uses tensorization
to invoke AVX instruction directly.
-It also enables TVM to compile to ASICs -
-checkout :ref:`vta-index` for details.
We also demonstrates how to use inline assembly importing,
which helps users inject asm easily into the schedule.
diff --git a/docs/_sources/index.rst.txt b/docs/_sources/index.rst.txt
index 3abc39e82f..041931552b 100644
--- a/docs/_sources/index.rst.txt
+++ b/docs/_sources/index.rst.txt
@@ -75,7 +75,6 @@ driving its costs down.
dev/how_to/how_to.rst
reference/langref/index
topic/microtvm/index
- topic/vta/index
.. toctree::
:maxdepth: 1
diff --git a/docs/_sources/reference/langref/relay_pattern.rst.txt b/docs/_sources/reference/langref/relay_pattern.rst.txt
index 16211b2cb1..a80c55323b 100644
--- a/docs/_sources/reference/langref/relay_pattern.rst.txt
+++ b/docs/_sources/reference/langref/relay_pattern.rst.txt
@@ -20,7 +20,7 @@
Pattern Matching in Relay
=========================
-There are many places in TVM where we identify pure data-flow sub-graphs of the Relay program and attempt to transform them in some way example passes include fusion, quantization, external code generation, and device specific optimizations such as bitpacking, and layer slicing used by VTA.
+There are many places in TVM where we identify pure data-flow sub-graphs of the Relay program and attempt to transform them in some way example passes include fusion, quantization, external code generation, and device specific optimizations.
Many of these passes today require a lots of boring boilerplate code in order to implement as well as requiring users to think in terms of visitors and AST matching. Many of these transformations can easily be described in terms of graph rewrites. In order to build a rewriter or other advanced machinery we first need a language of patterns to describe what we can match.
diff --git a/docs/_sources/topic/vta/dev/config.rst.txt b/docs/_sources/topic/vta/dev/config.rst.txt
deleted file mode 100644
index b3ec49e769..0000000000
--- a/docs/_sources/topic/vta/dev/config.rst.txt
+++ /dev/null
@@ -1,73 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-VTA Configuration
-=================
-
-The VTA stack incorporates both a hardware accelerator stack and
-a TVM based software stack.
-VTA incorporates flexibility out of the box: by modifying the
-``3rdparty/vta-hw/config/vta_config.json`` high-level configuration file,
-the user can change the shape of the tensor intrinsic,
-clock frequency, pipelining, data type width, and on-chip buffer sizes.
-
-Parameters Overview
--------------------
-
-We explain the parameters listed in the ``vta_config.json`` file in the table
-below.
-
-+-----------------------+------------+--------------------------------------------------------+
-| Attribute | Format | Description |
-+=======================+============+========================================================+
-| ``TARGET`` | String | The TVM device target. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``HW_VER`` | String | VTA hardware version number. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_INP_WIDTH`` | Int (log2) | Input data type signed integer width. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_WGT_WIDTH`` | Int (log2) | Weight data type signed integer width. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_ACC_WIDTH`` | Int (log2) | Accumulator data type signed integer width. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_BATCH`` | Int (log2) | VTA matrix multiply intrinsic input/output dimension 0.|
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_BLOCK`` | Int (log2) | VTA matrix multiply inner dimensions. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_UOP_BUFF_SIZE`` | Int (log2) | Micro-op on-chip buffer in Bytes. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_INP_BUFF_SIZE`` | Int (log2) | Input on-chip buffer in Bytes. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_WGT_BUFF_SIZE`` | Int (log2) | Weight on-chip buffer in Bytes. |
-+-----------------------+------------+--------------------------------------------------------+
-| ``LOG_ACC_BUFF_SIZE`` | Int (log2) | Accumulator on-chip buffer in Bytes. |
-+-----------------------+------------+--------------------------------------------------------+
-
-
- .. note::
-
- When a parameter name is preceded with ``LOG``, it means that it describes a value that can only be expressed a power of two.
- For that reason we describe these parameters by their log2 value.
- For instance, to describe an integer width of 8-bits for the input data types, we set the ``LOG_INP_WIDTH`` to be 3, which is the log2 of 8.
- Similarly, to descibe a 64kB micro-op buffer, we would set ``LOG_UOP_BUFF_SIZE`` to be 16.
-
-We provide additional detail below regarding each parameter:
-
- - ``TARGET``: Can be set to ``"pynq"``, ``"ultra96"``, ``"sim"`` (fast simulator), or ``"tsim"`` (cycle accurate sim with verilator).
- - ``HW_VER``: Hardware version which increments every time the VTA hardware design changes. This parameter is used to uniquely identity hardware bitstreams.
- - ``LOG_BATCH``: Equivalent to A in multiplication of shape (A, B) x (B, C), or typically, the batch dimension of inner tensor computation.
- - ``LOG_BLOCK``: Equivalent to B and C in multiplication of shape (A, B) x (B, C), or typically, the input/output channel dimensions of the inner tensor computation.
diff --git a/docs/_sources/topic/vta/dev/hardware.rst.txt b/docs/_sources/topic/vta/dev/hardware.rst.txt
deleted file mode 100644
index 8251278994..0000000000
--- a/docs/_sources/topic/vta/dev/hardware.rst.txt
+++ /dev/null
@@ -1,298 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-VTA Hardware Guide
-==================
-
-We present a top-down overview of the VTA hardware design.
-This hardware design guide covers VTA hardware at two levels:
-
- - An architectural overview of the VTA design and its ISA hardware-software
- interface.
- - A micro-architectural overview of the VTA hardware modules, and the
- micro-code specification for the compute core.
-
-VTA Overview
-------------
-
-VTA is a generic deep learning accelerator built for fast and efficient dense linear algebra.
-VTA incorporates a simple RISC-like processor that can perform dense linear algebra operations on rank 1 or 2 tensor registers.
-In addition the design adopts decoupled access-execute to hide memory access latency.
-
-
-To a broader extent, VTA can serve as a template deep learning accelerator design for full stack optimization, exposing a generic tensor computation interface to the compiler stack.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/blogpost/vta_overview.png
- :align: center
- :width: 80%
-
-The figure above gives a high-level overview of the VTA hardware organization.
-VTA is composed of four modules that communicate among each other via FIFO queues and local memory blocks (SRAM), to enable task-level pipeline parallelism:
-
-- The fetch module takes care of loading an instruction stream from DRAM. It also decodes those instructions to route them into one of three command queues.
-- The load module takes care of loading input and weight tensors from DRAM into data-specialized on-chip memories.
-- The compute module performs both dense linear algebra computation with its GEMM core, and general computation with its tensor ALU. It also takes care of loading data from DRAM into the register file, and loading micro-op kernels into the micro-op cache.
-- The store module stores results produced by the compute core back to DRAM.
-
-HLS Hardware Source Organization
---------------------------------
-
-The VTA design is currently specified in Vivado HLS C++, which is only supported
-by Xilinx toolchains.
-The VTA hardware sources are contained under ``3rdparty/vta-hw/hardware/xilinx/sources``:
-
- - ``vta.cc`` contains the definitions for each VTA module, as well as a top
- level behavioral model for the top-level VTA design.
- - ``vta.h`` contains type definitions using Xilinx ``ap_int`` types, and
- function prototypes declarations.
-
-In addition preprocessor macros are defined under ``3rdparty/vta-hw/include/vta/hw_spec.h``.
-Much of these macro definitions are derived from the parameters listed in the
-``3rdparty/vta-hw/config/vta_config.json`` file.
-The json file is processed by ``3rdparty/vta-hw/config/vta_config.py`` to produce a string of
-compile flags that define the preprocessor macros.
-That string is used by the makefile in order to set those high-level
-parameters in both the HLS hardware synthesis compiler, and the C++
-compiler that builds the VTA runtime.
-
-HLS Module Example
-~~~~~~~~~~~~~~~~~~
-
-We show a definition of one of the VTA modules defined in C++:
-
-.. code-block:: c
-
- void fetch(
- uint32_t insn_count,
- volatile insn_T *insns,
- hls::stream<insn_T> &load_queue,
- hls::stream<insn_T> &gemm_queue,
- hls::stream<insn_T> &store_queue) {
- #pragma HLS INTERFACE s_axilite port = insn_count bundle = CONTROL_BUS
- #pragma HLS INTERFACE m_axi port = insns offset = slave bundle = ins_port
- #pragma HLS INTERFACE axis port = load_queue
- #pragma HLS INTERFACE axis port = gemm_queue
- #pragma HLS INTERFACE axis port = store_queue
- #pragma HLS INTERFACE s_axilite port = return bundle = CONTROL_BUS
-
- INSN_DECODE: for (int pc = 0; pc < insn_count; pc++) {
- #pragma HLS PIPELINE II = 1
- // Read instruction fields
- insn_T insn = insns[pc];
- // Do some partial decoding
- opcode_T opcode = insn.range(VTA_INSN_MEM_0_1, VTA_INSN_MEM_0_0);
- memop_id_T memory_type = insn.range(VTA_INSN_MEM_5_1, VTA_INSN_MEM_5_0);
- // Push to appropriate instruction queue
- if (opcode == VTA_OPCODE_STORE) {
- store_queue.write(insn);
- } else if (opcode == VTA_OPCODE_LOAD &&
- (memory_type == VTA_MEM_ID_INP || memory_type == VTA_MEM_ID_WGT)) {
- load_queue.write(insn);
- } else {
- gemm_queue.write(insn);
- }
- }
- }
-
-A few observations on HLS coding:
- - *Parameters:* The parameter list of each function, combined with the
- interface pragmas define the hardware interface exposed by the
- generated hardware module.
-
- - Parameters passed by value indicate a read-only hardware memory-mapped
- register that the host can write to.
- This fetch function for instance has an ``insn_count`` parameter
- which will be synthesized as a memory mapped register for the host
- to write to, in order to set the length of a given VTA instruction
- sequence.
- - Pointer parameters can mean one of two things depending on the interface
- pragma being used.
-
- - When used with a ``m_axi`` interface pragma, an AXI requestor interface
- gets generated to provide DMA access to DRAM.
- - When used with a ``bram`` interface pragma, a BRAM interface gets
- generated to expose read and/or write ports to an FPGA block-RAM.
- - HLS streams being passed by reference combined with the ``axis`` interface
- pragma produce FIFO interfaces to the module. Hardware FIFOs provide a
- useful synchronization mechanism between modules.
- - *Pragmas*: Compiler pragmas are essential to define hardware implementation
- of each module. We list several pragmas used in the VTA design to communicate
- implementation requirements to the compiler.
-
- - ``HLS INTERFACE``: specifies the interface of the synthesized
- hardware module.
- - ``HLS PIPELINE``: defines hardware pipeline performance target by setting
- an initiation interval goal. When the ``II == 1`` target is set, it tells
- the compiler that the synthesized hardware pipeline should be able to
- execute one loop iteration per cycle.
- - ``HLS DEPENDENCE``: instructs the compiler to ignore certain types
- of dependence checks in a given loop. Consider a loop body that writes
- and reads to the same BRAM structure, and needs to achieve an II of 1.
- The HLS compiler has to assume worst-case scenario, whereby a read is
- issued to an address that a past write updated the cycle prior: this
- cannot be achieved given BRAM timing characteristics (it takes at least
- 2 cycles to see the updated value). Therefore in order to achieve an II of 1,
- the dependence checks have to be relaxed.
- Note that when turning this optimization on, it falls onto
- the software stack to prevent writes followed by reads to the same address.
-
- .. note::
- This `reference guide <https://www.xilinx.com/support/documentation/sw_manuals/xilinx2018_2/ug902-vivado-high-level-synthesis.pdf>`_
- provides a much more in-depth, and complete specification of HLS for the Xilinx 2018.2 toolchains.
-
-Architectural Overview
-----------------------
-
-Instruction Set Architecture
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-VTA's instruction set architecture (ISA) is composed of 4 CISC instructions that have a variable execution latency, two of which execute a micro-coded instruction sequence to perform computation.
-
-The VTA instructions are listed below:
-
-- ``LOAD`` instruction: loads a 2D tensor from DRAM into the input buffer, weight buffer, or register file. It can also load a micro-kernel into the micro-op cache. Supports dynamic padding when loading input and weight tiles.
-- ``GEMM`` instruction: performs a micro-op sequence of matrix-matrix multiplications over an input tensor and a weight tensors, and adds the result to a register-file tensor.
-- ``ALU`` instruction: performs a micro-op sequence of matrix-matrix ALU operations over register-file tensor data.
-- ``STORE`` instruction: stores a 2D tensor from the output buffer to DRAM.
-
-The ``LOAD`` instructions are executed by the load and compute modules depending on the store memory buffer location target.
-The ``GEMM`` and ``ALU`` instructions are executed by the compute module's GEMM core and tensor ALU.
-Finally, the ``STORE`` instructions are executed by the store module exclusively.
-The fields of each instruction is described in the figure below.
-The meaning of each field will be further explained in the :ref:`vta-uarch` section.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/developer/vta_instructions.png
- :align: center
- :width: 100%
-
-.. note::
- Note that the VTA ISA changes as VTA's architectural parameters are modified (i.e. GEMM core shape, data type, memory size etc.), and as a result the ISA does not guarantee compatibility across all variants of VTA.
- This is acceptable however, since the VTA runtime adapts to parameter changes, and produces binary code tailored for the version of the accelerator that gets generated.
- This exemplifies the co-design philosophy adopted by the VTA stack which embraces fluidity of the hardware-software interface.
-
-Dataflow Execution
-~~~~~~~~~~~~~~~~~~
-
-VTA relies on dependence FIFO queues between hardware modules to synchronize the execution of concurrent tasks.
-The figure below shows how a given hardware module can execute concurrently from its producer and consumer modules in a dataflow fashion through the use of dependence FIFO queues, and single-reader/single-writer SRAM buffers.
-Each module is connected to its consumer and producer via read-after-write (RAW) and write-after-read (WAR) dependence queues.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/developer/dataflow.png
- :align: center
- :width: 100%
-
-The pseudo-code above describes how a module executes a given instruction predicated on dependences with other instructions.
-First, the dependence flags within each instruction are decoded in hardware.
-If the instruction has an incoming RAW dependences, execution is predicated upon receiving a RAW dependence token from the producer module.
-Similarly, if the task has an incoming WAR dependence, execution is predicated upon receiving a WAR dependence token from the consumer module.
-Finally when the task is done, we check for outgoing RAW and WAR dependences, and notify the consumer and producer modules respectively.
-
-.. note::
- Note that the dependence tokens in this scenario are information-less.
- This is because the instructions executed by each module cannot be reordered by design, as they arrive in FIFO order.
-
-Pipeline Expandability
-~~~~~~~~~~~~~~~~~~~~~~
-
-The default VTA design is composed of four modules that describe a 3-stage ``load-compute-store`` task pipeline.
-Following the dataflow hardware organization principle, we can extend VTA the pipeline to include more stages.
-For example, we can envision separating the tensor ALU from the GEMM core in order to maximize the utilization of the GEMM core.
-This would result in a ``load-gemm-activate-store`` task pipeline which closely reflects the TPU design.
-Adding more stages has a cost however: it can add storage and extra logic overhead, which is why we opted for a default 3-stage pipeline.
-
-.. _vta-uarch:
-
-Microarchitectural Overview
----------------------------
-
-We describe the modules that compose the VTA design.
-The module definitions are contained in ``3rdparty/vta-hw/hardware/xilinx/sources/vta.cc``.
-
-Fetch Module
-~~~~~~~~~~~~
-
-VTA is programmed by a linear instruction stream.
-The fetch module is the entry point of VTA to the CPU and is programmed via three memory mapped registers:
-
-- The read-write ``control`` register starts the fetch module, and is read to check for its completion.
-- The write-only ``insn_count`` register sets the number of instructions to execute.
-- The write-only ``insns`` register sets the start address of the instruction stream in DRAM.
-
-The CPU prepares the instruction stream in DRAM in a physically-contiguous buffer prepared by the VTA runtime.
-When the instruction stream is ready, the CPU writes the start physical address into the ``insns`` register, the length of the instruction stream into the ``insn_count`` register, and asserts the start signal in the ``control`` register.
-This procedure starts VTA, which reads in the instruction stream from DRAM via DMA.
-
-Upon accessing the instruction stream, the fetch module partially decodes instructions, and pushes those instructions into command queues that feed into the load, compute, and store modules:
-
-- ``STORE`` instructions are pushed to the store command queue to be processed by the store module.
-- ``GEMM`` and ``ALU`` instructions are pushed to the compute command queue to be processed by the compute module.
-- ``LOAD`` instructions that describe a load operation of micro-op kernels or register file data are pushed to the compute command queue to be processed by the compute module.
-- ``LOAD`` instructions that describe a load operation of input or weight data are pushed to the load command queue to be processed by the load module.
-
-When one of the command queues becomes full, the fetch module stalls until the queue is not full.
-Consequently, the command queues are sized to be deep enough to allow for a wide execution window, and allow multiple tasks to be in flight concurrently across the ``load-compute-store`` pipeline.
-
-
-Compute Module
-~~~~~~~~~~~~~~
-
-VTA's compute module acts as a RISC processor that performs computation on tensor registers rather than scalar registers.
-Two functional units mutate the register file: the tensor ALU, and the GEMM core.
-
-The compute module executes RISC micro-ops from the micro-op cache.
-There are two types of compute micro-ops: ALU and GEMM operations.
-To minimize the footprint of micro-op kernels, while avoiding the need for control-flow instructions such as conditional jumps, the compute module executes micro-op sequences inside a two-level nested loop that computes the location of each tensor register location via an affine function.
-This compression approach helps reduce the micro-kernel instruction footprint, and applies to both matrix multiplication and 2D convolution, commonly found in neural network operators.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/developer/gemm_core.png
- :align: center
- :width: 100%
-
-The **GEMM core** evaluates GEMM instructions, by executing a micro-code sequence in a 2-level nested loop described in the Figure above.
-The GEMM core can perform one input-weight matrix multiplication per cycle.
-The dimensions of the single-cycle matrix multiplication defines a hardware *tensorization intrinsic* which the TVM compiler has to lower a computation schedule onto.
-This tensorization intrinsic is defined by the dimensions of the input, weight and accumulator tensors.
-Each data type can have a different integer precision: typically both weight and input types are low-precision (8-bits or less), while the accumulator tensor has a wider type to prevent overflows (32-bits).
-In order to keep the GEMM core busy, each of the input buffer, weight buffer, and register file have to expose sufficient read/write bandwidth.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/developer/alu_core.png
- :align: center
- :width: 100%
-
-The **Tensor ALU** supports a set of standard operations to implement common activation, normalization, and pooling operators.
-VTA being a modular design, the range of operators that the Tensor ALU supports can be extended for higher operator coverage, at the expense of higher resource utilization.
-The Tensor ALU can perform tensor-tensor operations, as well as tensor-scalar operations on an immediate value.
-The opcode of the tensor ALU, and the immediate value are specified by the high-level CISC instruction.
-The micro-code in the context of tensor ALU computation only takes care of specifying data access patterns.
-
-.. note::
- In terms of computational throughput, the Tensor ALU does not execute at a rate of one operation per cycle.
- The limitation comes from the lack of read-ports: since one register file tensor can be read per cycle, the tensor ALU has an initiation interval of at least 2 (i.e. performs at most 1 operation every 2 cycles).
- In addition, performing a single tensor-tensor operation at once can be expensive especially given that register file types are wide, typically 32-bit integers.
- As a result, in order to balance the resource utilization footprint of the Tensor ALU with the GEMM core, a tensor-tensor operation is by default performed via vector-vector operations over multiple cycles.
-
-
-Load and Store Modules
-~~~~~~~~~~~~~~~~~~~~~~
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/developer/2d_dma.png
- :align: center
- :width: 100%
-
-The load and store modules perform 2D DMA loads with a strided access pattern from DRAM to SRAM.
-In addition, the load module can insert 2D padding on the fly, which is useful when blocking 2D convolution.
-This means that VTA can tile 2D convolution inputs without paying the overhead of re-laying data out in DRAM to insert spatial padding around input and weight tiles.
diff --git a/docs/_sources/topic/vta/dev/index.rst.txt b/docs/_sources/topic/vta/dev/index.rst.txt
deleted file mode 100644
index 753af7a217..0000000000
--- a/docs/_sources/topic/vta/dev/index.rst.txt
+++ /dev/null
@@ -1,31 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-VTA Design and Developer Guide
-==============================
-
-This developer guide details the complete VTA-TVM hardware-software stack.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/blogpost/vta_stack.png
- :align: center
- :width: 60%
-
-.. toctree::
- :maxdepth: 2
-
- config
- hardware
diff --git a/docs/_sources/topic/vta/index.rst.txt b/docs/_sources/topic/vta/index.rst.txt
deleted file mode 100644
index d09b31bb36..0000000000
--- a/docs/_sources/topic/vta/index.rst.txt
+++ /dev/null
@@ -1,55 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-.. _vta-index:
-
-VTA: Versatile Tensor Accelerator
-=================================
-
-The Versatile Tensor Accelerator (VTA) is an open, generic, and customizable deep learning accelerator with a complete TVM-based compiler stack. We designed VTA to expose the most salient and common characteristics of mainstream deep learning accelerators. Together TVM and VTA form an end-to-end hardware-software deep learning system stack that includes hardware design, drivers, a JIT runtime, and an optimizing compiler stack based on TVM.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/blogpost/vta_overview.png
- :align: center
- :width: 60%
-
-VTA has the following key features:
-
-- Generic, modular, open-source hardware.
-- Streamlined workflow to deploy to FPGAs.
-- Simulator support to prototype compilation passes on regular workstations.
-- Pynq-based driver and JIT runtime for both simulated and FPGA hardware back-end.
-- End to end TVM stack integration.
-
-This page contains links to all the resources related to VTA:
-
-
-.. toctree::
- :maxdepth: 1
-
- install
- dev/index
- tutorials/index
-
-
-Literature
-----------
-
-- Read the VTA `release blog post`_.
-- Read the VTA tech report: `An Open Hardware Software Stack for Deep Learning`_.
-
-.. _release blog post: https://tvm.apache.org/2018/07/12/vta-release-announcement
-.. _An Open Hardware Software Stack for Deep Learning: https://arxiv.org/abs/1807.04188
diff --git a/docs/_sources/topic/vta/install.rst.txt b/docs/_sources/topic/vta/install.rst.txt
deleted file mode 100644
index ba76df410c..0000000000
--- a/docs/_sources/topic/vta/install.rst.txt
+++ /dev/null
@@ -1,488 +0,0 @@
-.. Licensed to the Apache Software Foundation (ASF) under one
- or more contributor license agreements. See the NOTICE file
- distributed with this work for additional information
- regarding copyright ownership. The ASF licenses this file
- to you under the Apache License, Version 2.0 (the
- "License"); you may not use this file except in compliance
- with the License. You may obtain a copy of the License at
-
-.. http://www.apache.org/licenses/LICENSE-2.0
-
-.. Unless required by applicable law or agreed to in writing,
- software distributed under the License is distributed on an
- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
- KIND, either express or implied. See the License for the
- specific language governing permissions and limitations
- under the License.
-
-VTA Installation Guide
-======================
-
-We present three installation guides, each extending on the previous one:
-
-1. `VTA Simulator Installation`_
-2. `Xilinx Pynq FPGA Setup`_
-3. `Intel DE10 FPGA Setup`_
-4. `Bitstream Generation with Xilinx Toolchains`_
-5. `Bitstream Generation with Intel Toolchains`_
-
-
-VTA Simulator Installation
---------------------------
-
-You need :ref:`TVM installed <installation>` on your machine. For a quick and
-easy start, checkout the :ref:`Docker Guide <docker-images>`.
-
-You'll need to set the following paths to use VTA:
-
-.. code:: bash
-
- export TVM_PATH=<path to TVM root>
- export VTA_HW_PATH=$TVM_PATH/3rdparty/vta-hw
-
-The VTA functional simulation library needs to be enabled when building TVM.
-
-.. code:: bash
-
- cd <tvm-root>
- mkdir build
- cp cmake/config.cmake build/.
- echo 'set(USE_VTA_FSIM ON)' >> build/config.cmake
- cd build && cmake .. && make -j4
-
-Add the VTA python library to your python path to run the VTA examples.
-
-.. code:: bash
-
- export PYTHONPATH=/path/to/vta/python:${PYTHONPATH}
-
-Testing your VTA Simulation Setup
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To ensure that you've properly installed the VTA python package, run the following 2D convolution testbench.
-
-.. code:: bash
-
- python <tvm root>/vta/tests/python/integration/test_benchmark_topi_conv2d.py
-
-You are invited to try out our :ref:`VTA programming tutorials <vta-tutorials>`.
-
- **Note**: You'll notice that for every convolution layer, the throughput gets reported in GOPS. These numbers are actually the computational throughput that the simulator achieves, by evaluating the convolutions in software.
-
-Advanced Configuration (optional)
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-VTA is a generic configurable deep learning accelerator.
-The configuration is specified by ``vta_config.json`` under ``3rdparty/vta-hw/config``.
-This file provides an architectural specification of the VTA accelerator to parameterize the TVM compiler stack and the VTA hardware stack.
-
-The VTA configuration file also specifies the TVM compiler target.
-When ``TARGET`` is set to ``sim``, all TVM workloads execute on the VTA simulator.
-You can modify the content of the configuration file to rebuild VTA to a different parameterization.
-To do so,
-
-.. code:: bash
-
- cd <tvm root>
- vim 3rdparty/vta-hw/config/vta_config.json
- # edit vta_config.json
- make
-
-
-
-Xilinx Pynq FPGA Setup
-----------------------
-
-This second guide extends the *VTA Simulator Installation* guide above to run FPGA hardware tests of the complete TVM and VTA software-hardware stack.
-In terms of hardware components you'll need:
-
-* The `Pynq <http://www.pynq.io/>`_ FPGA development board which can be acquired for $200, or $150 for academics from `Digilent <https://store.digilentinc.com/pynq-z1-python-productivity-for-zynq/>`_.
-* An Ethernet-to-USB adapter to connect the Pynq board to your development machine.
-* An 8+GB micro SD card.
-* An AC to DC 12V 3A power adapter.
-
-This guide covers the following themes:
-
-1. Pynq board setup instructions.
-2. Pynq-side RPC server build and deployment.
-3. Revisiting the test examples from the *VTA Simulator Installation* guide, this time executing on the Pynq board.
-
-Pynq Board Setup
-^^^^^^^^^^^^^^^^
-
-Setup your Pynq board based on the `Pynq board getting started tutorial <http://pynq.readthedocs.io/en/latest/getting_started.html>`_.
-
-You should follow the instructions up to and including the *Turning On the PYNQ-Z1* step (no need to pursue the tutorial beyond this point).
-
-* Make sure that you've downloaded the latest Pynq image, `PYNQ-Z1 v2.5 <http://www.pynq.io/board.html>`_, and have imaged your SD card with it (we recommend the free `Etcher <https://etcher.io/>`_ program).
-* For this test setup, follow the `"Connect to a Computer" <https://pynq.readthedocs.io/en/latest/getting_started/pynq_z1_setup.html>`_ Ethernet setup instructions. To be able to talk to the board, make sure to `assign your computer a static IP address <https://pynq.readthedocs.io/en/latest/appendix.html#assign-your-computer-a-static-ip>`_
-
-Once the board is powered on and connected to your development machine, try connecting to it to make sure you've properly set up your Pynq board:
-
-.. code:: bash
-
- # To connect to the Pynq board use the <username, password> combo: <xilinx, xilinx>
- ssh xilinx@192.168.2.99
-
-Pynq-Side RPC Server Build & Deployment
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Because the direct board-to-computer connection prevents the board from directly accessing the internet, we'll need to mount the Pynq's file system to your development machine's file system with `sshfs <https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh>`_. Next we directly clone the TVM repository into the sshfs mountpoint on your development machine.
-
-.. code:: bash
-
- # On the Host-side
- mkdir <mountpoint>
- sshfs xilinx@192.168.2.99:/home/xilinx <mountpoint>
- cd <mountpoint>
- git clone --recursive https://github.com/apache/tvm tvm
- # When finished, you can leave the moutpoint and unmount the directory
- cd ~
- sudo umount <mountpoint>
-
-Now that we've cloned the VTA repository in the Pynq's file system, we can ssh into it and launch the build of the TVM-based RPC server.
-The build process should take roughly 5 minutes.
-
-.. code:: bash
-
- ssh xilinx@192.168.2.99
- # Build TVM runtime library (takes 5 mins)
- cd /home/xilinx/tvm
- mkdir build
- cp cmake/config.cmake build/.
- echo 'set(USE_VTA_FPGA ON)' >> build/config.cmake
- # Copy pynq specific configuration
- cp 3rdparty/vta-hw/config/pynq_sample.json 3rdparty/vta-hw/config/vta_config.json
- cd build
- cmake ..
- make runtime vta -j2
- # FIXME (tmoreau89): remove this step by fixing the cmake build
- make clean; make runtime vta -j2
- # Build VTA RPC server (takes 1 min)
- cd ..
- sudo ./apps/vta_rpc/start_rpc_server.sh # pw is 'xilinx'
-
-
-You should see the following being displayed when starting the RPC server. In order to run the next examples, you'll need to leave the RPC server running in an ``ssh`` session.
-
-.. code:: bash
-
- INFO:root:RPCServer: bind to 0.0.0.0:9091
-
-
-Tips regarding the Pynq RPC Server:
-
-* The RPC server should be listening on port ``9091``. If not, an earlier process might have terminated unexpectedly and it's recommended in this case to just reboot the Pynq, and re-run the RPC server.
-* To kill the RPC server, just send the ``Ctrl + c`` command. You can re-run it with ``sudo ./apps/pynq_rpc/start_rpc_server.sh``.
-* If unresponsive, the board can be rebooted by power-cycling it with the physical power switch.
-
-Testing your Pynq-based Hardware Setup
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Before running the examples on your development machine, you'll need to configure your host environment as follows:
-
-.. code:: bash
-
- # On the Host-side
- export VTA_RPC_HOST=192.168.2.99
- export VTA_RPC_PORT=9091
-
-
-In addition, you'll need to edit the ``vta_config.json`` file on the host to indicate that we are targeting the Pynq platform, by setting the ``TARGET`` field to ``"pynq"``.
-> Note: in contrast to our simulation setup, there are no libraries to compile on the host side since the host offloads all of the computation to the Pynq board.
-
-.. code:: bash
-
- # On the Host-side
- cd <tvm root>
- cp 3rdparty/vta-hw/config/pynq_sample.json 3rdparty/vta-hw/config/vta_config.json
-
-
-This time again, we will run the 2D convolution testbench.
-Beforehand, we need to program the Pynq board FPGA with a VTA bitstream, and build the VTA runtime via RPC.
-The following ``test_program_rpc.py`` script will perform two operations:
-
-* FPGA programming, by downloading a pre-compiled bitstream from a `VTA bitstream repository <https://github.com/uwsampl/vta-distro>`_ that matches the default ``vta_config.json`` configuration set by the host, and sending it over to the Pynq via RPC to program the Pynq's FPGA.
-* Runtime building on the Pynq, which needs to be run every time the ``vta_config.json`` configuration is modified. This ensures that the VTA software runtime that generates the accelerator's executable via just-in-time (JIT) compilation matches the specifications of the VTA design that is programmed on the FPGA. The build process takes about 30 seconds to complete so be patient!
-
-.. code:: bash
-
- # On the Host-side
- python <tvm root>/vta/tests/python/pynq/test_program_rpc.py
-
-
-We are now ready to run the 2D convolution testbench in hardware.
-
-.. code:: bash
-
- # On the Host-side
- python <tvm root>/vta/tests/python/integration/test_benchmark_topi_conv2d.py
-
-The performance metrics measured on the Pynq board will be reported for each convolutional layer.
-
-**Tip**: You can track progress of the FPGA programming and the runtime rebuilding steps by looking at the RPC server's logging messages in your Pynq ``ssh`` session.
-
-You can also try out our :ref:`VTA programming tutorials <vta-tutorials>`.
-
-Intel DE10 FPGA Setup
----------------------
-
-Similar to the Pynq-side setup steps, this third guide bring us the details on how can we setup up the Linux environment for Intel FPGA boards like DE10-Nano.
-
-In terms of hardware components, you would need the `DE10-Nano Development Kit <https://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&No=1046>`_, which can be acquired for $130, or $100 for academics from `Terasic <https://www.terasic.com.tw/>`_. A microSD card would be delivered the kit. Power cables and USB cables would be included as well. However, an additional Ethernet cable would be needed to connect the board to LAN.
-
-The rest part of this guide would provide the steps to
-
-* Flash the microSD card with latest Angstrom Linux image
-* Cross-compilation setup
-* Device-side RPC server setup and deployment
-
-DE10-Nano Board Setup
-^^^^^^^^^^^^^^^^^^^^^
-
-Before powering up the device, we need to flash the microSD card image with latest Angstrom Linux image.
-
-Flash SD Card and Boot Angstrom Linux
-"""""""""""""""""""""""""""""""""""""
-
-To flash SD card and boot Linux on DE10-Nano, it is recommended to navigate to the `Resource <https://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=167&No=1046&PartNo=4>`_ tab of the DE10-Nano product page from Terasic Inc.
-After registration and login on the webpage, the prebuilt Angstrom Linux image would be available for downloading and flashing.
-Specifically, to flash the downloaded Linux SD card image into your physical SD card:
-
-First, extract the gzipped archive file.
-
-.. code:: bash
-
- tar xf de10-nano-image-Angstrom-v2016.12.socfpga-sdimg.2017.03.31.tgz
-
-This would produce a single SD card image named ``de10-nano-image-Angstrom-v2016.12.socfpga-sdimg`` (approx. 2.4 GB), it contains all the file systems to boot Angstrom Linux.
-
-Second, plugin a SD card that is ready to flash in your PC, and identify the device id for the disk with ``fdisk -l``, or ``gparted`` if you feel better to use GUI. The typical device id for your disk would likely to be ``/dev/sdb``.
-
-Then, flash the disk image into your physical SD card with the following command:
-
-.. code:: bash
-
- # NOTE: root privilege is typically required to run the following command.
- dd if=de10-nano-image-Angstrom-v2016.12.socfpga-sdimg of=/dev/sdb status=progress
-
-This would take a few minutes for your PC to write the whole file systems into the SD card.
-After this process completes, you are ready to unmount the SD card and insert it into your DE10-Nano board.
-Now you can connect the power cable and serial port to boot the Angstrom Linux.
-
- **Note**: When boot up from the microSD card, you might notice the incompatibility of the Linux kernel ``zImage`` in the microSD card.
- In this case, you might need to build the ``zImage`` file of your own from `socfpga-4.9.78-ltsi <https://github.com/altera-opensource/linux-socfpga/tree/socfpga-4.9.78-ltsi>`_ branch of the `linux-socfpga <https://github.com/altera-opensource/linux-socfpga>`_ repository.
- For a quick fix, you can also download a prebuilt version of the ``zImage`` file `from this link <https://raw.githubusercontent.com/liangfu/de10-nano-supplement/master/zImage>`_.
-
-After connecting the usb cables to the DE10-Nano board, power on the board by connecting the power cable. You may then connect to the serial port of the device by using ``minicom`` on your host PC:
-
-.. code:: bash
-
- # NOTE: root privilege is typically required to run the following command.
- minicom -D /dev/ttyUSB0
-
-The default user name for the device would be ``root``, and the password is empty for the default user.
-
-You may now start to install supporting Python3 packages (TVM has dropped the support for Python2), specifically, they are ``numpy``, ``attrs`` and ``decorator``.
-
- **Note**: You might fail to install ``numpy`` by using ``pip3`` on the DE10-Nano device.
- In that case, you have the option to either build your own filesystem image for the board from `meta-de10-nano <https://github.com/intel/meta-de10-nano>`_ repository;
- an alternative option is to download prebuilt packages from existing Linux distributions, e.g. Debian.
- For a quick fix, we have concatenated the supplementary binary files `here <https://raw.githubusercontent.com/liangfu/de10-nano-supplement/master/rootfs_supplement.tgz>`_, and you can extract the files into the root filesystem.
-
-Install Required Python Packages
-""""""""""""""""""""""""""""""""
-
-After accessing bash terminal from the serial port, we need to install required Python packages before building and installing TVM and VTA programs.
-
-Build Additional Components to Use VTA Bitstream
-""""""""""""""""""""""""""""""""""""""""""""""""
-
-To use the above built bitstream on DE10-Nano hardware, several additional components need to be compiled for the system.
-Specifically, to compile application executables for the system, you need to download and install `SoCEDS <http://fpgasoftware.intel.com/soceds/18.1/?edition=standard&download_manager=dlm3&platform=linux>`_ (recommended), or alternatively install the ``g++-arm-linux-gnueabihf`` package on your host machine. You would also need a ``cma`` kernel module to allocate contigous memory, and a driver for communicating with the VTA subsystem.
-
-
-Bitstream Generation with Xilinx Toolchains
--------------------------------------------
-
-If you're interested in generating the Xilinx FPGA bitstream on your own instead of using the pre-built VTA bitstreams, follow the instructions below.
-
-Xilinx Toolchain Installation
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-We recommend using Vivado 2020.1 since our scripts have been tested to work on this version of the Xilinx toolchains.
-Our guide is written for Linux (Ubuntu) installation.
-
-You’ll need to install Xilinx’ FPGA compilation toolchain, `Vivado HL WebPACK 2020.1 <https://www.xilinx.com/products/design-tools/vivado.html>`_, which a license-free version of the Vivado HLx toolchain.
-
-Obtaining and Launching the Vivado GUI Installer
-""""""""""""""""""""""""""""""""""""""""""""""""
-
-1. Go to the `download webpage <https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vivado-design-tools/2020-1.html>`_, and download the Linux Self Extracting Web Installer for Vivado HLx 2020.1: WebPACK and Editions.
-2. You’ll have to sign in with a Xilinx account. This requires a Xilinx account creation that will take 2 minutes.
-3. Complete the Name and Address Verification by clicking “Next”, and you will get the opportunity to download a binary file, called ``Xilinx_Unified_2020.1_0602_1208_Lin64.bin``.
-4. Now that the file is downloaded, go to your ``Downloads`` directory, and change the file permissions so it can be executed:
-
-.. code:: bash
-
- chmod u+x Xilinx_Unified_2020.1_0602_1208_Lin64.bin
-
-5. Now you can execute the binary:
-
-.. code:: bash
-
- ./Xilinx_Unified_2020.1_0602_1208_Lin64.bin
-
-Xilinx Vivado GUI Installer Steps
-"""""""""""""""""""""""""""""""""
-
-At this point you've launched the Vivado 2020.1 Installer GUI program.
-
-1. Click “Next” on the "Welcome" screen.
-2. On the "Select Install Type" screen, enter your Xilinx user credentials under the “User Authentication” box and select the “Download and Install Now” option before clicking “Next”.
-3. On the "Accept License Agreements" screen, accept all terms before clicking “Next”.
-4. On the "Select Edition to Install" screen, select the “Vivado HL WebPACK” before clicking “Next”.
-5. Under the "Vivado HL WebPACK" screen, before hitting “Next", check the following options (the rest should be unchecked):
- * Design Tools -> Vivado Design Suite -> Vivado
- * Devices -> Production Devices -> SoCs -> Zynq-7000 (if you are targeting the Pynq board)
- * Devices -> Production Devices -> SoCs -> UltraScale+ MPSoC (if you are targeting the Ultra-96 board)
-6. Your total download size should be about 5GB and the amount of Disk Space Required 23GB.
-7. On the "Select Destination Directory" screen, set the installation directory before clicking “Next”. It might highlight some paths as red - that’s because the installer doesn’t have the permission to write to the directory. In that case select a path that doesn’t require special write permissions (e.g. your home directory).
-8. On the "Installation Summary" screen, hit “Install”.
-9. An "Installation Progress" window will pop-up to track progress of the download and the installation.
-10. This process will take about 20-30 minutes depending on your connection speed.
-11. A pop-up window will inform you that the installation completed successfully. Click "OK".
-12. Finally the "Vivado License Manager" will launch. Select "Get Free ISE WebPACK, ISE/Vivado IP or PetaLinux License" and click "Connect Now" to complete the license registration process.
-
-Environment Setup
-"""""""""""""""""
-
-The last step is to update your ``~/.bashrc`` with the following lines. This will include all of the Xilinx binary paths so you can launch compilation scripts from the command line.
-
-.. code:: bash
-
- # Xilinx Vivado 2020.1 environment
- export XILINX_VIVADO=${XILINX_PATH}/Vivado/2020.1
- export PATH=${XILINX_VIVADO}/bin:${PATH}
-
-HLS-based Custom VTA Bitstream Compilation for Pynq
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-High-level hardware parameters are listed in the VTA configuration file and can be customized by the user.
-For this custom VTA bitstream compilation exercise, we'll change the frequency of our design, so it can be clocked a little faster.
-
-* Set the ``HW_FREQ`` field to ``142``. The Pynq board supports 100, 142, 167 and 200MHz clocks. Note that the higher the frequency, the harder it will be to close timing. Increasing the frequency can lead to timing violation and thus faulty hardware execution.
-* Set the ``HW_CLK_TARGET`` to ``6``. This parameters refers to the target clock period in nano seconds for HLS - a lower clock period leads to more aggressive pipelining to achieve timing closure at higher frequencies. Technically a 142MHz clock would require a 7ns target, but we intentionally lower the clock target to 6ns to more aggressively pipeline our design.
-
-Bitstream generation is driven by a top-level ``Makefile`` under ``<tvm root>/3rdparty/vta-hw/hardware/xilinx/``.
-
-If you just want to simulate the VTA design in software emulation to make sure that it is functional, enter:
-
-.. code:: bash
-
- cd <tvm root>/3rdparty/vta-hw/hardware/xilinx
- make ip MODE=sim
-
-
-If you just want to generate the HLS-based VTA IP cores without launching the entire design place and route, enter:
-
-.. code:: bash
-
- make ip
-
-You'll be able to view the HLS synthesis reports under ``<tvm root>/3rdparty/vta-hw/build/hardware/xilinx/hls/<configuration>/<block>/solution0/syn/report/<block>_csynth.rpt``
-
- **Note**: The ``<configuration>`` name is a string that summarizes the VTA configuration parameters listed in the ``vta_config.json``. The ``<block>`` name refers to the specific module (or HLS function) that compose the high-level VTA pipeline.
-
-Finally to run the full hardware compilation and generate the VTA bitstream, run ``make``.
-
-This process is lengthy, and can take around up to an hour to complete depending on your machine's specs.
-We recommend setting the ``VTA_HW_COMP_THREADS`` variable in the Makefile to take full advantage of all the cores on your development machine.
-
-Once the compilation completes, the generated bitstream can be found under ``<tvm root>/3rdparty/vta-hw/build/hardware/xilinx/vivado/<configuration>/export/vta.bit``.
-
-Using A Custom Bitstream
-^^^^^^^^^^^^^^^^^^^^^^^^
-
-We can program the new VTA FPGA bitstream by setting the bitstream path of the ``vta.program_fpga()`` function in the tutorial examples, or in the ``test_program_rpc.py`` script.
-
-.. code:: python
-
- vta.program_fpga(remote, bitstream="<tvm root>/3rdparty/vta-hw/build/hardware/xilinx/vivado/<configuration>/export/vta.bit")
-
-Instead of downloading a pre-built bitstream from the VTA bitstream repository, TVM will instead use the new bitstream you just generated, which is a VTA design clocked at a higher frequency.
-Do you observe a noticeable performance increase on the ImageNet classification example?
-
-
-
-Bitstream Generation with Intel Toolchains
--------------------------------------------
-
-If you're interested in generating the Xilinx FPGA bitstream on your own instead of using the pre-built VTA bistreams, follow the instructions below.
-
-Intel Toolchain Installation
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-It is recommended to use ``Intel Quartus Prime 18.1``, since the test scripts contained in this document have been tested on this version.
-
-You would need to install Intel's FPGA compilation toolchain, `Quartus Prime Lite <http://fpgasoftware.intel.com/?edition=lite>`_, which is a license-free version of the Intel Quartus Prime software.
-
-Obtaining and Launching the Quartus GUI Installer
-"""""""""""""""""""""""""""""""""""""""""""""""""
-
-1. Go to the `download center <http://fpgasoftware.intel.com/?edition=lite>`_, and download the linux version of "Quartus Prime (include Nios II EDS)" and "Cyclone V device support" files in the "Separate file" tab. This avoid downloading unused device support files.
-2. Sign in the form if you have an account, or register on the right side of the web page to create an account.
-3. After signed in, you are able to download the installer and the device support files.
-4. Now that the files are downloaded, go to your ``Downloads`` directory, and change the file permissions:
-
-.. code:: bash
-
- chmod u+x QuartusLiteSetup-18.1.0.625-linux.run
-
-5. Now ensure both the installer and device support files are in the same directory, and you can run the install with:
-
-.. code:: bash
-
- ./QuartusLiteSetup-18.1.0.625-linux.run
-
-6. Follow the instructions on the pop-up GUI form, and install all the content in the ``/usr/local`` directory. After installation, ``/usr/local/intelFPGA_lite/18.1`` would be created and the Quartus program along with other programs would be available in the folder.
-
-Environment Setup
-"""""""""""""""""
-
-Similar to what should be done for Xilinx toolchain, the following line should be added to your ``~/.bashrc``.
-
-.. code:: bash
-
- # Intel Quartus 18.1 environment
- export QUARTUS_ROOTDIR="/usr/local/intelFPGA_lite/18.1/quartus"
- export PATH=${QUARTUS_ROOTDIR}/bin:${PATH}
- export PATH=${QUARTUS_ROOTDIR}/sopc_builder/bin:${PATH}
-
-This would add quartus binary path into your ``PATH`` environment variable, so you can launch compilation scripts from the command line.
-
-Chisel-based Custom VTA Bitstream Compilation for DE10-Nano
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-Similar to the HLS-based design, high-level hardware parameters in Chisel-based design are listed in the VTA configuration file `Configs.scala <https://github.com/apache/tvm/blob/main/3rdparty/vta-hw/hardware/chisel/src/main/scala/core/Configs.scala>`_, and they can be customized by the user.
-
-For Intel FPGA, bitstream generation is driven by a top-level ``Makefile`` under ``<tvm root>/3rdparty/vta-hw/hardware/intel``.
-
-If you just want to generate the Chisel-based VTA IP core for the DE10-Nano board without compiling the design for the FPGA hardware, enter:
-
-.. code:: bash
-
- cd <tvm root>/3rdparty/vta-hw/hardware/intel
- make ip
-
-Then you'll be able to locate the generated verilog file at ``<tvm root>/3rdparty/vta-hw/build/hardware/intel/chisel/<configuration>/VTA.DefaultDe10Config.v``.
-
-If you would like to run the full hardware compilation for the ``de10nano`` board:
-
-.. code:: bash
-
- make
-
-This process might be a bit lengthy, and might take up to half an hour to complete depending on the performance of your PC. The Quartus Prime software would automatically detect the number of cores available on your PC and try to utilize all of them to perform such process.
-
-Once the compilation completes, the generated bistream can be found under ``<tvm root>/3rdparty/vta-hw/build/hardware/intel/quartus/<configuration>/export/vta.rbf``. You can also open the Quartus project file (.qpf) available at ``<tvm root>/3rdparty/vta-hw/build/hardware/intel/quartus/<configuration>/de10_nano_top.qpf`` to look around the generated reports.
diff --git a/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
deleted file mode 100644
index 2f87dba7a0..0000000000
--- a/docs/_sources/topic/vta/tutorials/frontend/deploy_detection.rst.txt
+++ /dev/null
@@ -1,473 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/frontend/deploy_detection.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_frontend_deploy_detection.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/66e1a42229aae7ed49ac268f520e6727/deploy_detection.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py:
-
-
-Deploy Pretrained Vision Detection Model from Darknet on VTA
-============================================================
-**Author**: `Hua Jiang <https://github.com/huajsj>`_
-
-This tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny
-inference onto the VTA accelerator design to perform Image detection tasks.
-It showcases Relay as a front end compiler that can perform quantization (VTA
-only supports int8/32 inference) as well as graph packing (in order to enable
-tensorization in the core) to massage the compute graph for the hardware target.
-
-.. GENERATED FROM PYTHON SOURCE LINES 30-48
-
-Install dependencies
---------------------
-To use the autotvm package in tvm, we need to install some extra dependencies.
-(change "3" to "2" if you use python2):
-
-.. code-block:: bash
-
- pip3 install "Pillow<7"
-
-YOLO-V3-tiny Model with Darknet parsing have dependancy with CFFI and CV2 library,
-we need to install CFFI and CV2 before executing this script.
-
-.. code-block:: bash
-
- pip3 install cffi
- pip3 install opencv-python
-
-Now return to the python code. Import packages.
-
-.. GENERATED FROM PYTHON SOURCE LINES 48-69
-
-.. code-block:: default
-
-
- from __future__ import absolute_import, print_function
-
- import sys
- import os
- import time
- import matplotlib.pyplot as plt
- import numpy as np
- import tvm
- import vta
- from tvm import rpc, autotvm, relay
- from tvm.relay.testing import yolo_detection, darknet
- from tvm.relay.testing.darknet import __darknetffi__
- from tvm.contrib import graph_executor, utils
- from tvm.contrib.download import download_testdata
- from vta.testing import simulator
- from vta.top import graph_pack
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 70-73
-
-Download yolo net configure file, weight file, darknet library file based on
-Model Name
-----------------------------------------------------------------------------
-
-.. GENERATED FROM PYTHON SOURCE LINES 73-100
-
-.. code-block:: default
-
- MODEL_NAME = "yolov3-tiny"
- REPO_URL = "https://github.com/dmlc/web-data/blob/main/darknet/"
-
- cfg_path = download_testdata(
- "https://github.com/pjreddie/darknet/blob/master/cfg/" + MODEL_NAME + ".cfg" + "?raw=true",
- MODEL_NAME + ".cfg",
- module="darknet",
- )
- weights_path = download_testdata(
- "https://pjreddie.com/media/files/" + MODEL_NAME + ".weights" + "?raw=true",
- MODEL_NAME + ".weights",
- module="darknet",
- )
-
- if sys.platform in ["linux", "linux2"]:
- darknet_lib_path = download_testdata(
- REPO_URL + "lib/" + "libdarknet2.0.so" + "?raw=true", "libdarknet2.0.so", module="darknet"
- )
- elif sys.platform == "darwin":
- darknet_lib_path = download_testdata(
- REPO_URL + "lib_osx/" + "libdarknet_mac2.0.so" + "?raw=true",
- "libdarknet_mac2.0.so",
- module="darknet",
- )
- else:
- raise NotImplementedError("Darknet lib is not supported on {} platform".format(sys.platform))
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 101-103
-
-Download yolo categories and illustration front.
-------------------------------------------------
-
-.. GENERATED FROM PYTHON SOURCE LINES 103-113
-
-.. code-block:: default
-
- coco_path = download_testdata(
- REPO_URL + "data/" + "coco.names" + "?raw=true", "coco.names", module="data"
- )
- font_path = download_testdata(
- REPO_URL + "data/" + "arial.ttf" + "?raw=true", "arial.ttf", module="data"
- )
- with open(coco_path) as f:
- content = f.readlines()
- names = [x.strip() for x in content]
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 114-117
-
-Define the platform and model targets.
---------------------------------------
-Execute on CPU vs. VTA, and define the model.
-
-.. GENERATED FROM PYTHON SOURCE LINES 117-141
-
-.. code-block:: default
-
-
- # Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
- env = vta.get_env()
- # Set ``device=arm_cpu`` to run inference on the CPU
- # or ``device=vta`` to run inference on the FPGA.
- device = "vta"
- target = env.target if device == "vta" else env.target_vta_cpu
-
- pack_dict = {
- "yolov3-tiny": ["nn.max_pool2d", "cast", 4, 186],
- }
-
- # Name of Darknet model to compile
- # The ``start_pack`` and ``stop_pack`` labels indicate where
- # to start and end the graph packing relay pass: in other words
- # where to start and finish offloading to VTA.
- # the number 4 indicate the ``start_pack`` index is 4, the
- # number 186 indicate the ``stop_pack index`` is 186, by using
- # name and index number, here we can located to correct place
- # where to start/end when there are multiple ``nn.max_pool2d``
- # or ``cast``, print(mod.astext(show_meta_data=False)) can help
- # to find operator name and index information.
- assert MODEL_NAME in pack_dict
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 142-146
-
-Obtain an execution remote.
----------------------------
-When target is 'pynq' or other FPGA backend, reconfigure FPGA and runtime.
-Otherwise, if target is 'sim', execute locally.
-
-.. GENERATED FROM PYTHON SOURCE LINES 146-180
-
-.. code-block:: default
-
-
- if env.TARGET not in ["sim", "tsim"]:
- # Get remote from tracker node if environment variable is set.
- # To set up the tracker, you'll need to follow the "Auto-tuning
- # a convolutional network for VTA" tutorial.
- tracker_host = os.environ.get("TVM_TRACKER_HOST", None)
- tracker_port = os.environ.get("TVM_TRACKER_PORT", None)
- # Otherwise if you have a device you want to program directly from
- # the host, make sure you've set the variables below to the IP of
- # your board.
- device_host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- device_port = os.environ.get("VTA_RPC_PORT", "9091")
- if not tracker_host or not tracker_port:
- remote = rpc.connect(device_host, int(device_port))
- else:
- remote = autotvm.measure.request_remote(
- env.TARGET, tracker_host, int(tracker_port), timeout=10000
- )
- # Reconfigure the JIT runtime and FPGA.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- reconfig_start = time.time()
- vta.reconfig_runtime(remote)
- vta.program_fpga(remote, bitstream=None)
- reconfig_time = time.time() - reconfig_start
- print("Reconfigured FPGA and RPC runtime in {0:.2f}s!".format(reconfig_time))
-
- # In simulation mode, host the RPC server locally.
- else:
- remote = rpc.LocalSession()
-
- # Get execution context from remote
- ctx = remote.ext_dev(0) if device == "vta" else remote.cpu(0)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 181-194
-
-Build the inference graph executor.
------------------------------------
-Using Darknet library load downloaded vision model and compile with Relay.
-The compilation steps are:
-
-1. Front end translation from Darknet into Relay module.
-2. Apply 8-bit quantization: here we skip the first conv layer,
- and dense layer which will both be executed in fp32 on the CPU.
-3. Perform graph packing to alter the data layout for tensorization.
-4. Perform constant folding to reduce number of operators (e.g. eliminate batch norm multiply).
-5. Perform relay build to object file.
-6. Load the object file onto remote (FPGA device).
-7. Generate graph executor, `m`.
-
-.. GENERATED FROM PYTHON SOURCE LINES 194-253
-
-.. code-block:: default
-
-
- # Load pre-configured AutoTVM schedules
- with autotvm.tophub.context(target):
- net = __darknetffi__.dlopen(darknet_lib_path).load_network(
- cfg_path.encode("utf-8"), weights_path.encode("utf-8"), 0
- )
- dshape = (env.BATCH, net.c, net.h, net.w)
- dtype = "float32"
-
- # Measure build start time
- build_start = time.time()
-
- # Start front end compilation
- mod, params = relay.frontend.from_darknet(net, dtype=dtype, shape=dshape)
-
- if target.device_name == "vta":
- # Perform quantization in Relay
- # Note: We set opt_level to 3 in order to fold batch norm
- with tvm.transform.PassContext(opt_level=3):
- with relay.quantize.qconfig(
- global_scale=23.0,
- skip_conv_layers=[0],
- store_lowbit_output=True,
- round_for_shift=True,
- ):
- mod = relay.quantize.quantize(mod, params=params)
- # Perform graph packing and constant folding for VTA target
- mod = graph_pack(
- mod["main"],
- env.BATCH,
- env.BLOCK_OUT,
- env.WGT_WIDTH,
- start_name=pack_dict[MODEL_NAME][0],
- stop_name=pack_dict[MODEL_NAME][1],
- start_name_idx=pack_dict[MODEL_NAME][2],
- stop_name_idx=pack_dict[MODEL_NAME][3],
- )
- else:
- mod = mod["main"]
-
- # Compile Relay program with AlterOpLayout disabled
- with vta.build_config(disabled_pass={"AlterOpLayout", "tir.CommonSubexprElimTIR"}):
- lib = relay.build(
- mod, target=tvm.target.Target(target, host=env.target_host), params=params
- )
-
- # Measure Relay build time
- build_time = time.time() - build_start
- print(MODEL_NAME + " inference graph built in {0:.2f}s!".format(build_time))
-
- # Send the inference library over to the remote RPC server
- temp = utils.tempdir()
- lib.export_library(temp.relpath("graphlib.tar"))
- remote.upload(temp.relpath("graphlib.tar"))
- lib = remote.load_module("graphlib.tar")
-
- # Graph executor
- m = graph_executor.GraphModule(lib["default"](ctx))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- /workspace/python/tvm/relay/build_module.py:345: DeprecationWarning: Please use input parameter mod (tvm.IRModule) instead of deprecated parameter mod (tvm.relay.function.Function)
- warnings.warn(
- yolov3-tiny inference graph built in 29.86s!
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 254-258
-
-Perform image detection inference.
-----------------------------------
-We run detect on an downloaded image
-Download test image
-
-.. GENERATED FROM PYTHON SOURCE LINES 258-323
-
-.. code-block:: default
-
- [neth, netw] = dshape[2:]
- test_image = "person.jpg"
- img_url = REPO_URL + "data/" + test_image + "?raw=true"
- img_path = download_testdata(img_url, test_image, "data")
- data = darknet.load_image(img_path, neth, netw).transpose(1, 2, 0)
-
- # Prepare test image for inference
- plt.imshow(data)
- plt.show()
- data = data.transpose((2, 0, 1))
- data = data[np.newaxis, :]
- data = np.repeat(data, env.BATCH, axis=0)
-
- # Set the network parameters and inputs
- m.set_input("data", data)
-
- # Perform inference and gather execution statistics
- # More on: :py:method:`tvm.runtime.Module.time_evaluator`
- num = 4 # number of times we run module for a single measurement
- rep = 3 # number of measurements (we derive std dev from this)
- timer = m.module.time_evaluator("run", ctx, number=num, repeat=rep)
-
- if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
- timer()
- sim_stats = simulator.stats()
- print("\nExecution statistics:")
- for k, v in sim_stats.items():
- # Since we execute the workload many times, we need to normalize stats
- # Note that there is always one warm up run
- # Therefore we divide the overall stats by (num * rep + 1)
- print("\t{:<16}: {:>16}".format(k, v // (num * rep + 1)))
- else:
- tcost = timer()
- std = np.std(tcost.results) * 1000
- mean = tcost.mean * 1000
- print("\nPerformed inference in %.2fms (std = %.2f) for %d samples" % (mean, std, env.BATCH))
- print("Average per sample inference time: %.2fms" % (mean / env.BATCH))
-
- # Get detection results from out
- thresh = 0.5
- nms_thresh = 0.45
- tvm_out = []
- for i in range(2):
- layer_out = {}
- layer_out["type"] = "Yolo"
- # Get the yolo layer attributes (n, out_c, out_h, out_w, classes, total)
- layer_attr = m.get_output(i * 4 + 3).numpy()
- layer_out["biases"] = m.get_output(i * 4 + 2).numpy()
- layer_out["mask"] = m.get_output(i * 4 + 1).numpy()
- out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
- layer_out["output"] = m.get_output(i * 4).numpy().reshape(out_shape)
- layer_out["classes"] = layer_attr[4]
- tvm_out.append(layer_out)
- thresh = 0.560
-
- # Show detection results
- img = darknet.load_image_color(img_path)
- _, im_h, im_w = img.shape
- dets = yolo_detection.fill_network_boxes((netw, neth), (im_w, im_h), thresh, 1, tvm_out)
- last_layer = net.layers[net.n - 1]
- yolo_detection.do_nms_sort(dets, last_layer.classes, nms_thresh)
- yolo_detection.draw_detections(font_path, img, dets, thresh, names, last_layer.classes)
- plt.imshow(img.transpose(1, 2, 0))
- plt.show()
-
-
-
-.. image-sg:: /topic/vta/tutorials/frontend/images/sphx_glr_deploy_detection_001.png
- :alt: deploy detection
- :srcset: /topic/vta/tutorials/frontend/images/sphx_glr_deploy_detection_001.png
- :class: sphx-glr-single-img
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
-
- Execution statistics:
- inp_load_nbytes : 25462784
- wgt_load_nbytes : 17558016
- acc_load_nbytes : 96128
- uop_load_nbytes : 5024
- out_store_nbytes: 3396224
- gemm_counter : 10578048
- alu_counter : 849056
-
-
-
-
-
-.. rst-class:: sphx-glr-timing
-
- **Total running time of the script:** ( 1 minutes 6.989 seconds)
-
-
-.. _sphx_glr_download_topic_vta_tutorials_frontend_deploy_detection.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: deploy_detection.py <deploy_detection.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: deploy_detection.ipynb <deploy_detection.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt
deleted file mode 100644
index a532819d8a..0000000000
--- a/docs/_sources/topic/vta/tutorials/frontend/index.rst.txt
+++ /dev/null
@@ -1,43 +0,0 @@
-
-
-.. _sphx_glr_topic_vta_tutorials_frontend:
-
-.. _vta-tutorial-frontend:
-
-Compile Deep Learning Models
-----------------------------
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny inference onto the...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/frontend/images/thumb/sphx_glr_deploy_detection_thumb.png
- :alt: Deploy Pretrained Vision Detection Model from Darknet on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Deploy Pretrained Vision Detection Model from Darknet on VTA</div>
- </div>
-
-
-.. raw:: html
-
- </div>
-
-
-.. toctree::
- :hidden:
-
- /topic/vta/tutorials/frontend/deploy_detection
-
diff --git a/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
deleted file mode 100644
index db4f0c2a96..0000000000
--- a/docs/_sources/topic/vta/tutorials/frontend/sg_execution_times.rst.txt
+++ /dev/null
@@ -1,12 +0,0 @@
-
-:orphan:
-
-.. _sphx_glr_topic_vta_tutorials_frontend_sg_execution_times:
-
-Computation times
-=================
-**01:06.989** total execution time for **topic_vta_tutorials_frontend** files:
-
-+--------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py` (``deploy_detection.py``) | 01:06.989 | 0.0 MB |
-+--------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/index.rst.txt b/docs/_sources/topic/vta/tutorials/index.rst.txt
deleted file mode 100644
index a8faf933fa..0000000000
--- a/docs/_sources/topic/vta/tutorials/index.rst.txt
+++ /dev/null
@@ -1,155 +0,0 @@
-:orphan:
-
-.. _vta-tutorials:
-
-VTA Tutorials
-=============
-This page contains tutorials about VTA and how to use TVM/Relay to target VTA.
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="In this tutorial, we will build on top of the vta-get-started tutorial and introduce additional...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/images/thumb/sphx_glr_matrix_multiply_thumb.png
- :alt: Simple Matrix Multiply
-
- :ref:`sphx_glr_topic_vta_tutorials_matrix_multiply.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Simple Matrix Multiply</div>
- </div>
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This is an introduction tutorial on how to use TVM to program the VTA design.">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/images/thumb/sphx_glr_vta_get_started_thumb.png
- :alt: Get Started with VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_vta_get_started.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Get Started with VTA</div>
- </div>
-
-
-.. raw:: html
-
- </div>
-
-
-.. toctree::
- :hidden:
-
- /topic/vta/tutorials/matrix_multiply
- /topic/vta/tutorials/vta_get_started
-
-
-Compile Deep Learning Models
-----------------------------
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an end-to-end demo, on how to run Darknet YoloV3-tiny inference onto the...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/frontend/images/thumb/sphx_glr_deploy_detection_thumb.png
- :alt: Deploy Pretrained Vision Detection Model from Darknet on VTA
-
- :ref:`sphx_glr_topic_vta_tutorials_frontend_deploy_detection.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Deploy Pretrained Vision Detection Model from Darknet on VTA</div>
- </div>
-
-
-.. raw:: html
-
- </div>
-
-Optimize Tensor Operators
--------------------------
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an overview on how to use TVM to map a 2D convolution workload efficient...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/optimize/images/thumb/sphx_glr_convolution_opt_thumb.png
- :alt: 2D Convolution Optimization
-
- :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">2D Convolution Optimization</div>
- </div>
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an overview on how to use TVM to map matrix multiplication efficiently o...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/optimize/images/thumb/sphx_glr_matrix_multiply_opt_thumb.png
- :alt: Matrix Multiply Blocking
-
- :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Matrix Multiply Blocking</div>
- </div>
-
-
-.. raw:: html
-
- </div>
-
-
-.. toctree::
- :hidden:
- :includehidden:
-
- /topic/vta/tutorials/frontend/index.rst
- /topic/vta/tutorials/optimize/index.rst
-
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/matrix_multiply.rst.txt b/docs/_sources/topic/vta/tutorials/matrix_multiply.rst.txt
deleted file mode 100644
index e8cfa7abd6..0000000000
--- a/docs/_sources/topic/vta/tutorials/matrix_multiply.rst.txt
+++ /dev/null
@@ -1,791 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/matrix_multiply.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_matrix_multiply.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/1ee0b869c5082223c5dfbb0fe4574252/matrix_multiply.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_matrix_multiply.py:
-
-
-.. _basic-mat-mult:
-
-Simple Matrix Multiply
-======================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-In this tutorial, we will build on top of the :ref:`vta-get-started` tutorial
-and introduce additional concepts required to implement matrix multiplication
-on VTA with the TVM workflow.
-
-.. GENERATED FROM PYTHON SOURCE LINES 30-34
-
-RPC Setup
----------
-We start by programming the Pynq's FPGA and building its RPC runtime
-as we did in the VTA introductory tutorial.
-
-.. GENERATED FROM PYTHON SOURCE LINES 34-73
-
-.. code-block:: default
-
-
- from __future__ import absolute_import, print_function
-
- import os
- import tvm
- from tvm import te
- import vta
- import numpy as np
- from tvm import rpc
- from tvm.contrib import utils
- from vta.testing import simulator
-
- # Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
- env = vta.get_env()
-
- # We read the Pynq RPC host IP address and port number from the OS environment
- host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
- # We configure both the bitstream and the runtime system on the Pynq
- # to match the VTA configuration specified by the vta_config.json file.
- if env.TARGET == "pynq" or env.TARGET == "de10nano":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
- # In simulation mode, host the RPC server locally.
- elif env.TARGET in ["sim", "tsim"]:
- remote = rpc.LocalSession()
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 74-91
-
-Computation Declaration
------------------------
-In this example we describe a simple matrix multiplication addition, which
-requires multiple computation stages, as shown in the dataflow diagram below.
-First we describe the input tensors :code:`A` and :code:`B` that are living
-in main memory.
-Second, we need to declare intermediate tensors :code:`A_buf` and
-:code:`B_buf`, which will live in VTA's on-chip buffers.
-Having this extra computational stage allows us to explicitly
-stage cached reads and writes.
-Third, we describe the matrix multiplication computation over
-:code:`A_buf` and :code:`B_buf` to produce the product matrix :code:`C_buf`.
-The last operation is a cast and copy back to DRAM, into results tensor
-:code:`C`.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/gemm_dataflow.png
- :align: center
-
-.. GENERATED FROM PYTHON SOURCE LINES 93-97
-
-Data Layout
-~~~~~~~~~~~
-We describe the placeholder tensors :code:`A`, and :code:`B` in a tiled data
-format to match the data layout requirements imposed by the VTA tensor core.
-
-.. GENERATED FROM PYTHON SOURCE LINES 99-141
-
-.. note::
-
- **Data Tiling**
-
- One source of complexity when targeting accelerators is to make sure
- that the data layout matches the layout imposed by the accelerator design.
- VTA is designed around a *tensor core* that performs, one matrix-matrix
- operation per cycle between an activation matrix and a weight matrix,
- adding the result matrix to an accumulator matrix, as shown in the
- figure below.
-
- .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/tensor_core.png
- :align: center
- :width: 480px
-
- The dimensions of that matrix-matrix multiplication are specified in
- the :code:`vta_config.json` configuration file.
- The activation matrix has a :code:`(BATCH, BLOCK_IN)` shape
- and the transposed weight matrix has a :code:`(BLOCK_OUT, BLOCK_IN)` shape,
- thus inferring that the resulting output matrix has a
- :code:`(BATCH, BLOCK_OUT)` shape.
- Consequently input and output tensors processed by VTA need to be
- tiled according to these aforementioned dimension.
-
- The diagram below shows the impact of data tiling on a matrix that is
- originally of shape (4, 8).
- Tiling by a (2, 2) tile shape ensures that data within each tile is
- contiguous.
- The resulting tiled tensor has a shape of (2, 4, 2, 2).
-
- .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/data_tiling.png
- :align: center
- :width: 480px
-
-We first define the variables :code:`m`, :code:`n`, :code:`o` to represent
-the shape of the matrix multiplication. These variables are multiplicative
-factors over the :code:`BLOCK_OUT`, :code:`BLOCK_IN`, and :code:`BATCH`
-tensor dimensions respectively.
-By default, the configuration file sets :code:`BATCH`, :code:`BLOCK_IN`, and
-:code:`BLOCK_OUT` to be 1, 16 and 16 respectively (:code:`BATCH` being set to
-1 implies that our compute building block is vector-matrix multiply).
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 143-163
-
-.. note::
-
- **Data Types**
-
- It's important to not only match the inner-tile
- dimension of VTA's tensor core, but also to match the specific data types
- expected by VTA.
- VTA for now only supports fixed point data types, which integer width is
- specified in the :code:`vta_config.json` file by :code:`INP_WIDTH` and
- :code:`WGT_WIDTH` for the activations and weights data types respectively.
- In addition, the accumulator data type integer width is specified by
- :code:`ACC_WIDTH`.
-
-By default, the configuration file sets :code:`INP_WIDTH`
-and :code:`WGT_WIDTH` to 8.
-The accumulator width :code:`ACC_WIDTH` is set to 32, in order to avoid
-overflow during accumulation.
-As a result, :code:`env.inp_dtype` and :code:`env.wgt_dtype` are all
-narrow 8-bit integers, while :code:`env.acc_dtype` is a standard 32-bit
-integer.
-
-.. GENERATED FROM PYTHON SOURCE LINES 163-179
-
-.. code-block:: default
-
-
- # Output channel factor m - total 16x16=256 output channels
- m = 16
- # Input channel factor n - total 16x16=256 input channels
- n = 16
- # Batch factor o (we use single batch inference)
- o = 1
- # A placeholder tensor in tiled data format
- A = te.placeholder((o, n, env.BATCH, env.BLOCK_IN), name="A", dtype=env.inp_dtype)
- # B placeholder tensor in tiled data format
- B = te.placeholder((m, n, env.BLOCK_OUT, env.BLOCK_IN), name="B", dtype=env.wgt_dtype)
- # A copy buffer
- A_buf = te.compute((o, n, env.BATCH, env.BLOCK_IN), lambda *i: A(*i), "A_buf")
- # B copy buffer
- B_buf = te.compute((m, n, env.BLOCK_OUT, env.BLOCK_IN), lambda *i: B(*i), "B_buf")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 180-200
-
-Matrix Multiplication
-~~~~~~~~~~~~~~~~~~~~~
-Now we're ready to describe the matrix multiplication result tensor :code:`C`,
-with another compute operation.
-The compute function takes the shape of the tensor, as well as a lambda
-function that describes the computation rule for each position of the tensor.
-
-In order to implement matrix multiplication, the lambda function needs to
-include a reduction formula over the input channel dimension axes.
-To create a reduction formula, we can declare a reduction axis using
-:code:`te.reduce_axis`, which takes in the range of reductions.
-:code:`te.sum` takes in the expression to be reduced as well as
-the reduction axes to compute the sum of value over all k in the declared
-ranges.
-
-Note that the reduction needs to be performed over 32-bit :code:`env.acc_dtype`
-accumulator data types.
-
-No computation happens during this phase, as we are only declaring how
-the computation should be done.
-
-.. GENERATED FROM PYTHON SOURCE LINES 200-215
-
-.. code-block:: default
-
-
- # Outer input feature reduction axis
- ko = te.reduce_axis((0, n), name="ko")
- # Inner input feature reduction axis
- ki = te.reduce_axis((0, env.BLOCK_IN), name="ki")
- # Describe the in-VTA matrix multiplication
- C_buf = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT),
- lambda bo, co, bi, ci: te.sum(
- A_buf[bo, ko, bi, ki].astype(env.acc_dtype) * B_buf[co, ko, ci, ki].astype(env.acc_dtype),
- axis=[ko, ki],
- ),
- name="C_buf",
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 216-220
-
-Casting the Results
-~~~~~~~~~~~~~~~~~~~
-After the computation is done, we'll need to send the results computed by VTA
-back to main memory.
-
-.. GENERATED FROM PYTHON SOURCE LINES 222-237
-
-.. note::
-
- **Memory Store Restrictions**
-
- One specificity of VTA is that it only supports DRAM stores in the narrow
- :code:`env.inp_dtype` data type format.
- This lets us reduce the data footprint for memory transfers, but also lets
- us quantize the wide accumulator data type down to a data format that
- matches the input activation data type.
- This means that in the context of neural network inference, the outputs
- of a given layer after activation can be consumed directly by the next
- layer.
-
-We perform one last typecast operation to the narrow
-input activation data format.
-
-.. GENERATED FROM PYTHON SOURCE LINES 237-243
-
-.. code-block:: default
-
-
- # Cast to output type, and send to main memory
- C = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT), lambda *i: C_buf(*i).astype(env.inp_dtype), name="C"
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 244-245
-
-This concludes the computation declaration part of this tutorial.
-
-.. GENERATED FROM PYTHON SOURCE LINES 247-260
-
-Scheduling the Computation
---------------------------
-While the above lines describes the computation rule, we can obtain
-:code:`C` in many ways.
-TVM asks the user to provide an implementation of the computation called
-*schedule*.
-
-A schedule is a set of transformations to an original computation that
-transforms the implementation of the computation without affecting
-correctness.
-This simple VTA programming tutorial aims to demonstrate basic schedule
-transformations that will map the original schedule down to VTA hardware
-primitives.
-
-.. GENERATED FROM PYTHON SOURCE LINES 263-267
-
-Default Schedule
-~~~~~~~~~~~~~~~~
-After we construct the schedule, by default the schedule computes
-:code:`C` in the following way:
-
-.. GENERATED FROM PYTHON SOURCE LINES 267-272
-
-.. code-block:: default
-
-
- # Let's take a look at the generated schedule
- s = te.create_schedule(C.op)
- print(tvm.lower(s, [A, B, C], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(A: T.Buffer((1, 16, 1, 16), "int8"), B: T.Buffer((16, 16, 16, 16), "int8"), C: T.Buffer((1, 16, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- A_buf = T.allocate([256], "int8", "global")
- B_buf = T.allocate([65536], "int8", "global")
- C_buf = T.allocate([256], "int32", "global")
- A_buf_1 = T.Buffer((256,), "int8", data=A_buf)
- for i1, i3 in T.grid(16, 16):
- cse_var_1: T.int32 = i1 * 16 + i3
- A_1 = T.Buffer((256,), "int8", data=A.data)
- A_buf_1[cse_var_1] = A_1[cse_var_1]
- B_buf_1 = T.Buffer((65536,), "int8", data=B_buf)
- for i0, i1, i2, i3 in T.grid(16, 16, 16, 16):
- cse_var_2: T.int32 = i0 * 4096 + i1 * 256 + i2 * 16 + i3
- B_1 = T.Buffer((65536,), "int8", data=B.data)
- B_buf_1[cse_var_2] = B_1[cse_var_2]
- C_buf_1 = T.Buffer((256,), "int32", data=C_buf)
- for co, ci in T.grid(16, 16):
- C_buf_1[co * 16 + ci] = 0
- for ko, ki in T.grid(16, 16):
- cse_var_3: T.int32 = co * 16 + ci
- C_buf_1[cse_var_3] = C_buf_1[cse_var_3] + T.Cast("int32", A_buf_1[ko * 16 + ki]) * T.Cast("int32", B_buf_1[co * 4096 + ko * 256 + ci * 16 + ki])
- for i1, i3 in T.grid(16, 16):
- cse_var_4: T.int32 = i1 * 16 + i3
- C_1 = T.Buffer((256,), "int8", data=C.data)
- C_1[cse_var_4] = T.Cast("int8", C_buf_1[cse_var_4])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 273-282
-
-Although this schedule makes sense, it won't compile to VTA.
-In order to obtain correct code generation, we need to apply scheduling
-primitives and code annotation that will transform the schedule into
-one that can be directly lowered onto VTA hardware intrinsics.
-Those include:
-
- - DMA copy operations which will take globally-scoped tensors and copy
- those into locally-scoped tensors.
- - Tensor operations that will perform the matrix multiplication.
-
-.. GENERATED FROM PYTHON SOURCE LINES 284-291
-
-Buffer Scopes
-~~~~~~~~~~~~~
-First, we set the scope of the buffers to tell TVM that these buffers
-will be living in the VTA's on-chip SRAM caches.
-Below, we tell TVM that :code:`A_buf`, :code:`B_buf`, :code:`C_buf`
-will respectively live in VTA's on-chip input, weight and accumulator
-memory.
-
-.. GENERATED FROM PYTHON SOURCE LINES 293-317
-
-.. note::
-
- **VTA's On-Chip SRAMs**
-
- VTA has three different memory scopes, each corresponding to different
- on-chip SRAM buffers.
-
- - :code:`env.inp_scope`: Input buffer, which is a read-only SRAM buffer
- that stores input matrices of shape :code:`(env.BATCH, env.BLOCK_IN)`
- of type :code:`env.inp_dtype`. The input buffer contains
- `2 ^ LOG_INP_BUFF_SIZE` matrix elements (as specified in the
- :code:`vta_config.json` file).
- - :code:`env.wgt_scope`: Weight buffer, which is a read-only SRAM buffer
- that stores weight matrices of shape :code:`(env.BLOCK_OUT, env.BLOCK_IN)`
- of type :code:`env.wgt_dtype`. The weight buffer contains
- `2 ^ LOG_WGT_BUFF_SIZE` matrix elements.
- - :code:`env.acc_scope`: Accumulator buffer, which is a read/write SRAM
- buffer that stores accumulator matrices of shape
- :code:`(env.BATCH, env.BLOCK_OUT)` of type :code:`env.acc_dtype`.
- The accumulator buffer is VTA's general purpose register file: it holds
- both intermediate results of convolutions and matrix multiplications
- as well as intermediate results of pooling, batch normalization, and
- activation layers. The accumulator buffer contains
- `2 ^ LOG_ACC_BUFF_SIZE` matrix elements.
-
-.. GENERATED FROM PYTHON SOURCE LINES 317-323
-
-.. code-block:: default
-
-
- # Set the intermediate tensor's scope to VTA's on-chip buffers
- s[A_buf].set_scope(env.inp_scope)
- s[B_buf].set_scope(env.wgt_scope)
- s[C_buf].set_scope(env.acc_scope)
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
-
- stage(C_buf, compute(C_buf, body=[T.reduce(T.comm_reducer(lambda x, y: x + y, [0]), source=[T.Cast("int32", A_buf[bo, ko, bi, ki]) * T.Cast("int32", B_buf[co, ko, ci, ki])], init=[], axis=[T.iter_var(ko, T.Range(0, 16), "CommReduce", ""), T.iter_var(ki, T.Range(0, 16), "CommReduce", "")], condition=T.bool(True), value_index=0)], axis=[T.iter_var(bo, T.Range(0, 1), "DataPar", ""), T.iter_var(co, T.Range(0, 16), "DataPar", ""), T.iter_var(bi, T.Range(0, 1), "DataPar", ""), T.iter_var(c [...]
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 324-337
-
-DMA Transfers
-~~~~~~~~~~~~~
-We need to schedule DMA transfers to move data living in DRAM to
-and from the VTA on-chip buffers.
-This can be achieved using the :code:`compute_at` schedule primitive
-which nests the copying of the buffers into the computation loop
-that performs the matrix multiplication.
-
-We insert :code:`dma_copy` pragmas to indicate to the compiler
-that the copy operations will be performed in bulk via DMA,
-which is common in hardware accelerators.
-Finally, we print the temporary schedule to observe the effects of
-moving the copy operations into the matrix multiplication loop.
-
-.. GENERATED FROM PYTHON SOURCE LINES 337-350
-
-.. code-block:: default
-
-
- # Move buffer copy into matrix multiply loop
- s[A_buf].compute_at(s[C_buf], ko)
- s[B_buf].compute_at(s[C_buf], ko)
-
- # Tag the buffer copies with the DMA pragma to insert a DMA transfer
- s[A_buf].pragma(s[A_buf].op.axis[0], env.dma_copy)
- s[B_buf].pragma(s[B_buf].op.axis[0], env.dma_copy)
- s[C].pragma(s[C].op.axis[0], env.dma_copy)
-
- # Let's take a look at the transformed schedule
- print(tvm.lower(s, [A, B, C], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(A: T.Buffer((1, 16, 1, 16), "int8"), B: T.Buffer((16, 16, 16, 16), "int8"), C: T.Buffer((1, 16, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- C_buf = T.allocate([256], "int32", "local.acc_buffer")
- A_buf = T.allocate([16], "int8", "local.inp_buffer")
- B_buf = T.allocate([16], "int8", "local.wgt_buffer")
- C_buf_1 = T.Buffer((256,), "int32", data=C_buf, scope="local.acc_buffer", align=16)
- for co, ci in T.grid(16, 16):
- C_buf_1[co * 16 + ci] = 0
- for ko in range(16):
- i0 = T.int32()
- A_buf_1 = T.Buffer((16,), "int8", data=A_buf, scope="local.inp_buffer", align=16)
- with T.attr(T.iter_var(i0, None, "DataPar", ""), "pragma_dma_copy", 1):
- for i3 in range(16):
- A_1 = T.Buffer((256,), "int8", data=A.data)
- A_buf_1[i3] = A_1[ko * 16 + i3]
- i0_1 = T.int32()
- B_buf_1 = T.Buffer((16,), "int8", data=B_buf, scope="local.wgt_buffer", align=256)
- with T.attr(T.iter_var(i0_1, None, "DataPar", ""), "pragma_dma_copy", 1):
- for i3 in range(16):
- B_1 = T.Buffer((65536,), "int8", data=B.data)
- B_buf_1[i3] = B_1[co * 4096 + ko * 256 + ci * 16 + i3]
- for ki in range(16):
- cse_var_1: T.int32 = co * 16 + ci
- C_buf_1[cse_var_1] = C_buf_1[cse_var_1] + T.Cast("int32", A_buf_1[ki]) * T.Cast("int32", B_buf_1[ki])
- i0 = T.int32()
- T.attr(T.iter_var(i0, None, "DataPar", ""), "pragma_dma_copy", 1)
- for i1, i3 in T.grid(16, 16):
- cse_var_2: T.int32 = i1 * 16 + i3
- C_1 = T.Buffer((256,), "int8", data=C.data)
- C_1[cse_var_2] = T.Cast("int8", C_buf_1[cse_var_2])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 351-370
-
-Tensorization
-~~~~~~~~~~~~~
-The last step of the schedule transformation consists in applying
-*tensorization* to our schedule.
-Tensorization is analogous to vectorization, but extends the concept
-to a higher-dimensional unit of computation.
-Consequently, tensorization imposes data layout constraints as discussed
-earlier when declaring the data layout input placeholders.
-We've already arranged our tensors in a tiled format, so the next thing
-we need to perform is loop reordering to accommodate for tensorization.
-
-Here we choose to move the outermost reduction axis all the way out.
-This dictates that we first iterate over input channels, then batch
-dimensions, and finally output channels.
-Lastly, we apply the tensorization scheduling primitive :code:`tensorize`
-along the outer axis of the inner-most matrix matrix multiplication tensor
-block.
-We print the finalized schedule that is ready for code-generation
-by the VTA runtime JIT compiler.
-
-.. GENERATED FROM PYTHON SOURCE LINES 370-379
-
-.. code-block:: default
-
-
- s[C_buf].reorder(
- ko, s[C_buf].op.axis[0], s[C_buf].op.axis[1], s[C_buf].op.axis[2], s[C_buf].op.axis[3], ki
- )
- s[C_buf].tensorize(s[C_buf].op.axis[2], env.gemm)
-
- # Let's take a look at the finalized schedule
- print(vta.lower(s, [A, B, C], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(A: T.Buffer((1, 16, 1, 16), "int8"), B: T.Buffer((16, 16, 16, 16), "int8"), C: T.Buffer((1, 16, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- vta = T.int32()
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2):
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushGEMMOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 0, 0)
- T.tir.vta.uop_push(0, 1, 0, 0, 0, 0, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 1)
- for ko in range(16):
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 1):
- T.tir.vta.coproc_dep_pop(2, 1)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), A.data, ko, 1, 1, 1, 0, 0, 0, 0, 0, 2)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), B.data, ko, 1, 16, 16, 0, 0, 0, 0, 0, 1)
- T.tir.vta.coproc_dep_push(1, 2)
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2)
- T.tir.vta.coproc_dep_pop(1, 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushGEMMOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 0, 1)
- T.tir.vta.uop_push(0, 0, 0, 0, 0, 0, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 1)
- T.tir.vta.coproc_dep_push(2, 3)
- T.tir.vta.coproc_dep_pop(2, 1)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 3):
- T.tir.vta.coproc_dep_pop(2, 3)
- T.call_extern("int32", "VTAStoreBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), 0, 4, C.data, 0, 16, 1, 16)
- T.tir.vta.coproc_sync()
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 380-381
-
-This concludes the scheduling portion of this tutorial.
-
-.. GENERATED FROM PYTHON SOURCE LINES 383-387
-
-TVM Compilation
----------------
-After we have finished specifying the schedule, we can compile it
-into a TVM function.
-
-.. GENERATED FROM PYTHON SOURCE LINES 387-403
-
-.. code-block:: default
-
-
- # Build GEMM VTA kernel
- my_gemm = vta.build(
- s, [A, B, C], tvm.target.Target("ext_dev", host=env.target_host), name="my_gemm"
- )
-
- # Write the compiled module into an object file.
- temp = utils.tempdir()
- my_gemm.save(temp.relpath("gemm.o"))
-
- # Send the executable over RPC
- remote.upload(temp.relpath("gemm.o"))
-
- # Load the compiled module
- f = remote.load_module("gemm.o")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 404-418
-
-Running the Function
---------------------
-The compiled TVM function uses a concise C API and can be invoked from
-code language.
-
-TVM provides an array API in python to aid quick testing and prototyping.
-The array API is based on `DLPack <https://github.com/dmlc/dlpack>`_ standard.
-
-- We first create a remote context (for remote execution on the Pynq).
-- Then :code:`tvm.nd.array` formats the data accordingly.
-- :code:`f()` runs the actual computation.
-- :code:`numpy()` copies the result array back in a format that can be
- interpreted.
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 418-442
-
-.. code-block:: default
-
-
- # Get the remote device context
- ctx = remote.ext_dev(0)
-
- # Initialize the A and B arrays randomly in the int range of (-128, 128]
- A_orig = np.random.randint(-128, 128, size=(o * env.BATCH, n * env.BLOCK_IN)).astype(A.dtype)
- B_orig = np.random.randint(-128, 128, size=(m * env.BLOCK_OUT, n * env.BLOCK_IN)).astype(B.dtype)
-
- # Apply packing to the A and B arrays from a 2D to a 4D packed layout
- A_packed = A_orig.reshape(o, env.BATCH, n, env.BLOCK_IN).transpose((0, 2, 1, 3))
- B_packed = B_orig.reshape(m, env.BLOCK_OUT, n, env.BLOCK_IN).transpose((0, 2, 1, 3))
-
- # Format the input/output arrays with tvm.nd.array to the DLPack standard
- A_nd = tvm.nd.array(A_packed, ctx)
- B_nd = tvm.nd.array(B_packed, ctx)
- C_nd = tvm.nd.array(np.zeros((o, m, env.BATCH, env.BLOCK_OUT)).astype(C.dtype), ctx)
-
- # Clear stats
- if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
-
- # Invoke the module to perform the computation
- f(A_nd, B_nd, C_nd)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 443-447
-
-Verifying Correctness
----------------------
-Compute the reference result with numpy and assert that the output of the
-matrix multiplication indeed is correct
-
-.. GENERATED FROM PYTHON SOURCE LINES 447-462
-
-.. code-block:: default
-
-
- # Compute reference result with numpy
- C_ref = np.dot(A_orig.astype(env.acc_dtype), B_orig.T.astype(env.acc_dtype)).astype(C.dtype)
- C_ref = C_ref.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
- np.testing.assert_equal(C_ref, C_nd.numpy())
-
- # Print stats
- if env.TARGET in ["sim", "tsim"]:
- sim_stats = simulator.stats()
- print("Execution statistics:")
- for k, v in sim_stats.items():
- print("\t{:<16}: {:>16}".format(k, v))
-
- print("Successful matrix multiply test!")
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Execution statistics:
- inp_load_nbytes : 256
- wgt_load_nbytes : 65536
- acc_load_nbytes : 0
- uop_load_nbytes : 8
- out_store_nbytes: 256
- gemm_counter : 256
- alu_counter : 0
- Successful matrix multiply test!
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 463-475
-
-Summary
--------
-This tutorial showcases the TVM workflow to implement a simple matrix
-multiplication example on VTA.
-The general workflow includes:
-
-- Programming the FPGA with the VTA bitstream over RPC.
-- Describing matrix multiplication via a series of computations.
-- Describing how we want to perform the computation using schedule primitives.
-- Compiling the function to the VTA target.
-- Running the compiled module and verifying it against a numpy implementation.
-
-
-
-.. _sphx_glr_download_topic_vta_tutorials_matrix_multiply.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: matrix_multiply.py <matrix_multiply.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: matrix_multiply.ipynb <matrix_multiply.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/optimize/convolution_opt.rst.txt b/docs/_sources/topic/vta/tutorials/optimize/convolution_opt.rst.txt
deleted file mode 100644
index f0c9d678ee..0000000000
--- a/docs/_sources/topic/vta/tutorials/optimize/convolution_opt.rst.txt
+++ /dev/null
@@ -1,806 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/optimize/convolution_opt.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_optimize_convolution_opt.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/b3f997c945cc7de3e03a1e0c4c73fabd/convolution_opt.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py:
-
-
-2D Convolution Optimization
-===========================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This tutorial provides an overview on how to use TVM to map a 2D convolution
-workload efficiently on the VTA design.
-We recommend covering the :ref:`vta-mat-mult-opt` tutorial first.
-
-2D convolution is dominant in most computer vision deep neural networks.
-In this tutorial, we will demonstrate TVM schedule optimizations to map
-2D convolution operators in NCHW layout onto VTA.
-We also introduce the notion of latency hiding, which allows us to
-maximize VTA's compute and memory resource utilization.
-
-.. GENERATED FROM PYTHON SOURCE LINES 34-37
-
-RPC Setup
----------
-We start by programming the Pynq's FPGA and building its RPC runtime.
-
-.. GENERATED FROM PYTHON SOURCE LINES 37-78
-
-.. code-block:: default
-
-
- from __future__ import absolute_import, print_function
-
- import os
- import tvm
- import tvm.testing
- from tvm import te
- import vta
- import numpy as np
-
- from tvm import rpc
- from tvm.contrib import utils
- from vta.testing import simulator
-
- # Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
- env = vta.get_env()
-
- # We read the Pynq RPC host IP address and port number from the OS environment
- host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
- # We configure both the bitstream and the runtime system on the Pynq
- # to match the VTA configuration specified by the vta_config.json file.
- if env.TARGET == "pynq":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
- # In simulation mode, host the RPC server locally.
- elif env.TARGET in ["sim", "tsim"]:
- remote = rpc.LocalSession()
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 79-126
-
-Computation Declaration
------------------------
-As a first step, we need to describe our 2D convolution computation
-in NCHW format.
-
-We define the 2D convolution shape by the batch size,
-spatial dimensions, input channels, output channels, kernel dimensions,
-kernel dimensions, padding dimensions, and stride dimensions.
-
-We pick the shape of the 9th convolutional layer of the ResNet-18
-architecture as our convolution workload parameters.
-
-We've added extra operators to the 2D convolution that apply
-shifting and clipping to the output in order to mimic a fixed-point
-convolution followed by a rectified linear activation.
-We describe the TVM dataflow graph of the 2D convolution layer below:
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/conv2d_dataflow.png
- :align: center
-
-This computation is intentionally too large to fit onto VTA's on-chip
-buffers all at once. Therefore in the scheduling phase we'll
-rely on computation blocking strategies to break the computation down into
-manageable chunks.
-
-.. note::
-
- *Spatial padding*
-
- Note that we'll need to import the TOPI library to apply spatial padding
- on the input feature map tensor.
- Spatial padding facilitates blocking in the context of 2D convolutions
- due to the fact that the same (x, y) spatial location of the input
- feature map of any given layer is read more than once if the convolution
- kernel window size is greater than one.
- On CPUs, and GPUs, one way to increase efficiency of memory accesses
- when parallelizing work is spatial packing, which requires data re-layout.
- VTA load DMA engine can insert padding automatically so that the original
- input feature map does not have to be re-packed in memory.
-
- We show the effect of VTA's on the fly spatial padding when data is being
- loaded from DRAM into VTA's SRAM, following a 2D strided and padded memory
- read.
-
- .. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/padding.png
- :align: center
- :width: 480px
-
-.. GENERATED FROM PYTHON SOURCE LINES 126-215
-
-.. code-block:: default
-
-
- from tvm import topi
-
- # 2D convolution layer dimensions taken from ResNet-18 architecture
- # (9th convolutional layer)
- batch_size = 1
- height = 14
- width = 14
- in_channels = 256
- out_channels = 256
- kernel_h = 3
- kernel_w = 3
- pad_h = 1
- pad_w = 1
- stride_h = 1
- stride_w = 1
- assert batch_size % env.BATCH == 0
- assert in_channels % env.BLOCK_IN == 0
- assert out_channels % env.BLOCK_OUT == 0
-
- # Input feature map: (N, IC, H, W, n, ic)
- data_shape = (
- batch_size // env.BATCH,
- in_channels // env.BLOCK_IN,
- height,
- width,
- env.BATCH,
- env.BLOCK_IN,
- )
- # Kernel: (OC, IC, H, W, oc, ic)
- kernel_shape = (
- out_channels // env.BLOCK_OUT,
- in_channels // env.BLOCK_IN,
- kernel_h,
- kernel_w,
- env.BLOCK_OUT,
- env.BLOCK_IN,
- )
- # Derive output feature map dimensions
- fout_height = (height + 2 * pad_h - kernel_h) // stride_h + 1
- fout_width = (width + 2 * pad_w - kernel_w) // stride_w + 1
- # Output feature map: (N, OC, H, W, n, oc)
- output_shape = (
- batch_size // env.BATCH,
- out_channels // env.BLOCK_OUT,
- fout_height,
- fout_width,
- env.BATCH,
- env.BLOCK_OUT,
- )
-
- # Convolution reduction axes
- dy = te.reduce_axis((0, kernel_h), name="dy")
- dx = te.reduce_axis((0, kernel_w), name="dx")
- ic = te.reduce_axis((0, in_channels // env.BLOCK_IN), name="ic")
- ic_tns = te.reduce_axis((0, env.BLOCK_IN), name="ic_tns")
-
- # Input placeholder tensors
- data = te.placeholder(data_shape, name="data", dtype=env.inp_dtype)
- kernel = te.placeholder(kernel_shape, name="kernel", dtype=env.wgt_dtype)
-
- # Copy buffers:
- # Apply spatial padding to input feature map
- data_buf = topi.nn.pad(data, [0, 0, pad_h, pad_w, 0, 0], name="data_buf")
- kernel_buf = te.compute(kernel_shape, lambda *i: kernel(*i), "kernel_buf")
-
- # Declare 2D convolution
- res_conv = te.compute(
- output_shape,
- lambda bo, co, i, j, bi, ci: te.sum(
- data_buf[bo, ic, i * stride_h + dy, j * stride_w + dx, bi, ic_tns].astype(env.acc_dtype)
- * kernel_buf[co, ic, dy, dx, ci, ic_tns].astype(env.acc_dtype),
- axis=[ic, dy, dx, ic_tns],
- ),
- name="res_conv",
- )
-
- # Add shift stage for fix-point normalization
- res_shr = te.compute(output_shape, lambda *i: res_conv(*i) >> 8, name="res_shr")
-
- # Apply clipping between (0, input max value)
- inp_max = (1 << (env.INP_WIDTH - 1)) - 1
- res_max = te.compute(output_shape, lambda *i: tvm.te.max(res_shr(*i), 0), "res_max")
- res_min = te.compute(output_shape, lambda *i: tvm.te.min(res_max(*i), inp_max), "res_min")
-
- # Result Tensor
- res = te.compute(output_shape, lambda *i: res_min(*i).astype(env.inp_dtype), name="res")
-
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 216-225
-
-Scheduling the Computation
---------------------------
-We'll look at a set of schedule transformations necessary to map the
-2D convolution onto VTA in an efficient fashion.
-Those include:
-
-- Computation blocking
-- Virtual threading to increase compute utilization
-- Lowering to VTA hardware intrinsics
-
-.. GENERATED FROM PYTHON SOURCE LINES 225-231
-
-.. code-block:: default
-
-
- # Create TVM schedule
- s = te.create_schedule(res.op)
- # Let's look at the default TVM schedule
- print(tvm.lower(s, [data, kernel, res], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(data: T.Buffer((1, 16, 14, 14, 1, 16), "int8"), kernel: T.Buffer((16, 16, 3, 3, 16, 16), "int8"), res: T.Buffer((1, 16, 14, 14, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- data_buf = T.allocate([65536], "int8", "global")
- kernel_buf = T.allocate([589824], "int8", "global")
- res_conv = T.allocate([50176], "int32", "global")
- data_buf_1 = T.Buffer((65536,), "int8", data=data_buf)
- for i1, i2, i3, i5 in T.grid(16, 16, 16, 16):
- cse_var_1: T.int32 = i3 * 16
- data_1 = T.Buffer((50176,), "int8", data=data.data)
- data_buf_1[i1 * 4096 + i2 * 256 + cse_var_1 + i5] = T.if_then_else(1 <= i2 and i2 < 15 and 1 <= i3 and i3 < 15, data_1[i1 * 3136 + i2 * 224 + cse_var_1 + i5 - 240], T.int8(0))
- kernel_buf_1 = T.Buffer((589824,), "int8", data=kernel_buf)
- for i0, i1, i2, i3, i4, i5 in T.grid(16, 16, 3, 3, 16, 16):
- cse_var_2: T.int32 = i0 * 36864 + i1 * 2304 + i2 * 768 + i3 * 256 + i4 * 16 + i5
- kernel_1 = T.Buffer((589824,), "int8", data=kernel.data)
- kernel_buf_1[cse_var_2] = kernel_1[cse_var_2]
- res_conv_1 = T.Buffer((50176,), "int32", data=res_conv)
- for co, i, j, ci in T.grid(16, 14, 14, 16):
- res_conv_1[co * 3136 + i * 224 + j * 16 + ci] = 0
- for ic, dy, dx, ic_tns in T.grid(16, 3, 3, 16):
- cse_var_4: T.int32 = j * 16
- cse_var_3: T.int32 = co * 3136 + i * 224 + cse_var_4 + ci
- res_conv_1[cse_var_3] = res_conv_1[cse_var_3] + T.Cast("int32", data_buf_1[ic * 4096 + i * 256 + dy * 256 + cse_var_4 + dx * 16 + ic_tns]) * T.Cast("int32", kernel_buf_1[co * 36864 + ic * 2304 + dy * 768 + dx * 256 + ci * 16 + ic_tns])
- res_conv_2 = T.Buffer((50176,), "int32", data=res_conv)
- for i1, i2, i3, i5 in T.grid(16, 14, 14, 16):
- cse_var_5: T.int32 = i1 * 3136 + i2 * 224 + i3 * 16 + i5
- res_conv_2[cse_var_5] = T.shift_right(res_conv_1[cse_var_5], 8)
- res_conv_3 = T.Buffer((50176,), "int32", data=res_conv)
- for i1, i2, i3, i5 in T.grid(16, 14, 14, 16):
- cse_var_6: T.int32 = i1 * 3136 + i2 * 224 + i3 * 16 + i5
- res_conv_3[cse_var_6] = T.max(res_conv_2[cse_var_6], 0)
- res_conv_4 = T.Buffer((50176,), "int32", data=res_conv)
- for i1, i2, i3, i5 in T.grid(16, 14, 14, 16):
- cse_var_7: T.int32 = i1 * 3136 + i2 * 224 + i3 * 16 + i5
- res_conv_4[cse_var_7] = T.min(res_conv_3[cse_var_7], 127)
- for i1, i2, i3, i5 in T.grid(16, 14, 14, 16):
- cse_var_8: T.int32 = i1 * 3136 + i2 * 224 + i3 * 16 + i5
- res_1 = T.Buffer((50176,), "int8", data=res.data)
- res_1[cse_var_8] = T.Cast("int8", res_conv_4[cse_var_8])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 232-241
-
-Blocking the Computation
-~~~~~~~~~~~~~~~~~~~~~~~~
-The 2D convolution is by default too large for activations or kernel weights
-to fit on VTA's on-chip buffers all at once.
-We apply blocking along input channels, output channels, and along
-the height spatial dimensions.
-We don't apply blocking along the width spatial dimension since it's
-the innermost dimension in the NCHW layout (and consequently to increase
-locality, it's best not to block along the innermost dimension).
-
-.. GENERATED FROM PYTHON SOURCE LINES 241-282
-
-.. code-block:: default
-
-
- # Let's define tiling sizes
- b_block = 1 // env.BATCH
- oc_block = 128 // env.BLOCK_OUT
- ic_block = 16 // env.BLOCK_IN
- h_block = 7
- w_block = 14
-
- # Tile the output tensor along the spatial and output channel dimensions
- # (since by default we are doing single batch inference, the split along
- # the batch dimension has no effect)
- b, oc, y, x, b_tns, oc_tns = s[res].op.axis
- b_out, b_inn = s[res].split(b, factor=b_block)
- oc_out, oc_inn = s[res].split(oc, factor=oc_block)
- y_out, y_inn = s[res].split(y, factor=h_block)
- x_out, x_inn = s[res].split(x, factor=w_block)
- s[res].reorder(b_out, oc_out, y_out, x_out, b_inn, oc_inn, y_inn, x_inn, b_tns, oc_tns)
-
- # Move intermediate computation into each output compute tile
- s[res_conv].compute_at(s[res], x_out)
- s[res_shr].compute_at(s[res], x_out)
- s[res_max].compute_at(s[res], x_out)
- s[res_min].compute_at(s[res], x_out)
-
- # Apply additional loop split along reduction axis (input channel)
- b_inn, oc_inn, y_inn, x_inn, b_tns, oc_tns = s[res_conv].op.axis
- ic_out, ic_inn = s[res_conv].split(ic, factor=ic_block)
-
- # Reorder axes.
- # 1) Group the VTA tensor axes in the inner most position: b_tns, oc_tns, ic_tns
- # to allow TVM to tensorize.
- # 2) We move the ic_out axis all the way out of the convolution loop to block
- # along the reduction axis.
- # 3) Now we re-order the block axes: b_inn, oc_inn, y_inn, x_inn, ic_inn, dy, dx.
- # VTA runtime/hardware requires us to write to a different output feature map
- # location for every VTA tensor operation.
- # This restriction requires us to order one of oc_inn, y_inn or x_inn right
- # before b_tns, since they all affect output feature map indexing.
- # Therefore, we choose to bring x_inn inside as shown below.
- s[res_conv].reorder(ic_out, b_inn, oc_inn, y_inn, ic_inn, dy, dx, x_inn, b_tns, oc_tns, ic_tns)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 283-298
-
-Virtual Threading
-~~~~~~~~~~~~~~~~~
-Virtual threading is a mechanism that increases task-level pipeline
-parallelism in the VTA hardware design.
-Put it another way, it increases compute resource utilization by hiding
-memory access latency.
-
-In the implementation below, virtual threading distributes work across two
-threads split along the output channel axis.
-We show how work is split when computing the 2D convolution in the figure
-below.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/virtual_threading.png
- :align: center
- :width: 480px
-
-.. GENERATED FROM PYTHON SOURCE LINES 298-310
-
-.. code-block:: default
-
-
- # VTA only supports 2 virtual threads
- v_threads = 2
-
- # Perform virtual thread split along output channel outer axis
- _, tx = s[res].split(oc_out, factor=v_threads)
- s[res].reorder(tx, b_out)
- s[res].bind(tx, te.thread_axis("cthread"))
-
- # Let's look at the current TVM schedule after blocking and virtual threading
- print(tvm.lower(s, [data, kernel, res], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(data: T.Buffer((1, 16, 14, 14, 1, 16), "int8"), kernel: T.Buffer((16, 16, 3, 3, 16, 16), "int8"), res: T.Buffer((1, 16, 14, 14, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- data_buf = T.allocate([65536], "int8", "global")
- kernel_buf = T.allocate([589824], "int8", "global")
- res_conv = T.allocate([25088], "int32", "global")
- data_buf_1 = T.Buffer((65536,), "int8", data=data_buf)
- for i1, i2, i3, i5 in T.grid(16, 16, 16, 16):
- cse_var_1: T.int32 = i3 * 16
- data_1 = T.Buffer((50176,), "int8", data=data.data)
- data_buf_1[i1 * 4096 + i2 * 256 + cse_var_1 + i5] = T.if_then_else(1 <= i2 and i2 < 15 and 1 <= i3 and i3 < 15, data_1[i1 * 3136 + i2 * 224 + cse_var_1 + i5 - 240], T.int8(0))
- kernel_buf_1 = T.Buffer((589824,), "int8", data=kernel_buf)
- for i0, i1, i2, i3, i4, i5 in T.grid(16, 16, 3, 3, 16, 16):
- cse_var_2: T.int32 = i0 * 36864 + i1 * 2304 + i2 * 768 + i3 * 256 + i4 * 16 + i5
- kernel_1 = T.Buffer((589824,), "int8", data=kernel.data)
- kernel_buf_1[cse_var_2] = kernel_1[cse_var_2]
- for i2_outer in range(2):
- res_conv_1 = T.Buffer((157351936,), "int32", data=res_conv)
- for co_init, i_init, j_init, ci_init in T.grid(8, 7, 14, 16):
- cse_var_3: T.int32 = co_init * 1568 + i_init * 224 + j_init * 16 + ci_init
- res_conv_1[cse_var_3] = 0
- res_conv_1[cse_var_3 + 12544] = 0
- for ic_outer, co, i, dy, dx, j, ci, ic_tns in T.grid(16, 8, 7, 3, 3, 14, 16, 16):
- cse_var_8: T.int32 = j * 16
- cse_var_7: T.int32 = co * 1568 + i * 224 + cse_var_8 + ci
- cse_var_6: T.int32 = cse_var_7 + 12544
- cse_var_5: T.int32 = co * 36864 + ic_outer * 2304 + dy * 768 + dx * 256 + ci * 16 + ic_tns
- cse_var_4: T.int32 = ic_outer * 4096 + i2_outer * 1792 + i * 256 + dy * 256 + cse_var_8 + dx * 16 + ic_tns
- res_conv_1[cse_var_7] = res_conv_1[cse_var_7] + T.Cast("int32", data_buf_1[cse_var_4]) * T.Cast("int32", kernel_buf_1[cse_var_5])
- res_conv_1[cse_var_6] = res_conv_1[cse_var_6] + T.Cast("int32", data_buf_1[cse_var_4]) * T.Cast("int32", kernel_buf_1[cse_var_5 + 294912])
- res_conv_2 = T.Buffer((157351936,), "int32", data=res_conv)
- for i1, i2, i3, i5 in T.grid(8, 7, 14, 16):
- cse_var_10: T.int32 = i1 * 1568 + i2 * 224 + i3 * 16 + i5
- cse_var_9: T.int32 = cse_var_10 + 12544
- res_conv_2[cse_var_10] = T.shift_right(res_conv_1[cse_var_10], 8)
- res_conv_2[cse_var_9] = T.shift_right(res_conv_1[cse_var_9], 8)
- res_conv_3 = T.Buffer((157351936,), "int32", data=res_conv)
- for i1, i2, i3, i5 in T.grid(8, 7, 14, 16):
- cse_var_12: T.int32 = i1 * 1568 + i2 * 224 + i3 * 16 + i5
- cse_var_11: T.int32 = cse_var_12 + 12544
- res_conv_3[cse_var_12] = T.max(res_conv_2[cse_var_12], 0)
- res_conv_3[cse_var_11] = T.max(res_conv_2[cse_var_11], 0)
- res_conv_4 = T.Buffer((157351936,), "int32", data=res_conv)
- for i1, i2, i3, i5 in T.grid(8, 7, 14, 16):
- cse_var_14: T.int32 = i1 * 1568 + i2 * 224 + i3 * 16 + i5
- cse_var_13: T.int32 = cse_var_14 + 12544
- res_conv_4[cse_var_14] = T.min(res_conv_3[cse_var_14], 127)
- res_conv_4[cse_var_13] = T.min(res_conv_3[cse_var_13], 127)
- for i1_inner, i2_inner, i3_inner, i5 in T.grid(8, 7, 14, 16):
- cse_var_18: T.int32 = i2_inner * 224
- cse_var_17: T.int32 = i3_inner * 16
- cse_var_16: T.int32 = i1_inner * 1568 + cse_var_18 + cse_var_17 + i5
- cse_var_15: T.int32 = i1_inner * 3136 + i2_outer * 1568 + cse_var_18 + cse_var_17 + i5
- res_1 = T.Buffer((50176,), "int8", data=res.data)
- res_1[cse_var_15] = T.Cast("int8", res_conv_4[cse_var_16])
- res_1[cse_var_15 + 25088] = T.Cast("int8", res_conv_4[cse_var_16 + 12544])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 311-318
-
-Lowering Copies to DMA Transfers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Next we set the buffer scopes to the corresponding on-chip VTA SRAM buffers.
-We move the load loops into the 2D convolution computation loop to stage
-memory loads such that they fit in the on-chip SRAM buffers.
-Finally we annotate the load/store loop outer axes with the DMA copy pragma
-to perform bulk memory transfers on VTA.
-
-.. GENERATED FROM PYTHON SOURCE LINES 318-340
-
-.. code-block:: default
-
-
- # Set scope of SRAM buffers
- s[data_buf].set_scope(env.inp_scope)
- s[kernel_buf].set_scope(env.wgt_scope)
- s[res_conv].set_scope(env.acc_scope)
- s[res_shr].set_scope(env.acc_scope)
- s[res_min].set_scope(env.acc_scope)
- s[res_max].set_scope(env.acc_scope)
-
- # Block data and kernel cache reads
- s[data_buf].compute_at(s[res_conv], ic_out)
- s[kernel_buf].compute_at(s[res_conv], ic_out)
-
- # Use DMA copy pragma on DRAM->SRAM operations
- s[data_buf].pragma(s[data_buf].op.axis[0], env.dma_copy)
- s[kernel_buf].pragma(s[kernel_buf].op.axis[0], env.dma_copy)
-
- # Use DMA copy pragma on SRAM->DRAM operation in each result block
- # (this implies that these copies should be performed along b_inn,
- # or result axis 4)
- s[res].pragma(s[res].op.axis[4], env.dma_copy)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 341-346
-
-Lowering Computation to VTA Compute Intrinsics
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The last phase is to lower the computation loops down to VTA hardware
-intrinsics by mapping the 2D convolution to tensor intrinsics,
-and mapping the shift, and clipping computation to the vector ALU.
-
-.. GENERATED FROM PYTHON SOURCE LINES 346-360
-
-.. code-block:: default
-
-
- # Apply tensorization over the batch tensor tile axis
- s[res_conv].tensorize(b_tns, env.gemm)
-
- # Add an ALU pragma over the shift and clipping operations
- s[res_shr].pragma(s[res_shr].op.axis[0], env.alu)
- s[res_min].pragma(s[res_min].op.axis[0], env.alu)
- s[res_max].pragma(s[res_max].op.axis[0], env.alu)
-
- # Let's look at the final lowered TVM schedule after lowering memory
- # loads/stores down to DMA copy intrinsics, and the computation down to
- # VTA compute intrinsics.
- print(vta.lower(s, [data, kernel, res], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(data: T.Buffer((1, 16, 14, 14, 1, 16), "int8"), kernel: T.Buffer((16, 16, 3, 3, 16, 16), "int8"), res: T.Buffer((1, 16, 14, 14, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- T.tir.vta.coproc_dep_push(3, 2)
- T.tir.vta.coproc_dep_push(3, 2)
- for i2_outer in range(2):
- for cthread_s in range(2):
- vta = T.int32()
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2)
- T.tir.vta.coproc_dep_pop(3, 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushGEMMOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 8, 98, 0, 0)
- T.call_extern("int32", "VTAUopLoopBegin", 7, 14, 0, 0)
- for j_init in range(14):
- T.tir.vta.uop_push(0, 1, cthread_s * 784 + j_init, 0, 0, 0, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 1)
- vta = T.int32()
- for ic_outer in range(16):
- cse_var_6: T.int32 = i2_outer * 7
- cse_var_5: T.int32 = ic_outer * 9
- cse_var_4: T.int32 = T.max(1 - cse_var_6, 0)
- cse_var_3: T.int32 = T.max(cse_var_6 - 6, 0)
- cse_var_2: T.int32 = 9 - cse_var_4 - cse_var_3
- cse_var_1: T.int32 = ic_outer * 196 + i2_outer * 98 + cse_var_4 * 14 - 14
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 1):
- T.tir.vta.coproc_dep_pop(2, 1)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), data.data, cse_var_1, 14, cse_var_2, 14, 1, cse_var_4, 1, cse_var_3, 0, 2)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), kernel.data, cse_var_5, 9, 8, 144, 0, 0, 0, 0, 0, 1)
- T.tir.vta.coproc_dep_push(1, 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 1):
- T.tir.vta.coproc_dep_pop(2, 1)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), data.data, cse_var_1, 14, cse_var_2, 14, 1, cse_var_4, 1, cse_var_3, 144, 2)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), kernel.data, cse_var_5 + 1152, 9, 8, 144, 0, 0, 0, 0, 72, 1)
- T.tir.vta.coproc_dep_push(1, 2)
- for cthread_s in range(2):
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2)
- T.tir.vta.coproc_dep_pop(1, 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushGEMMOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 8, 98, 0, 9)
- T.call_extern("int32", "VTAUopLoopBegin", 7, 14, 16, 0)
- for dy, dx, j in T.grid(3, 3, 14):
- T.tir.vta.uop_push(0, 0, cthread_s * 784 + j, cthread_s * 144 + dy * 16 + j + dx, cthread_s * 72 + dy * 3 + dx, 0, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 1)
- T.tir.vta.coproc_dep_pop(2, 1)
- T.tir.vta.coproc_dep_pop(2, 1)
- for cthread_s in range(2):
- cse_var_7: T.int32 = cthread_s * 784
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 784, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, cse_var_7, cse_var_7, 0, 3, 1, 8)
- T.call_extern("int32", "VTAUopLoopEnd")
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 784, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, cse_var_7, cse_var_7, 0, 1, 1, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 784, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, cse_var_7, cse_var_7, 0, 0, 1, 127)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 3)
- for cthread_s in range(2):
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 3)
- T.tir.vta.coproc_dep_pop(2, 3)
- for i1_inner, i2_inner, i3_inner in T.grid(8, 7, 14):
- cse_var_8: T.int32 = i2_inner * 14
- T.call_extern("int32", "VTAStoreBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), cthread_s * 784 + i1_inner * 98 + cse_var_8 + i3_inner, 4, res.data, cthread_s * 1568 + i1_inner * 196 + i2_outer * 98 + cse_var_8 + i3_inner, 1, 1, 1)
- T.tir.vta.coproc_dep_push(3, 2)
- T.tir.vta.coproc_dep_pop(3, 2)
- T.tir.vta.coproc_dep_pop(3, 2)
- T.tir.vta.coproc_sync()
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 361-367
-
-TVM Compilation and Verification
---------------------------------
-After specifying the schedule, we can compile it into a TVM function.
-We save the module so we can send it over RPC.
-We run the function and verify it against a numpy implementation to
-ensure correctness.
-
-.. GENERATED FROM PYTHON SOURCE LINES 367-451
-
-.. code-block:: default
-
-
- # This library facilitates 2D convolution testing
- from tvm.topi.testing import conv2d_nchw_python
-
- # Compile the TVM module
- with vta.build_config(disabled_pass={"tir.CommonSubexprElimTIR"}):
- my_conv = vta.build(
- s, [data, kernel, res], tvm.target.Target("ext_dev", host=env.target_host), name="my_conv"
- )
- temp = utils.tempdir()
- my_conv.save(temp.relpath("conv2d.o"))
- remote.upload(temp.relpath("conv2d.o"))
- f = remote.load_module("conv2d.o")
-
- # Get the remote device context
- ctx = remote.ext_dev(0)
-
- # Initialize the data and kernel arrays randomly in the int range
- # of (-128, 128] in NCHW layout
- data_np = np.random.randint(-128, 128, size=(batch_size, in_channels, height, width)).astype(
- data.dtype
- )
- kernel_np = np.random.randint(
- -128, 128, size=(out_channels, in_channels, kernel_h, kernel_w)
- ).astype(kernel.dtype)
-
- # Apply packing to the data and kernel arrays from a 2D NCHW
- # to a 4D NCHWnc packed layout
- data_packed = data_np.reshape(
- batch_size // env.BATCH, env.BATCH, in_channels // env.BLOCK_IN, env.BLOCK_IN, height, width
- ).transpose((0, 2, 4, 5, 1, 3))
-
- kernel_packed = kernel_np.reshape(
- out_channels // env.BLOCK_OUT,
- env.BLOCK_OUT,
- in_channels // env.BLOCK_IN,
- env.BLOCK_IN,
- kernel_h,
- kernel_w,
- ).transpose((0, 2, 4, 5, 1, 3))
-
- # Format the input/output arrays with tvm.nd.array to the DLPack standard
- data_nd = tvm.nd.array(data_packed, ctx)
- kernel_nd = tvm.nd.array(kernel_packed, ctx)
- res_nd = tvm.nd.array(np.zeros(output_shape).astype(res.dtype), ctx)
-
- # Clear stats
- if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
-
- # Invoke the module to perform the computation
- f(data_nd, kernel_nd, res_nd)
-
- # Verify against numpy implementation
- res_ref = conv2d_nchw_python(
- data_np.astype(env.acc_dtype),
- kernel_np.astype(env.acc_dtype),
- (stride_h, stride_w),
- (pad_h, pad_w),
- ).astype(env.acc_dtype)
- res_ref = res_ref >> env.INP_WIDTH
- res_ref = np.clip(res_ref, 0, inp_max)
- res_ref = res_ref.astype(res.dtype)
- res_ref = res_ref.reshape(
- (
- batch_size // env.BATCH,
- env.BATCH,
- out_channels // env.BLOCK_OUT,
- env.BLOCK_OUT,
- fout_height,
- fout_width,
- )
- ).transpose((0, 2, 4, 5, 1, 3))
- tvm.testing.assert_allclose(res_ref, res_nd.numpy())
-
- # Print stats
- if env.TARGET in ["sim", "tsim"]:
- sim_stats = simulator.stats()
- print("Execution statistics:")
- for k, v in sim_stats.items():
- print("\t{:<16}: {:>16}".format(k, v))
-
- print("Successful 2D convolution test!")
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Execution statistics:
- inp_load_nbytes : 114688
- wgt_load_nbytes : 1179648
- acc_load_nbytes : 0
- uop_load_nbytes : 1144
- out_store_nbytes: 50176
- gemm_counter : 451584
- alu_counter : 9408
- Successful 2D convolution test!
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 452-459
-
-Summary
--------
-This tutorial demonstrates how TVM scheduling primitives can be used to
-lower 2D convolution onto hardware accelerator intrinsics, making
-use of hardware specific optimizations, such as latency hiding with
-virtual threading.
-
-
-
-.. _sphx_glr_download_topic_vta_tutorials_optimize_convolution_opt.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: convolution_opt.py <convolution_opt.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: convolution_opt.ipynb <convolution_opt.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/optimize/index.rst.txt b/docs/_sources/topic/vta/tutorials/optimize/index.rst.txt
deleted file mode 100644
index 202d8f1c3a..0000000000
--- a/docs/_sources/topic/vta/tutorials/optimize/index.rst.txt
+++ /dev/null
@@ -1,59 +0,0 @@
-
-
-.. _sphx_glr_topic_vta_tutorials_optimize:
-
-Optimize Tensor Operators
--------------------------
-
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnails">
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an overview on how to use TVM to map a 2D convolution workload efficient...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/optimize/images/thumb/sphx_glr_convolution_opt_thumb.png
- :alt: 2D Convolution Optimization
-
- :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">2D Convolution Optimization</div>
- </div>
-
-
-.. raw:: html
-
- <div class="sphx-glr-thumbcontainer" tooltip="This tutorial provides an overview on how to use TVM to map matrix multiplication efficiently o...">
-
-.. only:: html
-
- .. image:: /topic/vta/tutorials/optimize/images/thumb/sphx_glr_matrix_multiply_opt_thumb.png
- :alt: Matrix Multiply Blocking
-
- :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py`
-
-.. raw:: html
-
- <div class="sphx-glr-thumbnail-title">Matrix Multiply Blocking</div>
- </div>
-
-
-.. raw:: html
-
- </div>
-
-
-.. toctree::
- :hidden:
-
- /topic/vta/tutorials/optimize/convolution_opt
- /topic/vta/tutorials/optimize/matrix_multiply_opt
-
diff --git a/docs/_sources/topic/vta/tutorials/optimize/matrix_multiply_opt.rst.txt b/docs/_sources/topic/vta/tutorials/optimize/matrix_multiply_opt.rst.txt
deleted file mode 100644
index 9e662e60fb..0000000000
--- a/docs/_sources/topic/vta/tutorials/optimize/matrix_multiply_opt.rst.txt
+++ /dev/null
@@ -1,667 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/optimize/matrix_multiply_opt.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_optimize_matrix_multiply_opt.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/4d3f955a709b320db0d42740fead8ac1/matrix_multiply_opt.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py:
-
-
-.. _vta-mat-mult-opt:
-
-Matrix Multiply Blocking
-========================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This tutorial provides an overview on how to use TVM to map matrix
-multiplication efficiently on the VTA design.
-We recommend covering the :ref:`basic-mat-mult` tutorial first.
-
-In this tutorial, we will demonstrate TVM schedule optimizations to break large
-neural network operators down onto smaller blocks to achieve computation within
-limited hardware accelerator resources.
-
-.. GENERATED FROM PYTHON SOURCE LINES 34-37
-
-RPC Setup
----------
-We start by programming the Pynq's FPGA and building its RPC runtime.
-
-.. GENERATED FROM PYTHON SOURCE LINES 37-76
-
-.. code-block:: default
-
-
- from __future__ import absolute_import, print_function
-
- import os
- import tvm
- from tvm import te
- import vta
- import numpy as np
- from tvm import rpc
- from tvm.contrib import utils
- from vta.testing import simulator
-
- # Load VTA parameters from the 3rdparty/vta-hw/config/vta_config.json file
- env = vta.get_env()
-
- # We read the Pynq RPC host IP address and port number from the OS environment
- host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
- # We configure both the bitstream and the runtime system on the Pynq
- # to match the VTA configuration specified by the vta_config.json file.
- if env.TARGET == "pynq":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
- # In simulation mode, host the RPC server locally.
- elif env.TARGET in ["sim", "tsim"]:
- remote = rpc.LocalSession()
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 77-98
-
-Computation Declaration
------------------------
-As a first step, we need to describe our matrix multiplication computation.
-We define the matrix multiplication as the computation one would find in a
-fully connected layer, defined by its batch size, input channels, and output
-channels.
-These have to be integer multiples of the VTA tensor shape:
-:code:`BATCH`, :code:`BLOCK_IN`, and :code:`BLOCK_OUT` respectively.
-
-We've added extra operators to the matrix multiplication that apply
-shifting and clipping to the output in order to mimic a fixed-point
-matrix multiplication followed by a rectified linear activation.
-We describe the TVM dataflow graph of the fully connected layer below:
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/fc_dataflow.png
- :align: center
-
-This computation is intentionally too large to fit onto VTA's on-chip
-buffers all at once. Therefore in the scheduling phase we'll
-rely on computation blocking strategies to break the computation down into
-manageable chunks.
-
-.. GENERATED FROM PYTHON SOURCE LINES 98-152
-
-.. code-block:: default
-
-
- # Fully connected layer dimensions: 1024 x 1024
- batch_size = 1
- in_channels = 1024
- out_channels = 1024
- assert batch_size % env.BATCH == 0
- assert in_channels % env.BLOCK_IN == 0
- assert out_channels % env.BLOCK_OUT == 0
-
- # Let's derive the tiled input tensor shapes
- data_shape = (batch_size // env.BATCH, in_channels // env.BLOCK_IN, env.BATCH, env.BLOCK_IN)
- weight_shape = (
- out_channels // env.BLOCK_OUT,
- in_channels // env.BLOCK_IN,
- env.BLOCK_OUT,
- env.BLOCK_IN,
- )
- output_shape = (batch_size // env.BATCH, out_channels // env.BLOCK_OUT, env.BATCH, env.BLOCK_OUT)
- num_ops = in_channels * out_channels * batch_size * 2
-
- # Reduction axes
- ic = te.reduce_axis((0, in_channels // env.BLOCK_IN), name="ic")
- ic_tns = te.reduce_axis((0, env.BLOCK_IN), name="ic_tns")
-
- # Input placeholder tensors
- data = te.placeholder(data_shape, name="data", dtype=env.inp_dtype)
- weight = te.placeholder(weight_shape, name="weight", dtype=env.wgt_dtype)
-
- # Copy buffers
- data_buf = te.compute(data_shape, lambda *i: data(*i), "data_buf")
- weight_buf = te.compute(weight_shape, lambda *i: weight(*i), "weight_buf")
-
- # Declare matrix multiply computation
- res_gemm = te.compute(
- output_shape,
- lambda bo, co, bi, ci: te.sum(
- data_buf[bo, ic, bi, ic_tns].astype(env.acc_dtype)
- * weight_buf[co, ic, ci, ic_tns].astype(env.acc_dtype),
- axis=[ic, ic_tns],
- ),
- name="res_gem",
- )
-
- # Add shift stage for fix-point normalization
- res_shr = te.compute(output_shape, lambda *i: res_gemm(*i) >> env.INP_WIDTH, name="res_shr")
-
- # Apply clipping between (0, input max value)
- inp_max = (1 << (env.INP_WIDTH - 1)) - 1
- res_max = te.compute(output_shape, lambda *i: tvm.te.max(res_shr(*i), 0), "res_max")
- res_min = te.compute(output_shape, lambda *i: tvm.te.min(res_max(*i), inp_max), "res_min")
-
- # Apply typecast to input data type before sending results back
- res = te.compute(output_shape, lambda *i: res_min(*i).astype(env.inp_dtype), name="res")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 153-161
-
-Scheduling the Computation
---------------------------
-We'll look at a set of schedule transformations necessary to map the
-matrix multiplications onto VTA in an efficient fashion.
-Those include:
-
-- Computation blocking
-- Lowering to VTA hardware intrinsics
-
-.. GENERATED FROM PYTHON SOURCE LINES 161-168
-
-.. code-block:: default
-
-
-
- # Create TVM schedule
- s = te.create_schedule(res.op)
- # Let's look at the default TVM schedule
- print(tvm.lower(s, [data, weight, res], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(data: T.Buffer((1, 64, 1, 16), "int8"), weight: T.Buffer((64, 64, 16, 16), "int8"), res: T.Buffer((1, 64, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- data_buf = T.allocate([1024], "int8", "global")
- weight_buf = T.allocate([1048576], "int8", "global")
- res_gem = T.allocate([1024], "int32", "global")
- data_buf_1 = T.Buffer((1024,), "int8", data=data_buf)
- for i1, i3 in T.grid(64, 16):
- cse_var_1: T.int32 = i1 * 16 + i3
- data_1 = T.Buffer((1024,), "int8", data=data.data)
- data_buf_1[cse_var_1] = data_1[cse_var_1]
- weight_buf_1 = T.Buffer((1048576,), "int8", data=weight_buf)
- for i0, i1, i2, i3 in T.grid(64, 64, 16, 16):
- cse_var_2: T.int32 = i0 * 16384 + i1 * 256 + i2 * 16 + i3
- weight_1 = T.Buffer((1048576,), "int8", data=weight.data)
- weight_buf_1[cse_var_2] = weight_1[cse_var_2]
- res_gem_1 = T.Buffer((1024,), "int32", data=res_gem)
- for co, ci in T.grid(64, 16):
- res_gem_1[co * 16 + ci] = 0
- for ic, ic_tns in T.grid(64, 16):
- cse_var_3: T.int32 = co * 16 + ci
- res_gem_1[cse_var_3] = res_gem_1[cse_var_3] + T.Cast("int32", data_buf_1[ic * 16 + ic_tns]) * T.Cast("int32", weight_buf_1[co * 16384 + ic * 256 + ci * 16 + ic_tns])
- res_gem_2 = T.Buffer((1024,), "int32", data=res_gem)
- for i1, i3 in T.grid(64, 16):
- cse_var_4: T.int32 = i1 * 16 + i3
- res_gem_2[cse_var_4] = T.shift_right(res_gem_1[cse_var_4], 8)
- res_gem_3 = T.Buffer((1024,), "int32", data=res_gem)
- for i1, i3 in T.grid(64, 16):
- cse_var_5: T.int32 = i1 * 16 + i3
- res_gem_3[cse_var_5] = T.max(res_gem_2[cse_var_5], 0)
- res_gem_4 = T.Buffer((1024,), "int32", data=res_gem)
- for i1, i3 in T.grid(64, 16):
- cse_var_6: T.int32 = i1 * 16 + i3
- res_gem_4[cse_var_6] = T.min(res_gem_3[cse_var_6], 127)
- for i1, i3 in T.grid(64, 16):
- cse_var_7: T.int32 = i1 * 16 + i3
- res_1 = T.Buffer((1024,), "int8", data=res.data)
- res_1[cse_var_7] = T.Cast("int8", res_gem_4[cse_var_7])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 169-223
-
-Blocking the Computation
-~~~~~~~~~~~~~~~~~~~~~~~~
-The matrix multiplication is by default too large for activations or weights
-to fit on VTA's on-chip buffers all at once.
-We block the (1, 1024) by (1024, 1024) matrix multiplication into
-smaller (1, 256) by (256, 256) matrix multiplications so the intermediate
-tensors can fit on the accelerator's on-chip SRAM.
-This approach is similar to blocking techniques applied to CPUs and GPUs in
-order to increase cache hit rate.
-
-We perform blocking along each axes (the batch axis being untouched since
-we are performing singe-batch inference).
-We also leave the inner-most tensorization axes as-is in order to allow
-TVM to pattern-match tensorization.
-We show the outcome of blocking on the computation schedule in the diagram
-below:
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/blocking.png
- :align: center
- :width: 480px
-
-.. note::
-
- The code after loop splitting and reordering is equivalent to the following
- pseudo-code. We ignore the batch axis since we are only performing single-batch
- inference in this example:
-
- .. code-block:: c
-
- for (int oc_out = 0; oc_out < 4; ++oc_out) {
- // Initialization loop
- for (int oc_inn = 0; oc_inn < 16; ++oc_inn) {
- for (int oc_tns = 0; oc_tns < 16; ++oc_tns) {
- int j = (oc_out * 16 + oc_inn) * 16 + oc_tns;
- C[0][j] = 0;
- }
- }
- for (int ic_out = 0; ic_out < 4; ++ic_out) {
- // Block loop
- for (int oc_inn = 0; oc_inn < 16; ++oc_inn) {
- for (int ic_inn = 0; ic_inn < 16; ++ic_inn) {
- // Tensorization loop
- for (int oc_tns = 0; oc_tns < 16; ++oc_tns) {
- for (int ic_tns = 0; ic_tns < 16; ++ic_tns) {
- int i = (ic_out * 16 + ic_inn) * 16 + ic_tns;
- int j = (oc_out * 16 + oc_inn) * 16 + oc_tns;
- C[0][i] = C[0][i] + A[0][i] * B[j][i];
- }
- }
- }
- }
- }
- }
- }
-
-.. GENERATED FROM PYTHON SOURCE LINES 223-254
-
-.. code-block:: default
-
-
- # Let's define tiling sizes (expressed in multiples of VTA tensor shape size)
- b_block = 1 // env.BATCH
- i_block = 256 // env.BLOCK_IN
- o_block = 256 // env.BLOCK_OUT
-
- # Tile the output tensor along the batch and output channel dimensions
- # (since by default we are doing single batch inference, the split along
- # the batch dimension has no effect)
- b, oc, b_tns, oc_tns = s[res].op.axis
- b_out, b_inn = s[res].split(b, b_block)
- oc_out, oc_inn = s[res].split(oc, o_block)
- s[res].reorder(b_out, oc_out, b_inn, oc_inn)
-
- # Move intermediate computation into each output compute tile
- s[res_gemm].compute_at(s[res], oc_out)
- s[res_shr].compute_at(s[res], oc_out)
- s[res_max].compute_at(s[res], oc_out)
- s[res_min].compute_at(s[res], oc_out)
-
- # Apply additional loop split along reduction axis (input channel)
- b_inn, oc_inn, b_tns, oc_tns = s[res_gemm].op.axis
- ic_out, ic_inn = s[res_gemm].split(ic, i_block)
-
- # Reorder axes. We move the ic_out axis all the way out of the GEMM
- # loop to block along the reduction axis
- s[res_gemm].reorder(ic_out, b_inn, oc_inn, ic_inn, b_tns, oc_tns, ic_tns)
-
- # Let's look at the current TVM schedule after blocking
- print(tvm.lower(s, [data, weight, res], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(data: T.Buffer((1, 64, 1, 16), "int8"), weight: T.Buffer((64, 64, 16, 16), "int8"), res: T.Buffer((1, 64, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- data_buf = T.allocate([1024], "int8", "global")
- weight_buf = T.allocate([1048576], "int8", "global")
- res_gem = T.allocate([256], "int32", "global")
- data_buf_1 = T.Buffer((1024,), "int8", data=data_buf)
- for i1, i3 in T.grid(64, 16):
- cse_var_1: T.int32 = i1 * 16 + i3
- data_1 = T.Buffer((1024,), "int8", data=data.data)
- data_buf_1[cse_var_1] = data_1[cse_var_1]
- weight_buf_1 = T.Buffer((1048576,), "int8", data=weight_buf)
- for i0, i1, i2, i3 in T.grid(64, 64, 16, 16):
- cse_var_2: T.int32 = i0 * 16384 + i1 * 256 + i2 * 16 + i3
- weight_1 = T.Buffer((1048576,), "int8", data=weight.data)
- weight_buf_1[cse_var_2] = weight_1[cse_var_2]
- for i1_outer in range(4):
- res_gem_1 = T.Buffer((256,), "int32", data=res_gem)
- for co_init, ci_init in T.grid(16, 16):
- res_gem_1[co_init * 16 + ci_init] = 0
- for ic_outer, co, ic_inner, ci, ic_tns in T.grid(4, 16, 16, 16, 16):
- cse_var_3: T.int32 = co * 16 + ci
- res_gem_1[cse_var_3] = res_gem_1[cse_var_3] + T.Cast("int32", data_buf_1[ic_outer * 256 + ic_inner * 16 + ic_tns]) * T.Cast("int32", weight_buf_1[i1_outer * 262144 + co * 16384 + ic_outer * 4096 + ic_inner * 256 + ci * 16 + ic_tns])
- res_gem_2 = T.Buffer((256,), "int32", data=res_gem)
- for i1, i3 in T.grid(16, 16):
- cse_var_4: T.int32 = i1 * 16 + i3
- res_gem_2[cse_var_4] = T.shift_right(res_gem_1[cse_var_4], 8)
- res_gem_3 = T.Buffer((256,), "int32", data=res_gem)
- for i1, i3 in T.grid(16, 16):
- cse_var_5: T.int32 = i1 * 16 + i3
- res_gem_3[cse_var_5] = T.max(res_gem_2[cse_var_5], 0)
- res_gem_4 = T.Buffer((256,), "int32", data=res_gem)
- for i1, i3 in T.grid(16, 16):
- cse_var_6: T.int32 = i1 * 16 + i3
- res_gem_4[cse_var_6] = T.min(res_gem_3[cse_var_6], 127)
- for i1_inner, i3 in T.grid(16, 16):
- cse_var_7: T.int32 = i1_inner * 16
- res_1 = T.Buffer((1024,), "int8", data=res.data)
- res_1[i1_outer * 256 + cse_var_7 + i3] = T.Cast("int8", res_gem_4[cse_var_7 + i3])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 255-262
-
-Lowering Copies to DMA Transfers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Next we set the buffer scopes to the corresponding on-chip VTA SRAM buffers.
-We move the load loops into the matrix multiply computation loop to stage
-memory loads such that they fit in the on-chip SRAM buffers.
-Finally we annotate the load/store loop outer axes with the DMA copy pragma
-to perform bulk memory transfers on VTA.
-
-.. GENERATED FROM PYTHON SOURCE LINES 262-284
-
-.. code-block:: default
-
-
- # Set scope of SRAM buffers
- s[data_buf].set_scope(env.inp_scope)
- s[weight_buf].set_scope(env.wgt_scope)
- s[res_gemm].set_scope(env.acc_scope)
- s[res_shr].set_scope(env.acc_scope)
- s[res_min].set_scope(env.acc_scope)
- s[res_max].set_scope(env.acc_scope)
-
- # Block data and weight cache reads
- s[data_buf].compute_at(s[res_gemm], ic_out)
- s[weight_buf].compute_at(s[res_gemm], ic_out)
-
- # Use DMA copy pragma on DRAM->SRAM operations
- s[data_buf].pragma(s[data_buf].op.axis[0], env.dma_copy)
- s[weight_buf].pragma(s[weight_buf].op.axis[0], env.dma_copy)
-
- # Use DMA copy pragma on SRAM->DRAM operation
- # (this implies that these copies should be performed along b_inn,
- # or result axis 2)
- s[res].pragma(s[res].op.axis[2], env.dma_copy)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 285-290
-
-Lowering Computation to VTA Compute Intrinsics
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The last phase is to lower the computation loops down to VTA hardware
-intrinsics by mapping the matrix multiplication to tensor intrinsics,
-and mapping the shift, and clipping computation to the vector ALU.
-
-.. GENERATED FROM PYTHON SOURCE LINES 290-304
-
-.. code-block:: default
-
-
- # Apply tensorization over the batch tensor tile axis
- s[res_gemm].tensorize(b_tns, env.gemm)
-
- # Add an ALU pragma over the shift and clipping operations
- s[res_shr].pragma(s[res_shr].op.axis[0], env.alu)
- s[res_min].pragma(s[res_min].op.axis[0], env.alu)
- s[res_max].pragma(s[res_max].op.axis[0], env.alu)
-
- # Let's look at the final lowered TVM schedule after lowering memory
- # loads/stores down to DMA copy intrinsics, and the computation down to
- # VTA compute intrinsics.
- print(vta.lower(s, [data, weight, res], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(data: T.Buffer((1, 64, 1, 16), "int8"), weight: T.Buffer((64, 64, 16, 16), "int8"), res: T.Buffer((1, 64, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- T.tir.vta.coproc_dep_push(3, 2)
- for i1_outer in range(4):
- vta = T.int32()
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2):
- T.tir.vta.coproc_dep_pop(3, 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushGEMMOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 0, 0)
- T.tir.vta.uop_push(0, 1, 0, 0, 0, 0, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 1)
- for ic_outer in range(4):
- cse_var_1: T.int32 = ic_outer * 16
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 1):
- T.tir.vta.coproc_dep_pop(2, 1)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), data.data, cse_var_1, 16, 1, 16, 0, 0, 0, 0, 0, 2)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), weight.data, i1_outer * 1024 + cse_var_1, 16, 16, 64, 0, 0, 0, 0, 0, 1)
- T.tir.vta.coproc_dep_push(1, 2)
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2)
- T.tir.vta.coproc_dep_pop(1, 2)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushGEMMOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 0, 16)
- T.call_extern("int32", "VTAUopLoopBegin", 16, 0, 1, 1)
- T.tir.vta.uop_push(0, 0, 0, 0, 0, 0, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 1)
- T.tir.vta.coproc_dep_pop(2, 1)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2):
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, 0, 0, 0, 3, 1, 8)
- T.call_extern("int32", "VTAUopLoopEnd")
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, 0, 0, 0, 1, 1, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 16, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, 0, 0, 0, 0, 1, 127)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 3)
- T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 3)
- T.tir.vta.coproc_dep_pop(2, 3)
- for i1_inner in range(16):
- T.call_extern("int32", "VTAStoreBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), i1_inner, 4, res.data, i1_outer * 16 + i1_inner, 1, 1, 1)
- T.tir.vta.coproc_dep_push(3, 2)
- T.tir.vta.coproc_sync()
- T.tir.vta.coproc_dep_pop(3, 2)
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 305-311
-
-TVM Compilation and Verification
---------------------------------
-After specifying the schedule, we can compile it into a TVM function.
-We save the module so we can send it over RPC.
-We run the function and verify it against a numpy implementation to
-ensure correctness.
-
-.. GENERATED FROM PYTHON SOURCE LINES 311-367
-
-.. code-block:: default
-
-
- # Compile the TVM module
- my_gemm = vta.build(
- s, [data, weight, res], tvm.target.Target("ext_dev", host=env.target_host), name="my_gemm"
- )
- temp = utils.tempdir()
- my_gemm.save(temp.relpath("gemm.o"))
- remote.upload(temp.relpath("gemm.o"))
- f = remote.load_module("gemm.o")
-
- # Get the remote device context
- ctx = remote.ext_dev(0)
-
- # Initialize the data and weight arrays randomly in the int range of (-128, 128]
- data_np = np.random.randint(-128, 128, size=(batch_size, in_channels)).astype(data.dtype)
- weight_np = np.random.randint(-128, 128, size=(out_channels, in_channels)).astype(weight.dtype)
-
- # Apply packing to the data and weight arrays from a 2D to a 4D packed layout
- data_packed = data_np.reshape(
- batch_size // env.BATCH, env.BATCH, in_channels // env.BLOCK_IN, env.BLOCK_IN
- ).transpose((0, 2, 1, 3))
- weight_packed = weight_np.reshape(
- out_channels // env.BLOCK_OUT, env.BLOCK_OUT, in_channels // env.BLOCK_IN, env.BLOCK_IN
- ).transpose((0, 2, 1, 3))
-
- # Format the input/output arrays with tvm.nd.array to the DLPack standard
- data_nd = tvm.nd.array(data_packed, ctx)
- weight_nd = tvm.nd.array(weight_packed, ctx)
- res_nd = tvm.nd.array(np.zeros(output_shape).astype(res.dtype), ctx)
-
- # Clear stats
- if env.TARGET in ["sim", "tsim"]:
- simulator.clear_stats()
-
- # Invoke the module to perform the computation
- f(data_nd, weight_nd, res_nd)
-
- # Verify against numpy implementation
- res_ref = np.dot(data_np.astype(env.acc_dtype), weight_np.T.astype(env.acc_dtype))
- res_ref = res_ref >> env.INP_WIDTH
- res_ref = np.clip(res_ref, 0, inp_max)
- res_ref = res_ref.astype(res.dtype)
- res_ref = res_ref.reshape(
- batch_size // env.BATCH, env.BATCH, out_channels // env.BLOCK_OUT, env.BLOCK_OUT
- ).transpose((0, 2, 1, 3))
- np.testing.assert_equal(res_ref, res_nd.numpy())
-
- # Print stats
- if env.TARGET in ["sim", "tsim"]:
- sim_stats = simulator.stats()
- print("Execution statistics:")
- for k, v in sim_stats.items():
- print("\t{:<16}: {:>16}".format(k, v))
-
- print("Successful blocked matrix multiply test!")
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Execution statistics:
- inp_load_nbytes : 4096
- wgt_load_nbytes : 1048576
- acc_load_nbytes : 0
- uop_load_nbytes : 20
- out_store_nbytes: 1024
- gemm_counter : 4096
- alu_counter : 192
- Successful blocked matrix multiply test!
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 368-375
-
-Summary
--------
-This tutorial demonstrates how TVM scheduling primitives can achieve
-computation blocking for a matrix multiplication example.
-This allows us to map arbitrarily large computation onto limited
-hardware accelerator resources.
-
-
-
-.. _sphx_glr_download_topic_vta_tutorials_optimize_matrix_multiply_opt.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: matrix_multiply_opt.py <matrix_multiply_opt.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: matrix_multiply_opt.ipynb <matrix_multiply_opt.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
deleted file mode 100644
index b3fc8b0a27..0000000000
--- a/docs/_sources/topic/vta/tutorials/optimize/sg_execution_times.rst.txt
+++ /dev/null
@@ -1,14 +0,0 @@
-
-:orphan:
-
-.. _sphx_glr_topic_vta_tutorials_optimize_sg_execution_times:
-
-Computation times
-=================
-**00:03.333** total execution time for **topic_vta_tutorials_optimize** files:
-
-+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_optimize_convolution_opt.py` (``convolution_opt.py``) | 00:02.823 | 0.0 MB |
-+--------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_optimize_matrix_multiply_opt.py` (``matrix_multiply_opt.py``) | 00:00.510 | 0.0 MB |
-+--------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt b/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
deleted file mode 100644
index a1bd2ed019..0000000000
--- a/docs/_sources/topic/vta/tutorials/sg_execution_times.rst.txt
+++ /dev/null
@@ -1,14 +0,0 @@
-
-:orphan:
-
-.. _sphx_glr_topic_vta_tutorials_sg_execution_times:
-
-Computation times
-=================
-**00:00.873** total execution time for **topic_vta_tutorials** files:
-
-+---------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_matrix_multiply.py` (``matrix_multiply.py``) | 00:00.470 | 0.0 MB |
-+---------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_topic_vta_tutorials_vta_get_started.py` (``vta_get_started.py``) | 00:00.404 | 0.0 MB |
-+---------------------------------------------------------------------------------+-----------+--------+
diff --git a/docs/_sources/topic/vta/tutorials/vta_get_started.rst.txt b/docs/_sources/topic/vta/tutorials/vta_get_started.rst.txt
deleted file mode 100644
index 55cbadb262..0000000000
--- a/docs/_sources/topic/vta/tutorials/vta_get_started.rst.txt
+++ /dev/null
@@ -1,717 +0,0 @@
-
-.. DO NOT EDIT. THIS FILE WAS AUTOMATICALLY GENERATED BY
-.. TVM'S MONKEY-PATCHED VERSION OF SPHINX-GALLERY. TO MAKE
-.. CHANGES, EDIT THE SOURCE PYTHON FILE:
-.. "topic/vta/tutorials/vta_get_started.py"
-
-.. only:: html
-
- .. note::
- :class: sphx-glr-download-link-note
-
- This tutorial can be used interactively with Google Colab! You can also click
- :ref:`here <sphx_glr_download_topic_vta_tutorials_vta_get_started.py>` to run the Jupyter notebook locally.
-
- .. image:: https://raw.githubusercontent.com/tlc-pack/web-data/main/images/utilities/colab_button.svg
- :align: center
- :target: https://colab.research.google.com/github/apache/tvm-site/blob/asf-site/docs/_downloads/83b9961c758069912464db3443fffc06/vta_get_started.ipynb
- :width: 300px
-
-.. rst-class:: sphx-glr-example-title
-
-.. _sphx_glr_topic_vta_tutorials_vta_get_started.py:
-
-
-.. _vta-get-started:
-
-Get Started with VTA
-====================
-**Author**: `Thierry Moreau <https://homes.cs.washington.edu/~moreau/>`_
-
-This is an introduction tutorial on how to use TVM to program the VTA design.
-
-In this tutorial, we will demonstrate the basic TVM workflow to implement
-a vector addition on the VTA design's vector ALU.
-This process includes specific scheduling transformations necessary to lower
-computation down to low-level accelerator operations.
-
-To begin, we need to import TVM which is our deep learning optimizing compiler.
-We also need to import the VTA python package which contains VTA specific
-extensions for TVM to target the VTA design.
-
-.. GENERATED FROM PYTHON SOURCE LINES 35-43
-
-.. code-block:: default
-
- from __future__ import absolute_import, print_function
-
- import os
- import tvm
- from tvm import te
- import vta
- import numpy as np
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 44-59
-
-Loading in VTA Parameters
-~~~~~~~~~~~~~~~~~~~~~~~~~
-VTA is a modular and customizable design. Consequently, the user
-is free to modify high-level hardware parameters that affect
-the hardware design layout.
-These parameters are specified in the :code:`vta_config.json` file by their
-:code:`log2` values.
-These VTA parameters can be loaded with the :code:`vta.get_env`
-function.
-
-Finally, the TVM target is also specified in the :code:`vta_config.json` file.
-When set to *sim*, execution will take place inside of a behavioral
-VTA simulator.
-If you want to run this tutorial on the Pynq FPGA development platform,
-follow the *VTA Pynq-Based Testing Setup* guide.
-
-.. GENERATED FROM PYTHON SOURCE LINES 59-62
-
-.. code-block:: default
-
-
- env = vta.get_env()
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 63-67
-
-FPGA Programming
-----------------
-When targeting the Pynq FPGA development board, we need to configure
-the board with a VTA bitstream.
-
-.. GENERATED FROM PYTHON SOURCE LINES 67-101
-
-.. code-block:: default
-
-
- # We'll need the TVM RPC module and the VTA simulator module
- from tvm import rpc
- from tvm.contrib import utils
- from vta.testing import simulator
-
- # We read the Pynq RPC host IP address and port number from the OS environment
- host = os.environ.get("VTA_RPC_HOST", "192.168.2.99")
- port = int(os.environ.get("VTA_RPC_PORT", "9091"))
-
- # We configure both the bitstream and the runtime system on the Pynq
- # to match the VTA configuration specified by the vta_config.json file.
- if env.TARGET == "pynq" or env.TARGET == "de10nano":
-
- # Make sure that TVM was compiled with RPC=1
- assert tvm.runtime.enabled("rpc")
- remote = rpc.connect(host, port)
-
- # Reconfigure the JIT runtime
- vta.reconfig_runtime(remote)
-
- # Program the FPGA with a pre-compiled VTA bitstream.
- # You can program the FPGA with your own custom bitstream
- # by passing the path to the bitstream file instead of None.
- vta.program_fpga(remote, bitstream=None)
-
- # In simulation mode, host the RPC server locally.
- elif env.TARGET in ("sim", "tsim", "intelfocl"):
- remote = rpc.LocalSession()
-
- if env.TARGET in ["intelfocl"]:
- # program intelfocl aocx
- vta.program_fpga(remote, bitstream="vta.bitstream")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 102-124
-
-Computation Declaration
------------------------
-As a first step, we need to describe our computation.
-TVM adopts tensor semantics, with each intermediate result
-represented as multi-dimensional array. The user needs to describe
-the computation rule that generates the output tensors.
-
-In this example we describe a vector addition, which requires multiple
-computation stages, as shown in the dataflow diagram below.
-First we describe the input tensors :code:`A` and :code:`B` that are living
-in main memory.
-Second, we need to declare intermediate tensors :code:`A_buf` and
-:code:`B_buf`, which will live in VTA's on-chip buffers.
-Having this extra computational stage allows us to explicitly
-stage cached reads and writes.
-Third, we describe the vector addition computation which will
-add :code:`A_buf` to :code:`B_buf` to produce :code:`C_buf`.
-The last operation is a cast and copy back to DRAM, into results tensor
-:code:`C`.
-
-.. image:: https://raw.githubusercontent.com/uwsampl/web-data/main/vta/tutorial/vadd_dataflow.png
- :align: center
-
-.. GENERATED FROM PYTHON SOURCE LINES 126-139
-
-Input Placeholders
-~~~~~~~~~~~~~~~~~~
-We describe the placeholder tensors :code:`A`, and :code:`B` in a tiled data
-format to match the data layout requirements imposed by the VTA vector ALU.
-
-For VTA's general purpose operations such as vector adds, the tile size is
-:code:`(env.BATCH, env.BLOCK_OUT)`.
-The dimensions are specified in
-the :code:`vta_config.json` configuration file and are set by default to
-a (1, 16) vector.
-
-In addition, A and B's data types also needs to match the :code:`env.acc_dtype`
-which is set by the :code:`vta_config.json` file to be a 32-bit integer.
-
-.. GENERATED FROM PYTHON SOURCE LINES 139-149
-
-.. code-block:: default
-
-
- # Output channel factor m - total 64 x 16 = 1024 output channels
- m = 64
- # Batch factor o - total 1 x 1 = 1
- o = 1
- # A placeholder tensor in tiled data format
- A = te.placeholder((o, m, env.BATCH, env.BLOCK_OUT), name="A", dtype=env.acc_dtype)
- # B placeholder tensor in tiled data format
- B = te.placeholder((o, m, env.BATCH, env.BLOCK_OUT), name="B", dtype=env.acc_dtype)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 150-164
-
-Copy Buffers
-~~~~~~~~~~~~
-One specificity of hardware accelerators, is that on-chip memory has to be
-explicitly managed.
-This means that we'll need to describe intermediate tensors :code:`A_buf`
-and :code:`B_buf` that can have a different memory scope than the original
-placeholder tensors :code:`A` and :code:`B`.
-
-Later in the scheduling phase, we can tell the compiler that :code:`A_buf`
-and :code:`B_buf` will live in the VTA's on-chip buffers (SRAM), while
-:code:`A` and :code:`B` will live in main memory (DRAM).
-We describe A_buf and B_buf as the result of a compute
-operation that is the identity function.
-This can later be interpreted by the compiler as a cached read operation.
-
-.. GENERATED FROM PYTHON SOURCE LINES 164-170
-
-.. code-block:: default
-
-
- # A copy buffer
- A_buf = te.compute((o, m, env.BATCH, env.BLOCK_OUT), lambda *i: A(*i), "A_buf")
- # B copy buffer
- B_buf = te.compute((o, m, env.BATCH, env.BLOCK_OUT), lambda *i: B(*i), "B_buf")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 171-180
-
-Vector Addition
-~~~~~~~~~~~~~~~
-Now we're ready to describe the vector addition result tensor :code:`C`,
-with another compute operation.
-The compute function takes the shape of the tensor, as well as a lambda
-function that describes the computation rule for each position of the tensor.
-
-No computation happens during this phase, as we are only declaring how
-the computation should be done.
-
-.. GENERATED FROM PYTHON SOURCE LINES 180-188
-
-.. code-block:: default
-
-
- # Describe the in-VTA vector addition
- C_buf = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT),
- lambda *i: A_buf(*i).astype(env.acc_dtype) + B_buf(*i).astype(env.acc_dtype),
- name="C_buf",
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 189-193
-
-Casting the Results
-~~~~~~~~~~~~~~~~~~~
-After the computation is done, we'll need to send the results computed by VTA
-back to main memory.
-
-.. GENERATED FROM PYTHON SOURCE LINES 195-206
-
-.. note::
-
- **Memory Store Restrictions**
-
- One specificity of VTA is that it only supports DRAM stores in the narrow
- :code:`env.inp_dtype` data type format.
- This lets us reduce the data footprint for memory transfers (more on this
- in the basic matrix multiply example).
-
-We perform one last typecast operation to the narrow
-input activation data format.
-
-.. GENERATED FROM PYTHON SOURCE LINES 206-212
-
-.. code-block:: default
-
-
- # Cast to output type, and send to main memory
- C = te.compute(
- (o, m, env.BATCH, env.BLOCK_OUT), lambda *i: C_buf(*i).astype(env.inp_dtype), name="C"
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 213-214
-
-This concludes the computation declaration part of this tutorial.
-
-.. GENERATED FROM PYTHON SOURCE LINES 217-230
-
-Scheduling the Computation
---------------------------
-While the above lines describes the computation rule, we can obtain
-:code:`C` in many ways.
-TVM asks the user to provide an implementation of the computation called
-*schedule*.
-
-A schedule is a set of transformations to an original computation that
-transforms the implementation of the computation without affecting
-correctness.
-This simple VTA programming tutorial aims to demonstrate basic schedule
-transformations that will map the original schedule down to VTA hardware
-primitives.
-
-.. GENERATED FROM PYTHON SOURCE LINES 233-237
-
-Default Schedule
-~~~~~~~~~~~~~~~~
-After we construct the schedule, by default the schedule computes
-:code:`C` in the following way:
-
-.. GENERATED FROM PYTHON SOURCE LINES 237-243
-
-.. code-block:: default
-
-
- # Let's take a look at the generated schedule
- s = te.create_schedule(C.op)
-
- print(tvm.lower(s, [A, B, C], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(A: T.Buffer((1, 64, 1, 16), "int32"), B: T.Buffer((1, 64, 1, 16), "int32"), C: T.Buffer((1, 64, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- A_buf = T.allocate([1024], "int32", "global")
- B_buf = T.allocate([1024], "int32", "global")
- A_buf_1 = T.Buffer((1024,), "int32", data=A_buf)
- for i1, i3 in T.grid(64, 16):
- cse_var_1: T.int32 = i1 * 16 + i3
- A_1 = T.Buffer((1024,), "int32", data=A.data)
- A_buf_1[cse_var_1] = A_1[cse_var_1]
- B_buf_1 = T.Buffer((1024,), "int32", data=B_buf)
- for i1, i3 in T.grid(64, 16):
- cse_var_2: T.int32 = i1 * 16 + i3
- B_1 = T.Buffer((1024,), "int32", data=B.data)
- B_buf_1[cse_var_2] = B_1[cse_var_2]
- A_buf_2 = T.Buffer((1024,), "int32", data=A_buf)
- for i1, i3 in T.grid(64, 16):
- cse_var_3: T.int32 = i1 * 16 + i3
- A_buf_2[cse_var_3] = A_buf_1[cse_var_3] + B_buf_1[cse_var_3]
- for i1, i3 in T.grid(64, 16):
- cse_var_4: T.int32 = i1 * 16 + i3
- C_1 = T.Buffer((1024,), "int8", data=C.data)
- C_1[cse_var_4] = T.Cast("int8", A_buf_2[cse_var_4])
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 244-253
-
-Although this schedule makes sense, it won't compile to VTA.
-In order to obtain correct code generation, we need to apply scheduling
-primitives and code annotation that will transform the schedule into
-one that can be directly lowered onto VTA hardware intrinsics.
-Those include:
-
- - DMA copy operations which will take globally-scoped tensors and copy
- those into locally-scoped tensors.
- - Vector ALU operations that will perform the vector add.
-
-.. GENERATED FROM PYTHON SOURCE LINES 255-264
-
-Buffer Scopes
-~~~~~~~~~~~~~
-First, we set the scope of the copy buffers to indicate to TVM that these
-intermediate tensors will be stored in the VTA's on-chip SRAM buffers.
-Below, we tell TVM that :code:`A_buf`, :code:`B_buf`, :code:`C_buf`
-will live in VTA's on-chip *accumulator buffer* which serves as
-VTA's general purpose register file.
-
-Set the intermediate tensors' scope to VTA's on-chip accumulator buffer
-
-.. GENERATED FROM PYTHON SOURCE LINES 264-268
-
-.. code-block:: default
-
- s[A_buf].set_scope(env.acc_scope)
- s[B_buf].set_scope(env.acc_scope)
- s[C_buf].set_scope(env.acc_scope)
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
-
- stage(C_buf, compute(C_buf, body=[A_buf[i0, i1, i2, i3] + B_buf[i0, i1, i2, i3]], axis=[T.iter_var(i0, T.Range(0, 1), "DataPar", ""), T.iter_var(i1, T.Range(0, 64), "DataPar", ""), T.iter_var(i2, T.Range(0, 1), "DataPar", ""), T.iter_var(i3, T.Range(0, 16), "DataPar", "")], reduce_axis=[], tag=, attrs={}))
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 269-276
-
-DMA Transfers
-~~~~~~~~~~~~~
-We need to schedule DMA transfers to move data living in DRAM to
-and from the VTA on-chip buffers.
-We insert :code:`dma_copy` pragmas to indicate to the compiler
-that the copy operations will be performed in bulk via DMA,
-which is common in hardware accelerators.
-
-.. GENERATED FROM PYTHON SOURCE LINES 276-283
-
-.. code-block:: default
-
-
- # Tag the buffer copies with the DMA pragma to map a copy loop to a
- # DMA transfer operation
- s[A_buf].pragma(s[A_buf].op.axis[0], env.dma_copy)
- s[B_buf].pragma(s[B_buf].op.axis[0], env.dma_copy)
- s[C].pragma(s[C].op.axis[0], env.dma_copy)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 284-291
-
-ALU Operations
-~~~~~~~~~~~~~~
-VTA has a vector ALU that can perform vector operations on tensors
-in the accumulator buffer.
-In order to tell TVM that a given operation needs to be mapped to the
-VTA's vector ALU, we need to explicitly tag the vector addition loop
-with an :code:`env.alu` pragma.
-
-.. GENERATED FROM PYTHON SOURCE LINES 291-299
-
-.. code-block:: default
-
-
- # Tell TVM that the computation needs to be performed
- # on VTA's vector ALU
- s[C_buf].pragma(C_buf.op.axis[0], env.alu)
-
- # Let's take a look at the finalized schedule
- print(vta.lower(s, [A, B, C], simple_mode=True))
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- # from tvm.script import ir as I
- # from tvm.script import tir as T
-
- @I.ir_module
- class Module:
- @T.prim_func
- def main(A: T.Buffer((1, 64, 1, 16), "int32"), B: T.Buffer((1, 64, 1, 16), "int32"), C: T.Buffer((1, 64, 1, 16), "int8")):
- T.func_attr({"from_legacy_te_schedule": T.bool(True), "tir.noalias": T.bool(True)})
- vta = T.int32()
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 2):
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), A.data, 0, 64, 1, 64, 0, 0, 0, 0, 0, 3)
- T.call_extern("int32", "VTALoadBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), B.data, 0, 64, 1, 64, 0, 0, 0, 0, 64, 3)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_uop_scope", "VTAPushALUOp"):
- T.call_extern("int32", "VTAUopLoopBegin", 64, 1, 1, 0)
- T.tir.vta.uop_push(1, 0, 0, 64, 0, 2, 0, 0)
- T.call_extern("int32", "VTAUopLoopEnd")
- T.tir.vta.coproc_dep_push(2, 3)
- with T.attr(T.iter_var(vta, None, "ThreadIndex", "vta"), "coproc_scope", 3):
- T.tir.vta.coproc_dep_pop(2, 3)
- T.call_extern("int32", "VTAStoreBuffer2D", T.tvm_thread_context(T.tir.vta.command_handle()), 0, 4, C.data, 0, 64, 1, 64)
- T.tir.vta.coproc_sync()
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 300-301
-
-This concludes the scheduling portion of this tutorial.
-
-.. GENERATED FROM PYTHON SOURCE LINES 303-314
-
-TVM Compilation
----------------
-After we have finished specifying the schedule, we can compile it
-into a TVM function. By default TVM compiles into a type-erased
-function that can be directly called from python side.
-
-In the following line, we use :code:`tvm.build` to create a function.
-The build function takes the schedule, the desired signature of the
-function(including the inputs and outputs) as well as target language
-we want to compile to.
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 314-318
-
-.. code-block:: default
-
- my_vadd = vta.build(
- s, [A, B, C], tvm.target.Target("ext_dev", host=env.target_host), name="my_vadd"
- )
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 319-327
-
-Saving the Module
-~~~~~~~~~~~~~~~~~
-TVM lets us save our module into a file so it can loaded back later. This
-is called ahead-of-time compilation and allows us to save some compilation
-time.
-More importantly, this allows us to cross-compile the executable on our
-development machine and send it over to the Pynq FPGA board over RPC for
-execution.
-
-.. GENERATED FROM PYTHON SOURCE LINES 327-335
-
-.. code-block:: default
-
-
- # Write the compiled module into an object file.
- temp = utils.tempdir()
- my_vadd.save(temp.relpath("vadd.o"))
-
- # Send the executable over RPC
- remote.upload(temp.relpath("vadd.o"))
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 336-339
-
-Loading the Module
-~~~~~~~~~~~~~~~~~~
-We can load the compiled module from the file system to run the code.
-
-.. GENERATED FROM PYTHON SOURCE LINES 339-342
-
-.. code-block:: default
-
-
- f = remote.load_module("vadd.o")
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 343-357
-
-Running the Function
---------------------
-The compiled TVM function uses a concise C API and can be invoked from
-any language.
-
-TVM provides an array API in python to aid quick testing and prototyping.
-The array API is based on `DLPack <https://github.com/dmlc/dlpack>`_ standard.
-
-- We first create a remote context (for remote execution on the Pynq).
-- Then :code:`tvm.nd.array` formats the data accordingly.
-- :code:`f()` runs the actual computation.
-- :code:`numpy()` copies the result array back in a format that can be
- interpreted.
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 357-377
-
-.. code-block:: default
-
-
- # Get the remote device context
- ctx = remote.ext_dev(0)
-
- # Initialize the A and B arrays randomly in the int range of (-128, 128]
- A_orig = np.random.randint(-128, 128, size=(o * env.BATCH, m * env.BLOCK_OUT)).astype(A.dtype)
- B_orig = np.random.randint(-128, 128, size=(o * env.BATCH, m * env.BLOCK_OUT)).astype(B.dtype)
-
- # Apply packing to the A and B arrays from a 2D to a 4D packed layout
- A_packed = A_orig.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
- B_packed = B_orig.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
-
- # Format the input/output arrays with tvm.nd.array to the DLPack standard
- A_nd = tvm.nd.array(A_packed, ctx)
- B_nd = tvm.nd.array(B_packed, ctx)
- C_nd = tvm.nd.array(np.zeros((o, m, env.BATCH, env.BLOCK_OUT)).astype(C.dtype), ctx)
-
- # Invoke the module to perform the computation
- f(A_nd, B_nd, C_nd)
-
-
-
-
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 378-382
-
-Verifying Correctness
----------------------
-Compute the reference result with numpy and assert that the output of the
-matrix multiplication indeed is correct
-
-.. GENERATED FROM PYTHON SOURCE LINES 382-389
-
-.. code-block:: default
-
-
- # Compute reference result with numpy
- C_ref = (A_orig.astype(env.acc_dtype) + B_orig.astype(env.acc_dtype)).astype(C.dtype)
- C_ref = C_ref.reshape(o, env.BATCH, m, env.BLOCK_OUT).transpose((0, 2, 1, 3))
- np.testing.assert_equal(C_ref, C_nd.numpy())
- print("Successful vector add test!")
-
-
-
-
-
-.. rst-class:: sphx-glr-script-out
-
- .. code-block:: none
-
- Successful vector add test!
-
-
-
-
-.. GENERATED FROM PYTHON SOURCE LINES 390-406
-
-Summary
--------
-This tutorial provides a walk-through of TVM for programming the
-deep learning accelerator VTA with a simple vector addition example.
-The general workflow includes:
-
-- Programming the FPGA with the VTA bitstream over RPC.
-- Describing the vector add computation via a series of computations.
-- Describing how we want to perform the computation using schedule primitives.
-- Compiling the function to the VTA target.
-- Running the compiled module and verifying it against a numpy implementation.
-
-You are more than welcome to check other examples out and tutorials
-to learn more about the supported operations, schedule primitives
-and other features supported by TVM to program VTA.
-
-
-
-.. _sphx_glr_download_topic_vta_tutorials_vta_get_started.py:
-
-.. only:: html
-
- .. container:: sphx-glr-footer sphx-glr-footer-example
-
-
- .. container:: sphx-glr-download sphx-glr-download-python
-
- :download:`Download Python source code: vta_get_started.py <vta_get_started.py>`
-
- .. container:: sphx-glr-download sphx-glr-download-jupyter
-
- :download:`Download Jupyter notebook: vta_get_started.ipynb <vta_get_started.ipynb>`
-
-
-.. only:: html
-
- .. rst-class:: sphx-glr-signature
-
- `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
diff --git a/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt b/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
index 15dd777f8b..a1f77dcfb6 100644
--- a/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
+++ b/docs/_sources/tutorial/auto_scheduler_matmul_x86.rst.txt
@@ -318,7 +318,7 @@ We build the binary and check its correctness and performance.
.. code-block:: none
- Execution time of this operator: 95.234 ms
+ Execution time of this operator: 97.337 ms
@@ -434,7 +434,7 @@ operations.
.. rst-class:: sphx-glr-timing
- **Total running time of the script:** ( 1 minutes 28.683 seconds)
+ **Total running time of the script:** ( 1 minutes 30.383 seconds)
.. _sphx_glr_download_tutorial_auto_scheduler_matmul_x86.py:
diff --git a/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt b/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
index 5dfb4658d9..45c302519d 100644
--- a/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
+++ b/docs/_sources/tutorial/autotvm_matmul_x86.rst.txt
@@ -454,16 +454,16 @@ reduce variance, we take 5 measurements and average them.
waiting for device...
device available
Get devices for measurement successfully!
- No: 1 GFLOPS: 0.51/0.51 result: MeasureResult(costs=(0.5295417018,), error_no=MeasureErrorNo.NO_ERROR, all_cost=9.740242958068848, timestamp=1732199187.5855463) [('tile_y', [-1, 512]), ('tile_x', [-1, 1])],None,9
- No: 2 GFLOPS: 11.91/11.91 result: MeasureResult(costs=(0.0225395686,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.674910306930542, timestamp=1732199188.213378) [('tile_y', [-1, 64]), ('tile_x', [-1, 512])],None,96
- No: 3 GFLOPS: 2.99/11.91 result: MeasureResult(costs=(0.089797509,), error_no=MeasureErrorNo.NO_ERROR, all_cost=2.724971055984497, timestamp=1732199189.9177766) [('tile_y', [-1, 16]), ('tile_x', [-1, 2])],None,14
- No: 4 GFLOPS: 5.53/11.91 result: MeasureResult(costs=(0.0485790536,), error_no=MeasureErrorNo.NO_ERROR, all_cost=2.0303900241851807, timestamp=1732199190.956986) [('tile_y', [-1, 1]), ('tile_x', [-1, 4])],None,20
- No: 5 GFLOPS: 0.53/11.91 result: MeasureResult(costs=(0.5104986406,), error_no=MeasureErrorNo.NO_ERROR, all_cost=8.42898178100586, timestamp=1732199199.508022) [('tile_y', [-1, 256]), ('tile_x', [-1, 1])],None,8
- No: 6 GFLOPS: 0.93/11.91 result: MeasureResult(costs=(0.2891266914,), error_no=MeasureErrorNo.NO_ERROR, all_cost=4.888197422027588, timestamp=1732199204.3905077) [('tile_y', [-1, 512]), ('tile_x', [-1, 2])],None,19
- No: 7 GFLOPS: 9.87/11.91 result: MeasureResult(costs=(0.027204142400000002,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.7021477222442627, timestamp=1732199205.0879743) [('tile_y', [-1, 512]), ('tile_x', [-1, 32])],None,59
- No: 8 GFLOPS: 11.40/11.91 result: MeasureResult(costs=(0.0235398042,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6468513011932373, timestamp=1732199205.7355833) [('tile_y', [-1, 32]), ('tile_x', [-1, 512])],None,95
- No: 9 GFLOPS: 10.69/11.91 result: MeasureResult(costs=(0.025117900999999998,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6531922817230225, timestamp=1732199206.498607) [('tile_y', [-1, 4]), ('tile_x', [-1, 16])],None,42
- No: 10 GFLOPS: 3.39/11.91 result: MeasureResult(costs=(0.07918817980000001,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.5061841011047363, timestamp=1732199208.031469) [('tile_y', [-1, 512]), ('tile_x', [-1, 8])],None,39
+ No: 1 GFLOPS: 7.99/7.99 result: MeasureResult(costs=(0.0336051478,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.897508144378662, timestamp=1732370660.2403324) [('tile_y', [-1, 4]), ('tile_x', [-1, 8])],None,32
+ No: 2 GFLOPS: 2.05/7.99 result: MeasureResult(costs=(0.1308783936,), error_no=MeasureErrorNo.NO_ERROR, all_cost=3.498889446258545, timestamp=1732370662.6677737) [('tile_y', [-1, 32]), ('tile_x', [-1, 4])],None,25
+ No: 3 GFLOPS: 1.01/7.99 result: MeasureResult(costs=(0.2649759658,), error_no=MeasureErrorNo.NO_ERROR, all_cost=5.5517308712005615, timestamp=1732370667.1681492) [('tile_y', [-1, 64]), ('tile_x', [-1, 2])],None,16
+ No: 4 GFLOPS: 0.45/7.99 result: MeasureResult(costs=(0.59678075,), error_no=MeasureErrorNo.NO_ERROR, all_cost=10.854597330093384, timestamp=1732370676.9842567) [('tile_y', [-1, 512]), ('tile_x', [-1, 1])],None,9
+ No: 5 GFLOPS: 2.02/7.99 result: MeasureResult(costs=(0.1325672298,), error_no=MeasureErrorNo.NO_ERROR, all_cost=2.4208431243896484, timestamp=1732370679.5451212) [('tile_y', [-1, 128]), ('tile_x', [-1, 4])],None,27
+ No: 6 GFLOPS: 13.59/13.59 result: MeasureResult(costs=(0.0197560742,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.6387603282928467, timestamp=1732370680.120738) [('tile_y', [-1, 128]), ('tile_x', [-1, 64])],None,67
+ No: 7 GFLOPS: 7.31/13.59 result: MeasureResult(costs=(0.036739382599999995,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.8844423294067383, timestamp=1732370680.97085) [('tile_y', [-1, 2]), ('tile_x', [-1, 16])],None,41
+ No: 8 GFLOPS: 5.42/13.59 result: MeasureResult(costs=(0.049526222,), error_no=MeasureErrorNo.NO_ERROR, all_cost=1.06333327293396, timestamp=1732370682.0270138) [('tile_y', [-1, 1]), ('tile_x', [-1, 4])],None,20
+ No: 9 GFLOPS: 13.93/13.93 result: MeasureResult(costs=(0.0192743316,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.5494282245635986, timestamp=1732370682.6874218) [('tile_y', [-1, 16]), ('tile_x', [-1, 32])],None,54
+ No: 10 GFLOPS: 8.70/13.93 result: MeasureResult(costs=(0.0308465528,), error_no=MeasureErrorNo.NO_ERROR, all_cost=0.737847089767456, timestamp=1732370683.4514303) [('tile_y', [-1, 512]), ('tile_x', [-1, 256])],None,89
diff --git a/docs/_sources/tutorial/autotvm_relay_x86.rst.txt b/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
index 0fee6f943f..447d164905 100644
--- a/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
+++ b/docs/_sources/tutorial/autotvm_relay_x86.rst.txt
@@ -311,7 +311,7 @@ standard deviation.
.. code-block:: none
- {'mean': 470.6090286300059, 'median': 470.6100289999995, 'std': 1.5287288939173203}
+ {'mean': 494.78452854999995, 'median': 495.0281438499985, 'std': 2.4185807804328143}
@@ -582,29 +582,31 @@ the tuning data to.
.. code-block:: none
- [Task 1/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 1/25] Current/Best: 1.97/ 14.65 GFLOPS | Progress: (4/20) | 16.09 s [Task 1/25] Current/Best: 8.87/ 20.65 GFLOPS | Progress: (8/20) | 18.29 s [Task 1/25] Current/Best: 7.33/ 24.48 GFLOPS | Progress: (12/20) | 21.50 s [Task 1/25] Current/Best: 7.20/ 24.48 GFLOPS | Progress: (16/20) | 24.34 s [Task 1/25] Current/Best: 17.06/ 24.48 GFLOPS | Progress: (20/20) | 30.95 s Done.
- [Task 2/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 2/25] Current/Best: 12.64/ 19.07 GFLOPS | Progress: (4/20) | 5.00 s [Task 2/25] Current/Best: 17.92/ 19.07 GFLOPS | Progress: (8/20) | 6.65 s [Task 2/25] Current/Best: 6.12/ 19.07 GFLOPS | Progress: (12/20) | 8.27 s [Task 2/25] Current/Best: 6.50/ 19.64 GFLOPS | Progress: (16/20) | 9.67 s [Task 2/25] Current/Best: 17.37/ 22.76 GFLOPS | Progress: (20/20) | 11.27 s Done.
- [Task 3/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 3/25] Current/Best: 20.28/ 20.28 GFLOPS | Progress: (4/20) | 4.65 s [Task 3/25] Current/Best: 19.35/ 20.28 GFLOPS | Progress: (8/20) | 7.05 s [Task 3/25] Current/Best: 18.05/ 24.10 GFLOPS | Progress: (12/20) | 9.05 s [Task 3/25] Current/Best: 9.64/ 24.10 GFLOPS | Progress: (16/20) | 12.52 s [Task 3/25] Current/Best: 18.24/ 24.10 GFLOPS | Progress: (20/20) | 14.85 s Done.
- [Task 4/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 4/25] Current/Best: 11.90/ 18.16 GFLOPS | Progress: (4/20) | 4.79 s [Task 4/25] Current/Best: 10.87/ 18.16 GFLOPS | Progress: (8/20) | 7.18 s [Task 4/25] Current/Best: 15.78/ 18.16 GFLOPS | Progress: (12/20) | 10.48 s [Task 4/25] Current/Best: 20.47/ 20.47 GFLOPS | Progress: (16/20) | 12.30 s [Task 4/25] Current/Best: 5.35/ 20.47 GFLOPS | Progress: (20/20) | 14.76 s Done.
- [Task 5/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 5/25] Current/Best: 3.21/ 15.60 GFLOPS | Progress: (4/20) | 4.78 s [Task 5/25] Current/Best: 16.04/ 16.23 GFLOPS | Progress: (8/20) | 7.17 s [Task 5/25] Current/Best: 12.01/ 20.08 GFLOPS | Progress: (12/20) | 9.34 s [Task 5/25] Current/Best: 8.84/ 20.08 GFLOPS | Progress: (16/20) | 11.92 s [Task 5/25] Current/Best: 4.12/ 20.08 GFLOPS | Progress: (20/20) | 13.74 s Done.
- [Task 6/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 6/25] Current/Best: 9.06/ 16.23 GFLOPS | Progress: (4/20) | 5.09 s [Task 6/25] Current/Best: 5.06/ 16.23 GFLOPS | Progress: (8/20) | 7.93 s [Task 6/25] Current/Best: 15.19/ 22.70 GFLOPS | Progress: (12/20) | 10.14 s [Task 6/25] Current/Best: 10.00/ 22.70 GFLOPS | Progress: (16/20) | 13.21 s [Task 6/25] Current/Best: 9.31/ 22.70 GFLOPS | Progress: (20/20) | 16.03 s Done.
- [Task 7/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 7/25] Current/Best: 16.81/ 20.92 GFLOPS | Progress: (4/20) | 4.62 s [Task 7/25] Current/Best: 14.57/ 20.92 GFLOPS | Progress: (8/20) | 7.53 s [Task 7/25] Current/Best: 22.29/ 22.29 GFLOPS | Progress: (12/20) | 10.87 s [Task 7/25] Current/Best: 14.79/ 22.29 GFLOPS | Progress: (16/20) | 13.34 s [Task 7/25] Current/Best: 9.69/ 23.70 GFLOPS | Progress: (20/20) | 17.37 s Done.
- [Task 8/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 8/25] Current/Best: 14.54/ 14.54 GFLOPS | Progress: (4/20) | 5.89 s [Task 8/25] Current/Best: 12.06/ 16.37 GFLOPS | Progress: (8/20) | 8.32 s [Task 8/25] Current/Best: 7.76/ 16.37 GFLOPS | Progress: (12/20) | 19.96 s [Task 8/25] Current/Best: 14.95/ 16.73 GFLOPS | Progress: (16/20) | 23.72 s [Task 8/25] Current/Best: 2.94/ 16.73 GFLOPS | Progress: (20/20) | 29.86 s [Task 9/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 9/25] Current/Best: 11.69/ 18.35 GFLOPS | Progress: (4/20) | 7.75 s [Task 9/25] Current/Best: 16.84/ 23.14 GFLOPS | Progress: (8/20) | 9.62 s [Task 9/25] Current/Best: 6.92/ 23.14 GFLOPS | Progress: (12/20) | 20.96 s [Task 9/25] Current/Best: 16.46/ 23.14 GFLOPS | Progress: (16/20) | 24.49 s [Task 9/25] Current/Best: 18.77/ 23.14 GFLOPS | Progress: (20/20
) | 26.40 s [Task 10/25] Current/Best: 0.00/ 0.00 GFLOPS | Progress: (0/20) | 0.00 s [Task 10/25] Current/Best: 3.07/ 18.41 GFLOPS | Progress: (4/20) | 4.45 s [Task 10/25] Current/Best: 18.54/ 21.50 GFLOPS | Progress: (8/20) | 6.14 s [Task 10/25] Current/Best: 14.64/ 21.50 GFLOPS | Progress: (12/20) | 8.18 s [Task 10/25] Current/Best: 22.64/ 22.64 GFLOPS | Progress: (16/20) | 10.26 s ... 22416 lines suppressed ...