You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/02/08 18:50:15 UTC
[GitHub] [tvm] manupa-arm opened a new pull request #10193: [USMP] adding support for U2 and U3 usecases
manupa-arm opened a new pull request #10193:
URL: https://github.com/apache/tvm/pull/10193
This commit adds a MemoryPools argument for
the compilation flow according to RFC0029.
Moreover, it is used to provide support for
external pools from the application layer
that could be pinned for different memories
and/or be reused between multiple inferences
of a model.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm edited a comment on pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
manupa-arm edited a comment on pull request #10193:
URL: https://github.com/apache/tvm/pull/10193#issuecomment-1035137418
@Mousius I think I've addressed the comments. PTAL when you have some time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm edited a comment on pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
manupa-arm edited a comment on pull request #10193:
URL: https://github.com/apache/tvm/pull/10193#issuecomment-1035137418
@Mousius I think I've addressed the comments (Thanks! for catching that!). PTAL when you have some time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
Mousius commented on pull request #10193:
URL: https://github.com/apache/tvm/pull/10193#issuecomment-1038879556
Thanks @manupa-arm :smile_cat:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10193:
URL: https://github.com/apache/tvm/pull/10193#issuecomment-1034674593
cc : @Mousius @areusch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius merged pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
Mousius merged pull request #10193:
URL: https://github.com/apache/tvm/pull/10193
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] Mousius commented on a change in pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
Mousius commented on a change in pull request #10193:
URL: https://github.com/apache/tvm/pull/10193#discussion_r803489810
##########
File path: include/tvm/ir/memory_pools.h
##########
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/relay/executor.h
Review comment:
```suggestion
* \file tvm/ir/memory_pools.h
```
##########
File path: python/tvm/ir/memory_pools.py
##########
@@ -0,0 +1,132 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""Objects for Memory Pools to be used within the compilation"""
+
+from typing import Optional, List
+
+from tvm._ffi import register_object
+from tvm.runtime import Object
+from . import _ffi_api
+
+
+@register_object("ir.PoolInfo")
+class PoolInfo(Object):
+ """PoolInfo object holds information related to memory pools
+ where the statically sized allocate nodes will pooled into.
+
+ Parameters
+ ----------
+ pool_name : str
+ The name of the memory pool
+
+ target_access : Dict[Target, str]
+ A dictionary where keys describe which targets could
+ access the pool where value could take the values :
+ a) "rw" : read-write access
+ b) "ro" : write-only acesss
+
+ size_hint_bytes : Optional[int]
+ The expected size hint to be used by the allocator.
+ The default value would be -1 which means the pool
+ is not size restricted.
+
+ clock_frequency_hz : Optional[int]
+ The clock frequency that the memory pool runs at in Hz.
+ If not specified/known, this will default to -1 indicating
+ it hasn't been defined.
+
+ read_bandwidth_bytes_per_cycle : Optional[int]
+ The read bandwidth of the memory pool in bytes/cycle.
+ If not specified/known, this will default to -1 indicating
+ it hasn't been defined.
+
+ write_bandwidth_bytes_per_cycle : Optional[int]
+ The write bandwidth of the memory pool in bytes/cycle.
+ If not specified/known, this will default to -1 indicating
+ it hasn't been defined.
+
+ read_latency_cycles : Optional[int]
+ The read latency of the memory pool in cycles.
+ If not specified/known, this will default to 0.
+
+ write_latency_cycles : Optional[int]
+ The write latency of the memory pool in cycles.
+ If not specified/known, this will default to 0.
+
+ target_burst_bytes : Optional[Union[Dict[Target, int], None]]
+ The burst length of the memory pool in bytes per target.
+ If not specified/known for a given target, a burst length
+ of 1 byte will be assumed.
+
+ """
+
+ # The string parameter to indicate read and write access to a pool
+ # This needs to be kept in sync with kTargetPoolReadWriteAccess in
+ # include/tvm/tir/usmp/utils.h
+ READ_WRITE_ACCESS = "rw"
+ # The string parameter to indicate read only access to a pool
+ # This needs to be kept in sync with kTargetPoolReadOnlyAccess in
+ # include/tvm/tir/usmp/utils.h
+ READ_ONLY_ACCESS = "ro"
Review comment:
```suggestion
# The string parameter to indicate read and write access to a pool
# This needs to be kept in sync with kTargetPoolReadWriteAccess in
# include/tvm/ir/memory_pools.h
READ_WRITE_ACCESS = "rw"
# The string parameter to indicate read only access to a pool
# This needs to be kept in sync with kTargetPoolReadOnlyAccess in
# include/tvm/ir/memory_pools.h
READ_ONLY_ACCESS = "ro"
```
##########
File path: include/tvm/ir/memory_pools.h
##########
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+/*!
+ * \file tvm/relay/executor.h
+ * \brief The object definition for relay.build argument type of memory pools
+ */
+#ifndef TVM_IR_MEMORY_POOLS_H_
+#define TVM_IR_MEMORY_POOLS_H_
+
+#include <tvm/runtime/registry.h>
+#include <tvm/target/target.h>
+
+namespace tvm {
+
+/*!
+ * \brief Describes a pool of memory accessible by one or more targets.
+ */
+struct PoolInfoNode : public Object {
+ /*! \brief The name of the memory pool */
+ String pool_name;
+ /*! \brief The expected size hint to be used by the allocator.
+ * The size_hint_bytes is set to kUnrestrictedPoolSizeHint
+ * to indicate the pool is not size restricted.
+ */
+ Integer size_hint_bytes;
+ /*! \brief The accessibility from each Target */
+ Map<Target, String> target_access; // 'rw' or 'ro'
+ /*! \brief The clock frequency of the memory in Hz */
+ Integer clock_frequency_hz;
+ /*! \brief The read bandwidth in bytes/cycle */
+ Integer read_bandwidth_bytes_per_cycle;
+ /*! \brief The write bandwidth in bytes/cycle */
+ Integer write_bandwidth_bytes_per_cycle;
+ /*! \brief The read latency in cycles */
+ Integer read_latency_cycles;
+ /*! \brief The write latency in cycles */
+ Integer write_latency_cycles;
+ /*! \brief The burst length in bytes for each Target */
+ Map<Target, Integer> target_burst_bytes;
+ /*! \brief Whether pool is internally generated.
+ * The internal pools will be generated as part of
+ * the entry point code generation of the executor
+ */
+ bool is_internal = false;
+
+ void VisitAttrs(tvm::AttrVisitor* v) {
+ v->Visit("pool_name", &pool_name);
+ v->Visit("size_hint_bytes", &size_hint_bytes);
+ v->Visit("target_access", &target_access);
+ v->Visit("clock_frequency_hz", &clock_frequency_hz);
+ v->Visit("read_bandwidth_bytes_per_cycle", &read_bandwidth_bytes_per_cycle);
+ v->Visit("write_bandwidth_bytes_per_cycle", &write_bandwidth_bytes_per_cycle);
+ v->Visit("read_latency_cycles", &read_latency_cycles);
+ v->Visit("write_latency_cycles", &write_latency_cycles);
+ v->Visit("target_burst_bytes", &target_burst_bytes);
+ v->Visit("is_internal", &is_internal);
+ }
+
+ bool SEqualReduce(const PoolInfoNode* other, SEqualReducer equal) const {
+ return equal(pool_name, other->pool_name) && equal(size_hint_bytes, other->size_hint_bytes) &&
+ equal(target_access, other->target_access) &&
+ equal(target_access, other->target_access) &&
+ equal(clock_frequency_hz, other->clock_frequency_hz) &&
+ equal(read_bandwidth_bytes_per_cycle, other->read_bandwidth_bytes_per_cycle) &&
+ equal(write_bandwidth_bytes_per_cycle, other->write_bandwidth_bytes_per_cycle) &&
+ equal(read_latency_cycles, other->read_latency_cycles) &&
+ equal(write_latency_cycles, other->write_latency_cycles) &&
+ equal(target_burst_bytes, other->target_burst_bytes) &&
+ equal(is_internal, other->is_internal);
+ }
+
+ void SHashReduce(SHashReducer hash_reduce) const {
+ hash_reduce(pool_name);
+ hash_reduce(size_hint_bytes);
+ hash_reduce(target_access);
+ hash_reduce(clock_frequency_hz);
+ hash_reduce(read_bandwidth_bytes_per_cycle);
+ hash_reduce(write_bandwidth_bytes_per_cycle);
+ hash_reduce(read_latency_cycles);
+ hash_reduce(write_latency_cycles);
+ hash_reduce(target_burst_bytes);
+ hash_reduce(is_internal);
+ }
+
+ static constexpr const char* _type_key = "ir.PoolInfo";
+ TVM_DECLARE_FINAL_OBJECT_INFO(PoolInfoNode, Object);
+};
+
+class PoolInfo : public ObjectRef {
+ public:
+ /*!
+ * \brief The string parameter to indicate read and write access to a pool
+ * This needs to be kept in sync with PoolInfo.READ_WRITE_ACCESS in
+ * python/tvm/tir/usmp/utils.py
+ */
+ static constexpr const char* kTargetPoolReadWriteAccess = "rw";
+ /*!
+ * \brief The string parameter to indicate read only access to a pool
+ * This needs to be kept in sync with PoolInfo.READ_ONLY_ACCESS in
+ * python/tvm/tir/usmp/utils.py
+ */
+ static constexpr const char* kTargetPoolReadOnlyAccess = "ro";
Review comment:
```suggestion
/*!
* \brief The string parameter to indicate read and write access to a pool
* This needs to be kept in sync with PoolInfo.READ_WRITE_ACCESS in
* python/tvm/ir/memory_pools.py
*/
static constexpr const char* kTargetPoolReadWriteAccess = "rw";
/*!
* \brief The string parameter to indicate read only access to a pool
* This needs to be kept in sync with PoolInfo.READ_ONLY_ACCESS in
* python/tvm/ir/memory_pools.py
*/
static constexpr const char* kTargetPoolReadOnlyAccess = "ro";
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] manupa-arm commented on pull request #10193: [USMP] adding support for U2 and U3 usecases
Posted by GitBox <gi...@apache.org>.
manupa-arm commented on pull request #10193:
URL: https://github.com/apache/tvm/pull/10193#issuecomment-1035137418
@Mousius I think I've addressed the changes. PTAL when you have some time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org