[GitHub] [tvm] csullivan opened a new pull request #7690: [Texture support][Part 5] Graph runtime and memory planning support for 2d allocations

[GitBox <gi...@apache.org>]

csullivan opened a new pull request #7690:
URL: https://github.com/apache/tvm/pull/7690


   This PR utilizes the core StorageToken allocation framework provided by the graph memory planner, but extends it to additionally support optimization of two dimensional allocations. The storage_ids and storage_scopes for each Expr output are then provided to the graph runtime, from which a minimal set of one and two dimensional flat buffer and texture memory storage pools can be allocated, respectively. 
   
   **GraphMemoryPlanner**
   - The StorageAllocator Request, Alloc, and CheckRelease, as a few helpers have been refactored into a TokenAllocator, which can dispatch to a one or two dimensional token allocator based on the token storage scope. 
   - TokenAllocator1D's implementation is identical to what exists prior to this PR
   - TokenAllocator2D utilizes a similar bin packing strategy but also tracks the excess space that increasing a storage pool would incur. The strategy is as follows,
     - For a given token allocation request search among the idle texture pools, keeping track of which pool would require the least amount of additional memory. If a requested allocation fits within an existing pool, no additional memory would be required. If not, expand the pool which results in the least amount of added space.
     - For any set of free storage pools which do not require expansion for the given allocation request, choose the pool which would minimize the amount of wasted memory.
     -  Note also that during optimization requests are only queried against candidate pools of the same type as this is a requirement of texture memory alignment.
   
   **Graph runtime**
   - The graph runtime PoolEntry previously tracked a single dimension for allocation (size). This PR replaces that with a std::vector<int64_t> shape. For global scope, this shape is only one dimensional. For texture scope this shape is three dimensional (width, height, channel).
   - The graph runtime uses the storage ids and scope information to grow pool entries along one or two axes for storage pools of global or texture scope, respectively. 
   - For 1D pools nothing has changed, the size of the allocation is determined from the maximum number of bytes required for all entries in a storage pool, and the resulting pool is of kDLFloat type and byte aligned. 
   - For 2D texture pools the size of an image pool is determined from the maximum number of row and column elements required for all entries of a storage pool, and retains the type of the underlying pool entries.
   
   RFC in progress, once posted I will add a link here.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org