[GitHub] [incubator-mxnet] QueensGambit edited a comment on issue #16173: Saving and loading cudNN autotune and graph optimization

[GitBox <gi...@apache.org>]

QueensGambit edited a comment on issue #16173: Saving and loading cudNN autotune and graph optimization
URL: https://github.com/apache/incubator-mxnet/issues/16173#issuecomment-537932809
 
 
   Thank you for the feedback @KellenSunderland.
   I think the mini-batchsize should be included as a caching specification as well because optimization techniques like TensorRT depend on it.
   
   A different approach would be to define a save and load function for the [Executor class](https://github.com/apache/incubator-mxnet/blob/992c3c0dd90c0723de6934e826a49bad6569eeac/include/mxnet/executor.h#L53).
   The memory file of an executor handle would contain all additional platform specific definitions and optimization results. This would allow the user to run the full binding process once on a specific platform and later the option to bind it much quicker:
   
   ```python
   # mxnet/executor.py
   def save(filename_exec, type):
   """Saves the executor handle including specific optimization of the graph.
       Must be run after the executor handle was binded: `model.bind()`.
   
   Parameters
   ----------
   filename : str
       Path to the executor file (e.g. "executor.exec").
   References
   ----------
   `Saving and Loading of Executor handles \
   `_
   """
   ```
   
   In order to preferably avoid an additional copy of the model parameters, one needs to specifiy the `.params` and `.symbol` filepath when loading the executor handle:
   
   ```python
   # mxnet/executor.py
   def load(filename_exec, filename_symbol, filename_params):
   """Loads and binds the executor handle.
   
   Parameters
   ----------
   filename_exec : str
       Path to the executor file (e.g. "executor.exec").
   filename_symbol : str
       Path to the model architecture definition (e.g. "model.symbol").
   filename_params : str
       Path to the model weights (e.g. "model.params").
   References
   ----------
   `Saving and Loading of Executor handles \
   `_
   """
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services