[GitHub] [tvm] apeskov commented on pull request #11513: [BYOC][DNNL] Improve performance of DNNL BYOC dense operator

[GitBox <gi...@apache.org>]

apeskov commented on PR #11513:
URL: https://github.com/apache/tvm/pull/11513#issuecomment-1150836941

   One important comment about performance. Just to point out.
   
   In this patch you are using mechanic of auto detection proper layout inside of dnnl_json_runtime. It works correctly and dense primitive will use optimal layout. But it will execute weight reordering each inference call. This reordering significantly break performance (still better than previously, but less than possible).
   
   To avoid weight reordering it should be done once during `Init`. For that you need change dense weight pattern from `wildcard` to `is_constant`.
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org