Re: [PR] [RFC] Add NNEF frontend [tvm-rfcs]

["agoston-mc (via GitHub)" <gi...@apache.org>]

agoston-mc commented on PR #108:
URL: https://github.com/apache/tvm-rfcs/pull/108#issuecomment-2100973263

   We have updated the PR with Relax frontend, but we have also kept the Relay, as an option, thinking it could be useful to have both, because we noticed performance differences during testing.
   
   We observed that Relax by the default build pipeline is significantly slower than Relay. (On CPU we observed 2 orders of magnitude slower runs, while on GPU 3-5x slower, the models we tested were mobilenet and resnet variants, all static models) We observed the same with the ONNX Relax frontend, so we suspect the issue is with the compilation, not with the frontends. Is this a normal situation with the current state of development of Relax? 
   By using Meta Schedule with a custom pipeline (with only a ValidateOps transformation), we managed to match or surpass the speed of Relay, but in many cases using the 'zero' or 'default_build' pipelines did not improve the performance.
   What is the recommended workflow to be able to reach the performance of Relay reliably (at least on static models)? 
   
   Anyways this does not affect the frontend code in the PR so we could move on with that as the frontend is ready to be submitted to TVM. We are just curious for debug/measurement reasons, as we were surprised by the results.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org