[GitHub] kice commented on issue #13715: Add C++ Predictor class for inference

[GitBox <gi...@apache.org>]

kice commented on issue #13715: Add C++ Predictor class for inference
URL: https://github.com/apache/incubator-mxnet/pull/13715#issuecomment-455729792
 
 
   > Hey, @kice , would you mind sharing a bit more about your use case(s) for C++ API? We (one of the Apache MXNet development teams) are trying to decide where to take it next as its current implementation is a bit messy. We hope your reply could share pointers to what is that needed for you that C++ doesn't currently do or does inefficiently. It seems you are trying to create a higher level abstraction for inference. I wonder if it would make sense from your use case perspective to implement even higher abstractions like ObjectDetector API or ImageClassifier API. Would you mind sharing a bit more what you are trying to use it for and how?
   
   @ddavydenko To summary at the begining, I think MXNet should focus more on inference and easy deployment. If possible, please fix INT8 quantization and make a MXNet Lite.
   
   I think my use cases are a little bit minority; I use MXNet to process video, more specific single image super-resolution. Before MXNet, I used Caffe1 a lot (see https://github.com/lltcggie/waifu2x-caffe); but since Caffe1 is very old and inefficient, I switched to MXNet (https://github.com/kice/vs_mxnet). In this case, it is very sensitive to runtime speed; and in fact, there are some of my friends also use MXNet just for perforamce boost. That's also the reason why I tried to use INT8 quantization (and MXNet dont have proper support for it). I also tried TVM, but there are more hazards that MXNet and I just gave up.
   
   That's for end users; I as a developer, it is good to see some higer level warppers but I perfer the ablity to use my models. But it will be impossible without easy deployment on Windows (and maybe other platforms). If the end users cannot install and run the program powered by MXNet, then the use cases for MXNet is very limited. For higher abstractions, if I need to feed the program a lot of data and parameters, like which json symbol, which params files, making the C API more easy to use is much better.
   
   Speaking of custom models, I perfer to use tf or pytorch to train my models, and port the trained models to MXNet; since more people use TF and pytorch and more resources on them. But compare to them, MXNet is by far the best and the fastest way to be used in Windows.
   
   For now, I am working on a project that can upsample old games that dont have HD resource to HD in real time. Base on my calculation, a RTX 2070 can upscale anything to 1080p by useing Tensor Cores or a GTX 1080 with INT8 inference. With TensorRT, that will make it even better. However, these things look like wont happend on MXNet anytime soon.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services