You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tvm.apache.org by huajsj via TVM Discuss <no...@discuss.tvm.ai> on 2020/05/21 20:39:03 UTC
[TVM Discuss] [Development] VTA First Conv Layer Optimize
Hi There,
VTA first conv layer is running on CPU and not get offload into FPGA, in most case that is a performance bottle neck and need optimization, following are some idea about the
optimization, please kindly comments.
Regards
Hua
1. training network to make first conv layer support int8 input and weight, add feature
into vta to support using 16*16 MAC to compute 3 input channel compute.
2. When running on arm-cpu, seems like only one cpu get used for first conv compute,
we may can do parallel to running first conv in multiple cpu for accelerate.
---
[Visit Topic](https://discuss.tvm.ai/t/vta-first-conv-layer-optimize/6766/1) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/a560279b6579e8d93ca0a5906b926ec0a2ee716fa8c681d4aae77f7e876e51a1).
[TVM Discuss] [Development] VTA First Conv Layer Optimize
Posted by Augusto Capone via TVM Discuss <no...@discuss.tvm.ai>.
Hi @hjiang,
I use Sony's framework [NNabla](https://github.com/sony/nnabla) to train the networks but I then convert them to ONNX or Tensorflow in order to use them with TVM. Accuracy loss gets around 4%.
Regards
Augusto
---
[Visit Topic](https://discuss.tvm.ai/t/vta-first-conv-layer-optimize/6766/4) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/927ea146bff0669acaa83b548c7323cad731c3c52f6c5032c9896acad969dd7c).
[TVM Discuss] [Development] VTA First Conv Layer Optimize
Posted by huajsj via TVM Discuss <no...@discuss.tvm.ai>.
Hi @acapone13,
Thanks for following up this post and nice to know you are interested in VTA performance optimization related topic, about the resenet18 pretrained model, could I know which framework you use to generate
the model? and how much the accuracy lost is after the quantization?
Regards
Hua
---
[Visit Topic](https://discuss.tvm.ai/t/vta-first-conv-layer-optimize/6766/3) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/69f52d2871fcc606c346b4469e8536e5741520d368a7d9dea3f2062248c76562).
[TVM Discuss] [Development] VTA First Conv Layer Optimize
Posted by Augusto Capone via TVM Discuss <no...@discuss.tvm.ai>.
Hi @hjiang,
I'm working to deploy a pre-quantized ResNet network with VTA in which the first conv layer supports int8 input/weights. I think it would be an interesting feature even though most quantization works avoid quantizing the first layer. Both ideas are valid but it would be interesting to add this feature into VTA. Please let me know how can I help you to work on this.
Regards,
Augusto
---
[Visit Topic](https://discuss.tvm.ai/t/vta-first-conv-layer-optimize/6766/2) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/2632cb968646cc8eef749b49d3434fce5f4f165ebd67e7db8b39ddecbd2557bf).
[TVM Discuss] [Development] VTA First Conv Layer Optimize
Posted by Augusto Capone via TVM Discuss <no...@discuss.tvm.ai>.
HI @hjiang,
Sorry for the late response, I've had some other work to do. Thanks for the proposed solutions, I'll try this implementations with my model and I'll keep you updated.
Regards
Augusto
---
[Visit Topic](https://discuss.tvm.ai/t/vta-first-conv-layer-optimize/6766/6) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/e96edd783330fbb6d0b66e48ee4895771289c652c189e36bb0d4d4d8a5dd1cc2).
[TVM Discuss] [Development] VTA First Conv Layer Optimize
Posted by huajsj via TVM Discuss <no...@discuss.tvm.ai>.
Hi @acapone13,
to apply first conv2d layer into VTA, there are 2 solution/ steps, first is to padding first conv2d from 3 channel into VTA hardware match channel for example 16, after that we
would can run first quantized conv2d layer on VTA , for sure
the padding would increase compute OP number and impact the performance but that would can provide a baseline for next level perf optimization.
second solution is that for some non 1x1 kernel for example 3x3 kernel, provide special optimization, these optimization is that instead of doing traditional IMG2COL blocking, we can use every 3x3x3(27) data as the input data and do related padding, these would reduce the compute increase and can improve performance.
for the first #solution proposal that need to padding the input data layer from 3 to 16*n to match vta
hardware resource, for the padding part it would look like this PR https://github.com/apache/incubator-tvm/pull/4887, _const_shape_match is similar logic but it only do that for factor_out. if you have interest you can try some patch based on the said logic.
please kindly let me know if you have any better idea or any questions about the possible solutions.
Regards
Hua
---
[Visit Topic](https://discuss.tvm.ai/t/vta-first-conv-layer-optimize/6766/5) to respond.
You are receiving this because you enabled mailing list mode.
To unsubscribe from these emails, [click here](https://discuss.tvm.ai/email/unsubscribe/f1f5e09c82b4ec4c7d2a3428f00bcc597218ecbecb2d0d5d6313e61375e5b5f2).