You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "Aristide021 (via GitHub)" <gi...@apache.org> on 2025/08/09 20:47:58 UTC
[PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Aristide021 opened a new pull request, #18201:
URL: https://github.com/apache/tvm/pull/18201
Intel GNA (Gaussian Neural Accelerator) hardware is present in Intel Core Ultra processors. This backend enables TVM users to target Intel NPU hardware via OpenVINO's GNA integration.
Features:
- Pattern-based graph partitioning for GNA-compatible operations
- JSON serialization for OpenVINO runtime integration
- Support for dense/linear, 1D convolution, and ReLU operations
- Automatic shape and dtype extraction for optimization
- Comprehensive test coverage
Supported operations:
- Dense/Linear layers (relax.matmul)
- 1D Convolution (relax.nn.conv1d)
- ReLU activation (relax.nn.relu)
This provides community access to Intel NPU acceleration through TVM's compilation pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "mshr-h (via GitHub)" <gi...@apache.org>.
mshr-h commented on PR #18201:
URL: https://github.com/apache/tvm/pull/18201#issuecomment-3210509503
@Aristide021
Thanks for the PR! A couple of points and questions:
1. Status of GNA vs NPU
- The upstream GNA repo is archived and marked as not under active management. The OpenVINO docs also note that GNA is being discontinued and recommend using Intel's NPU as the low-power offload path on newer processors. Given that, would it make sense to position this backend as a stepping stone toward NPU (and/or clarify the long-term maintenance plan in the README/code comments)?
- https://github.com/intel/gna
- https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_GNA.html
2. CI & Software Emulation Mode
- According to the OpenVINO docs, GNA plugin supports Software Emulation Mode (CPU fallback) when GNA HW isn't present. If we enable that in tests, we could run E2E coverage in our CI.
I also think this backend can serve as a very good example for codegen in Relax. It shows a clean and minimal pattern: partitioning with basic ops, handing off to JSON, and keeping the implementation relatively lightweight. Adding a short HOWTO or developer note ("Writing a minimal Relax backend") that references this code could be very helpful for the community.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "Aristide021 (via GitHub)" <gi...@apache.org>.
Aristide021 commented on PR #18201:
URL: https://github.com/apache/tvm/pull/18201#issuecomment-4296425179
Superseded by #18247 and #19425 (merged).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "Aristide021 (via GitHub)" <gi...@apache.org>.
Aristide021 commented on PR #18201:
URL: https://github.com/apache/tvm/pull/18201#issuecomment-3218252605
> Thanks for the contribution, given GNA is archived, it perhaps does not make sense to maintain it in the main tree, adding ci will also add extra overhead here. However, i agree that having generic tutorials for BYOC NPU would be useful, if we can have something that support a current NPU that would be great
I'd be happy to refactor this into a generic NPU tutorial targeting Intel's current NPU plugin. Should this live in the tutorials section or as a contrib module? I can adapt the JSON architecture for educational purposes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "Aristide021 (via GitHub)" <gi...@apache.org>.
Aristide021 closed pull request #18201: [Backend][Relax] Add Intel GNA backend for NPU support
URL: https://github.com/apache/tvm/pull/18201
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: discuss-archive-unsubscribe@tvm.apache.org
For additional commands, e-mail: discuss-archive-help@tvm.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #18201:
URL: https://github.com/apache/tvm/pull/18201#issuecomment-3218332804
i think starting as contrib is fine, and we can have a tutorial explaination point to the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "Aristide021 (via GitHub)" <gi...@apache.org>.
Aristide021 commented on PR #18201:
URL: https://github.com/apache/tvm/pull/18201#issuecomment-3215308112
> @Aristide021 Thanks for the PR! A couple of points and questions:
>
> 1. Status of GNA vs NPU
>
> * The upstream GNA repo is archived and marked as not under active management. The OpenVINO docs also note that GNA is being discontinued and recommend using Intel's NPU as the low-power offload path on newer processors. Given that, would it make sense to position this backend as a stepping stone toward NPU (and/or clarify the long-term maintenance plan in the README/code comments)?
> * https://github.com/intel/gna
> * https://docs.openvino.ai/2023.3/openvino_docs_OV_UG_supported_plugins_GNA.html
> 2. CI & Software Emulation Mode
>
> * According to the OpenVINO docs, GNA plugin supports Software Emulation Mode (CPU fallback) when GNA HW isn't present. If we enable that in tests, we could run E2E coverage in our CI.
>
> I also think this backend can serve as a very good example for codegen in Relax. It shows a clean and minimal pattern: partitioning with basic ops, handing off to JSON, and keeping the implementation relatively lightweight. Adding a short HOWTO or developer note ("Writing a minimal Relax backend") that references this code could be very helpful for the community.
>
> cc @tqchen @Hzfengsy @cbalint13
Thanks for the review and the excellent points! You're correct about GNA being archived. I designed this backend as a stepping stone toward NPU support with OpenVINO runtime integration in mind. The JSON serialization approach should make the transition to Intel's current NPU path relatively straightforward.
For the CI integration with Software Emulation Mode, I think that's a great suggestion. I can add CPU fallback support to enable E2E testing without requiring actual GNA hardware.
I'd also be happy to add documentation, positioning this as a foundation for NPU backends, and include a developer guide if that would be helpful for the community.
I'll go ahead and update the PR description to clarify the NPU migration path. My next step will be to add CPU emulation support for testing. Please let me know if you have any other suggestions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Backend][Relax] Add Intel GNA backend for NPU support [tvm]
Posted by "tqchen (via GitHub)" <gi...@apache.org>.
tqchen commented on PR #18201:
URL: https://github.com/apache/tvm/pull/18201#issuecomment-3218212530
Thanks for the contribution, given GNA is archived, it perhaps does not make sense to maintain it in the main tree, adding ci will also add extra overhead here. However, i agree that having generic tutorials for BYOC NPU would be useful, if we can have something that support a current NPU that would be great
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org