You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by "JieGH (via GitHub)" <gi...@apache.org> on 2025/02/05 10:14:21 UTC
[I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
JieGH opened a new issue, #17625:
URL: https://github.com/apache/tvm/issues/17625
# TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1
## Expected Behavior
I have a vecter-based TVM design that shows poor performance against C implementation, 1/4 of performance approx.
The LLVM warns VLEN to be set as default 128 bits, but the Banana Pi should uses 256 bits instead.
TVM and LLVM should correctly detect and utilize a vector length (VLEN) of 256 bits on the Banana Pi K1 board.
## Actual Behavior
- The board reports `VLEN=256` when querying via `csrr a4, vlenb` in assembly.
-- The ASM code shows:
```
vector_test_compute_:
.Lfunc_begin1:
.loc 1 0 0
.cfi_startproc
csrr a4, vlenb
srli t0, a4, 1
li a2, 1024
bltu a2, t0, .LBB1_7
.Ltmp107:
addi a2, t0, -1
andi a7, a2, 1024
xori a6, a7, 1024
slli t1, a4, 1
mv a5, a1
mv a3, a0
mv a2, a6
vsetvli a4, zero, e32, m2, ta, ma
```
- However, LLVM outputs a warning: ``src/target/llvm/codegen_llvm.cc:185: Warning: Set native vector bits to be 128 for riscv64``
- Attempts to override with flags like `-mllvm -riscv-v-vector-bits-min=256 -mllvm -riscv-v-vector-bits-max=256` do not take effect, llvm says it illegal flags, then it recommend
- ```ValueError: Error when parsing target["mllvm"]: Cannot recognize 'mllvm'. Candidates are: cl-opt, opt-level, fast-math-ninf, fast-math-arcp, fast-math-nnan, fast-math, fast-math-contract, num-cores, device, libs, tag, mtriple, host, from_device, target_device_type, fast-math-reassoc, keys, mattr, fast-math-nsz, model, mabi, mcpu, jit, mfloat-abi. Target creation from string failed: llvm -jit=orcjit -mtriple=riscv64-linux-gnu -mcpu=generic-rv64 -mattr=+m,+a,+f,+d,+zfh,+v,+c -mllvm -riscv-v-vector-bits-min=256 -mllvm -riscv-v-vector-bits-max=256```
### Could anyone advice me how can I set the correct vector length to 256 bits? And from your @cbalint13 knowledge of RISCV, could you provide some advice? Thanks all.
## Environment
```
LLVM version 19.1.3
Optimized build with assertions.
Default target: riscv64-linux-gnu
Host CPU: generic-rv64
Registered Targets:
riscv32 - 32-bit RISC-V
riscv64 - 64-bit RISC-V
TVM: 0.18.0
Linux k1 6.1.15 #1.0.15.1 SMP PREEMPT Sat Sep 7 07:20:18 UTC 2024 riscv64 riscv64 riscv64 GNU/Linux
riscv64-linux-gnu-gcc --version
riscv64-linux-gnu-gcc (Ubuntu 13.2.0-4ubuntu3-bb2) 13.2.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
```
llvm -jit=orcjit -mtriple=riscv64-linux-gnu -mcpu=generic-rv64 -mattr=+m,+a,+f,+d,+zfh,+v,+c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
Posted by "JieGH (via GitHub)" <gi...@apache.org>.
JieGH commented on issue #17625:
URL: https://github.com/apache/tvm/issues/17625#issuecomment-2839289097
Thanks for the update. My temporary solution is to modify the LLVM backend code inside the TVM project to force VLEN to be 256 bits and verify the vlen from the ASM code
The method of setting `-march= rv64gcv_zvl256b` gives error `ValueError: Error when parsing target["march"]: Cannot recognize 'march'.`
I will now try TVM 0.20 with newer LLVM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
Posted by "JieGH (via GitHub)" <gi...@apache.org>.
JieGH commented on issue #17625:
URL: https://github.com/apache/tvm/issues/17625#issuecomment-2647652356
Hi @cbalint13, Thanks for the advice. I now have a method for choosing VLEN, which is using an additional flag:
` tvm.target.Target("llvm -jit=orcjit -mtriple=riscv64 -mcpu=spacemit-x60 - mattr=+64bit,+m,+a,+f,+d,+c,+zfh,+v,+zvl256b")`
By specifying zvl256b flags, it means enable 'Zvl' (Minimum Vector Length) 256. This indeed has an impact on the execution's performance.
However,
1) the TVM still warns the 128bit sets as default bit length despite the zvl flags having been enabled and having an impact on performance. I have not yet run the latest update you posted at Handle vector width (VLEN) for RISCV arches #17631
2) I searched zvl flags for a given matrix mul problem; I changed the zvl and measured the performance. The best performance appeared at the vector length that the chip should not support. For example if I set zvl256b, the execution takes 491ms to complete, if I set zvl8192b, the execution takes 384 ms to finish, which has over 20% speed up. There are something wrong here.
Any comments on this? Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
Posted by "jerryzj (via GitHub)" <gi...@apache.org>.
jerryzj commented on issue #17625:
URL: https://github.com/apache/tvm/issues/17625#issuecomment-2813367252
Simply set `-march=rv64gcv_zvl256b` when compiling should works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
Posted by "cbalint13 (via GitHub)" <gi...@apache.org>.
cbalint13 commented on issue #17625:
URL: https://github.com/apache/tvm/issues/17625#issuecomment-2636471223
Hi @JieGH
Regarding arch specific technical details {CPU: type, variations, features} TVM relay on LLVM itself (it queries the very library). AFAIK, when its about RISCV land, there are limited [infos within LLVM available](https://github.com/topperc/llvm-project/blob/main/llvm/lib/Target/RISCV/RISCVProcessors.td), limited to interest, maintainance, with things mostly coming from SiFive folks.
Let me look and tackle this problem and come up with best way for TVM, here are some possibilities on my mind:
0. Let me look once again in LLVM what info regarding VLEN can be queried, and implement such query for the VLEN.
If the LLVM query don't work (cpu variation is not enlisted in LLVM) let's further override within TVM:
1. Let's also make the flag for TVM like "-riscv-v-vector-bits-min=256" to work properly (currently TVM hardcode it to 256).
2. Let's also declare the arches in TVM way (spacemit, th1520, C906, etc) [see here](https://github.com/apache/tvm/blob/9404fb5a59766d19fb89944c8c4283b44c30977a/python/tvm/target/target.py#L600), but this requires maintenance on long term.
I believe {0 + 1, 2} can be implemented all together to offer users multiple ways of control.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
Posted by "cbalint13 (via GitHub)" <gi...@apache.org>.
cbalint13 commented on issue #17625:
URL: https://github.com/apache/tvm/issues/17625#issuecomment-2648825520
H@JieGH ,
> Hi @cbalint13, Thanks for the advice. I now have a method for choosing VLEN, which is using an additional flag: ` tvm.target.Target("llvm -jit=orcjit -mtriple=riscv64 -mcpu=spacemit-x60 - mattr=+64bit,+m,+a,+f,+d,+c,+zfh,+v,+zvl256b")`
>
> By specifying zvl256b flags, it means enable 'Zvl' (Minimum Vector Length) 256. This indeed has an impact on the execution's performance. However,
Yes, another way to tell LLVM the VLEN is via the canonical flags, but we also need TVM itself to be aware of this.
> 1. the TVM still warns the 128bit sets as default bit length despite the zvl flags having been enabled and having an impact on performance. I have not yet run the latest update you posted at Handle vector width (VLEN) for RISCV arches [Handle vector width (VLEN) for RISCV arches #17631](https://github.com/apache/tvm/pull/17631)
You have an older LLVM, and it does not know about ```-mcpu=spacemit-x60```, so it will fall as a ```generic```.
* Can check llvm version from tvm side:
```
$ python3 -c "import tvm; print(tvm.target.codegen.llvm_version_major())"
20
```
* Can also look inside riscv64 what ```-mcpu``` options are there for your installed LLVM:
```
$ python -c "import tvm; print(tvm.target.codegen.llvm_get_cpu_archlist(tvm.target.Target('llvm -mtriple=riscv64--')))"
["generic", "generic-rv32", "generic-rv64", "mips-p8700", "rocket",
"rocket-rv32", "rocket-rv64", "rp2350-hazard3", "sifive-7-series",
"sifive-e20", "sifive-e21", "sifive-e24", "sifive-e31", "sifive-e34",
"sifive-e76", "sifive-p450", "sifive-p470", "sifive-p670", "sifive-s21",
"sifive-s51", "sifive-s54", "sifive-s76", "sifive-u54", "sifive-u74",
"sifive-x280", "spacemit-x60", "syntacore-scr1-base", "syntacore-scr1-max",
"syntacore-scr3-rv32", "syntacore-scr3-rv64", "syntacore-scr4-rv32",
"syntacore-scr4-rv64", "syntacore-scr5-rv32", "syntacore-scr5-rv64",
"syntacore-scr7", "tt-ascalon-d8", "veyron-v1", "xiangshan-nanhu"]
```
The flags (older LLVM) would be: *llvm -device=riscv_cpu -vector-width=256 -mtriple=riscv64-linux-gnu -mcpu=generic-rv64 -mattr=+64bit,+a,+c,+d,+f,+m,+v* (orcjit is already default, vector-with informs booth TVM and LLVM).
> 2. I searched zvl flags for a given matrix mul problem; I changed the zvl and measured the performance. The best performance appeared at the vector length that the chip should not support. For example if I set zvl256b, the execution takes 491ms to complete, if I set zvl8192b, the execution takes 384 ms to finish, which has over 20% speed up. There are something wrong here.>
> Any comments on this? Thanks.
Performance also depends on how LLVM optimizes things out, TVM have no highly-specialized optimizations for RISCV.
TVM emmits candidates/iterations as intermediate proposals (in auto-tunnig flow) and forwards to LLVM, while electing only the best performing ones. Not sure if you are also tring to tune your model/function, but without a tuning process TVM likely emits a subperforming variant, even for a simple matmul operation, there should be a warn on this:
```
WARNING:autotvm:One or more operators have not been tuned.
Please tune your model for better performance. Use DEBUG logging level to see more details.
```
The work done in https://github.com/apache/tvm/pull/17631 only informs TVM about VLEN intentions from LLVM side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1 [tvm]
Posted by "JieGH (via GitHub)" <gi...@apache.org>.
JieGH closed issue #17625: [Bug] TVM/LLVM sets RISC-V VLEN to 128 bits instead of 256 on Banana Pi K1
URL: https://github.com/apache/tvm/issues/17625
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org