You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "berkaysynnada (via GitHub)" <gi...@apache.org> on 2024/11/21 08:19:45 UTC
[I] recursive Dependency Causes "section too large" Error When Compiling for wasm32-unknown-unknown [datafusion]
berkaysynnada opened a new issue, #13513:
URL: https://github.com/apache/datafusion/issues/13513
### Describe the bug
I am encountering a problem when trying to compile wasmtest with the following command:
`RUSTFLAGS="-C link-arg=--max-memory=4294967296" cargo build --release --target wasm32-unknown-unknown`
The build fails with the following error:
```
error: failed to build archive at `.../target/wasm32-unknown-unknown/release/deps/libpsm-cc498cff4b514c30.rlib`: LLVM error: section too large
The following warnings were emitted during compilation:
warning: psm@0.1.24: warning: ...
```
I suspect the issue is related to the psm dependency, and it is coming with recursive crate, which was introduced in [#13310](https://github.com/apache/datafusion/pull/13310). I am unsure if this error can be resolved without removing or replacing this dependency.
### To Reproduce
Go in `datafusion/wasmtest` and` run cargo build --target wasm32-unknown-unknown`
### Expected behavior
It needs to compile
### Additional context
I tried again after removing `recursive` dep, and it works successfully
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "Eason0729 (via GitHub)" <gi...@apache.org>.
Eason0729 commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2500954555
I have try to reproduce the bug by either build wasmtest or build `stacker` with wasm32-unknown-unknown, but was unable to reproduce the issue.
Can you provide more information? (cargo and rustc version...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "peter-toth (via GitHub)" <gi...@apache.org>.
peter-toth commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2543156645
@berkaysynnada, is this sill an issue? https://github.com/rustwasm/wasm-pack/issues/1381#issuecomment-2153142927 seems to work for me too.
```
% export PATH=/opt/homebrew/Cellar/llvm/19.1.4/bin/:$PATH
% clang --version
Homebrew clang version 19.1.4
Target: arm64-apple-darwin23.6.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.4/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg
% rustc --version --verbose
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: aarch64-apple-darwin
release: 1.83.0
LLVM version: 19.1.1
% cargo clean
...
% cargo build --release --target wasm32-unknown-unknown
...
Compiling datafusion-wasmtest v43.0.0 (/Users/ptoth/git/apache/datafusion/datafusion/wasmtest)
Finished `release` profile [optimized] target(s) in 1m 36s
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "berkaysynnada (via GitHub)" <gi...@apache.org>.
berkaysynnada closed issue #13513: "recursive" Dependency Causes "section too large" Error When Compiling for wasm
URL: https://github.com/apache/datafusion/issues/13513
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "robtaylor (via GitHub)" <gi...@apache.org>.
robtaylor commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2606895980
I'm still getting this issue. Mac M4 with Sequoia 15.2.
```
% rustc --version --verbose
rustc 1.84.0 (9fc6b4312 2025-01-07)
binary: rustc
commit-hash: 9fc6b43126469e3858e2fe86cafb4f0fd5068869
commit-date: 2025-01-07
host: aarch64-apple-darwin
release: 1.84.0
LLVM version: 19.1.5
```
```
error: failed to build archive at `[cut]/wasm32-unknown-unknown/release/deps/libpsm-5686a1fe6f354cae.rlib`: LLVM error: section too large
warning: psm@0.1.24: warning: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: [cut]wasm32-unknown-unknown/release/build/psm-2da2e2ac984ce6da/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)
error: could not compile `psm` (lib) due to 1 previous error
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "blaginin (via GitHub)" <gi...@apache.org>.
blaginin commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2501933839
This obviously doesn't resolve the issue, but https://github.com/rustwasm/wasm-pack/issues/1381#issuecomment-2153142927 fixed it for me
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "Eason0729 (via GitHub)" <gi...@apache.org>.
Eason0729 commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2509777392
It compiles under 1.82.0 and 1.83.0(didn't use LLVM 15), I think it's LLVM version related.
In addition, I may need some help on how to use specific version of LLVM?
```
root@73b0a356b106:/stacker/psm# echo $PATH
/usr/lib/llvm-15/bin:/root/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
```
```
root@73b0a356b106:/stacker/psm# clang --version
Debian clang version 15.0.6
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm-15/bin
```
```
root@73b0a356b106:/stacker/psm# rustc --version --verbose
rustc 1.83.0 (90b35a623 2024-11-26)
binary: rustc
commit-hash: 90b35a6239c3d8bdabc530a6a0816f7ff89a0aaf
commit-date: 2024-11-26
host: x86_64-unknown-linux-gnu
release: 1.83.0
LLVM version: 19.1.1
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "berkaysynnada (via GitHub)" <gi...@apache.org>.
berkaysynnada commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2543177609
That worked for me now (and passed the tests). Perhaps I should have tried with cargo clean 😞
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [I] "recursive" Dependency Causes "section too large" Error When Compiling for wasm [datafusion]
Posted by "berkaysynnada (via GitHub)" <gi...@apache.org>.
berkaysynnada commented on issue #13513:
URL: https://github.com/apache/datafusion/issues/13513#issuecomment-2507725495
I have applied the @blaginin suggestion:
```
berkaysahin@Berkays-MacBook-Pro wasmtest % clang --version
Homebrew clang version 19.1.4
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /opt/homebrew/Cellar/llvm/19.1.4/bin
Configuration file: /opt/homebrew/etc/clang/arm64-apple-darwin23.cfg
```
@Eason0729 these are my cargo and rust versions
```
berkaysahin@Berkays-MacBook-Pro wasmtest % cargo --version
cargo 1.82.0 (8f40fc59f 2024-08-21)
berkaysahin@Berkays-MacBook-Pro wasmtest % rustc --version
rustc 1.82.0 (f6e511eec 2024-10-15)
```
When I build with: `cargo build --target wasm32-unknown-unknown `,
it successfully compiles and generates the binary.
However, in the release mode: `cargo build --release --target wasm32-unknown-unknown`,
```
Compiling psm v0.1.24
Compiling arrow-ipc v53.3.0
Compiling arrow-json v53.3.0
Compiling arrow-csv v53.3.0
Compiling stacker v0.1.17
Compiling recursive v0.1.1
error: failed to build archive at `/Users/berkaysahin/Desktop/datafusion-upstream/target/wasm32-unknown-unknown/release/deps/libpsm-cc498cff4b514c30.rlib`: LLVM error: section too large
The following warnings were emitted during compilation:
warning: psm@0.1.24: warning: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: archive library: /Users/berkaysahin/Desktop/datafusion-upstream/target/wasm32-unknown-unknown/release/build/psm-d273ebae732f9287/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)
error: could not compile `psm` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
```
I don't know why this still refers to Xcode path.
Since wasm-pack build goes with release flag, we need to solve this (assuming I am not the only one)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org