You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "NEUpanning (via GitHub)" <gi...@apache.org> on 2024/07/08 12:36:53 UTC
[I] Failed to build Velox backend [incubator-gluten]
NEUpanning opened a new issue, #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374
### Problem description
```
git clone https://github.com/apache/incubator-gluten.git gluten
cd gluten/ep/build-velox/src/
./get_velox.sh
# failed command
./build_velox.sh --build_tests=ON --build_test_utils=ON --build_benchmarks=ON --run_setup_script=OFF
```
### System information
Velox System Info v0.0.2
Commit: b0b7ae98c45c6df69ee7024b4047eec7b6e7266f
CMake Version: 3.28.1
System: Linux-3.10.0-862.mt20190308.130.el7.x86_64
Arch: x86_64
CPU Name: Model name: Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz
C++ Compiler: /opt/rh/devtoolset-10/root/usr/bin/c++
C++ Compiler Version: 10.2.1
C Compiler: /opt/rh/devtoolset-10/root/usr/bin/cc
C Compiler Version: 10.2.1
CMake Prefix Path: /usr/local;/usr;/;/usr/local;/usr/local;/usr/X11R6;/usr/pkg;/opt
\nThe results will be copied to your clipboard if xclip is installed.
### CMake log
```bash
[921/1563] Building CXX object velox/vector/arrow/CMakeFiles/velox_arrow_bridge.dir/Bridge.cpp.o
FAILED: velox/vector/arrow/CMakeFiles/velox_arrow_bridge.dir/Bridge.cpp.o
/usr/bin/ccache /opt/rh/devtoolset-10/root/usr/bin/c++ -DBOOST_ATOMIC_DYN_LINK -DBOOST_ATOMIC_NO_LIB -DBOOST_CONTEXT_DYN_LINK -DBOOST_CONTEXT_NO_LIB -DBOOST_FILESYSTEM_DYN_LINK -DBOOST_FILESYSTEM_NO_LIB -DBOOST_PROGRAM_OPTIONS_DYN_LINK -DBOOST_PROGRAM_OPTIONS_NO_LIB -DBOOST_REGEX_DYN_LINK -DBOOST_REGEX_NO_LIB -DBOOST_SYSTEM_DYN_LINK -DBOOST_SYSTEM_NO_LIB -DBOOST_THREAD_DYN_LINK -DBOOST_THREAD_NO_LIB -DFOLLY_HAVE_INT128_T=1 -DGFLAGS_IS_A_DLL=0 -DVELOX_ENABLE_PARQUET -I/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/_build/release/_deps/protobuf-src/src -I/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/. -I/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/external/xxhash -I/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/_build/release/_deps/re2-src -isystem /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox -isystem /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/external -Wno-miss
ing-field-initializers -march=native -std=c++17 -mno-avx512f -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused -Wno-unused-parameter -Wno-sign-compare -Wno-ignored-qualifiers -Wnon-virtual-dtor -Wno-implicit-fallthrough -Wno-class-memaccess -Wno-comment -Wno-int-in-bool-context -Wno-redundant-move -Wno-array-bounds -Wno-maybe-uninitialized -Wno-unused-result -Wno-format-overflow -Wno-strict-aliasing -Werror -O3 -DNDEBUG -std=gnu++17 -fPIC -fdiagnostics-color=always -MD -MT velox/vector/arrow/CMakeFiles/velox_arrow_bridge.dir/Bridge.cpp.o -MF velox/vector/arrow/CMakeFiles/velox_arrow_bridge.dir/Bridge.cpp.o.d -o velox/vector/arrow/CMakeFiles/velox_arrow_bridge.dir/Bridge.cpp.o -c /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/vector/arrow/Bridge.cpp
In file included from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/BaseVector.h:37,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/arrow/Bridge.h:20,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/vector/arrow/Bridge.cpp:17:
In member function ‘void facebook::velox::SelectivityVector::clearAll()’,
inlined from ‘std::enable_if_t<is_same_v<U, facebook::velox::StringView>, void> facebook::velox::SimpleVector<T>::invalidateIsAscii() [with U = facebook::velox::StringView; T = facebook::velox::StringView]’ at /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SimpleVector.h:315:55,
inlined from ‘void facebook::velox::SimpleVector<T>::resetDataDependentFlags(const facebook::velox::SelectivityVector*) [with T = facebook::velox::StringView]’ at /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SimpleVector.h:434:26:
/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SelectivityVector.h:163:18: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=]
163 | allSelected_ = false;
| ~~~~~~~~~~~~~^~~~~~~
In file included from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/common/memory/Memory.h:27,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/arrow/Bridge.h:19,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/vector/arrow/Bridge.cpp:17:
/usr/local/include/folly/Synchronized.h: In member function ‘void facebook::velox::SimpleVector<T>::resetDataDependentFlags(const facebook::velox::SelectivityVector*) [with T = facebook::velox::StringView]’:
/usr/local/include/folly/Synchronized.h:1004:17: note: at offset -4 to object ‘folly::Synchronized<facebook::velox::SelectivityVector>::mutex_’ with size 4 declared here
1004 | mutable Mutex mutex_;
| ^~~~~~
In file included from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/BaseVector.h:37,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/arrow/Bridge.h:20,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/vector/arrow/Bridge.cpp:17:
In member function ‘void facebook::velox::SelectivityVector::clearAll()’,
inlined from ‘std::enable_if_t<is_same_v<U, facebook::velox::StringView>, void> facebook::velox::SimpleVector<T>::invalidateIsAscii() [with U = facebook::velox::StringView; T = facebook::velox::StringView]’ at /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SimpleVector.h:315:55,
inlined from ‘void facebook::velox::SimpleVector<T>::resetDataDependentFlags(const facebook::velox::SelectivityVector*) [with T = facebook::velox::StringView]’ at /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SimpleVector.h:434:26:
/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SelectivityVector.h:163:18: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=]
163 | allSelected_ = false;
| ~~~~~~~~~~~~~^~~~~~~
In file included from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/common/memory/Memory.h:27,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/arrow/Bridge.h:19,
from /home/panning04/incubator-gluten/ep/build-velox/build/velox_ep/velox/vector/arrow/Bridge.cpp:17:
/usr/local/include/folly/Synchronized.h: In member function ‘void facebook::velox::SimpleVector<T>::resetDataDependentFlags(const facebook::velox::SelectivityVector*) [with T = facebook::velox::StringView]’:
/usr/local/include/folly/Synchronized.h:1004:17: note: at offset -3 to object ‘folly::Synchronized<facebook::velox::SelectivityVector>::mutex_’ with size 4 declared here
1004 | mutable Mutex mutex_;
| ^~~~~~
cc1plus: all warnings being treated as errors
[922/1563] Building CXX object _deps/duckdb-build/src/common/CMakeFiles/duckdb_common.dir/ub_duckdb_common.cpp.o
ninja: build stopped: subcommand failed.
make[1]: *** [Makefile:103: build] Error 1
make[1]: Leaving directory '/home/panning04/incubator-gluten/ep/build-velox/build/velox_ep'
make: *** [Makefile:110: release] Error 2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2222228194
Did you try static build?
It seems dynamic build is unreliable at the moment. You can follow the steps in CI to create a static build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "NEUpanning (via GitHub)" <gi...@apache.org>.
NEUpanning commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2244754286
@dcoliversun
1. Maybe you can try to remove velox build directory and build again.
2. I build gluten using [vcpkg](https://github.com/apache/incubator-gluten/tree/main/dev/vcpkg).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "majetideepak (via GitHub)" <gi...@apache.org>.
majetideepak commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2245404400
Opened https://github.com/facebookincubator/velox/pull/10530
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2227581466
@zhztheplayer I try with CI workflow on a clean machine, which is https://github.com/apache/incubator-gluten/blob/45cfdbe9d2093ecbf702bbc3cdb453c11904db5f/.github/workflows/build_bundle_package.yml. I get the same error.
```shell
yum install sudo patch java-1.8.0-openjdk-devel -y && \
cd $GITHUB_WORKSPACE/ep/build-velox/src && \
./get_velox.sh && \
source /opt/rh/devtoolset-9/enable && \
source /opt/gluten//dev/vcpkg/env.sh && \
cd $GITHUB_WORKSPACE/ && \
sed -i '/^headers/d' ep/build-velox/build/velox_ep/CMakeLists.txt && \
export NUM_THREADS=4
./dev/builddeps-veloxbe.sh --build_tests=OFF --build_benchmarks=OFF --enable_s3=OFF \
--enable_gcs=OFF --enable_hdfs=ON --enable_abfs=OFF
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "majetideepak (via GitHub)" <gi...@apache.org>.
majetideepak commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2245393310
@NEUpanning This needs another fix on the Velox side similar to the previous one.
```
In member function ‘void facebook::velox::SelectivityVector::updateBounds()’,
inlined from ‘void facebook::velox::SelectivityVector::deselect(const facebook::velox::SelectivityVector&)’ at /root/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SelectivityVector.h:208:17,
inlined from ‘void facebook::velox::SimpleVector<T>::resetDataDependentFlags(const facebook::velox::SelectivityVector*) [with T = facebook::velox::StringView]’ at /root/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SimpleVector.h:432:59:
/root/incubator-gluten/ep/build-velox/build/velox_ep/./velox/vector/SelectivityVector.h:297:20: error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=]
297 | allSelected_ = false;
| ~~~~~~~~~~~~~^~~~~~~
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "zhztheplayer (via GitHub)" <gi...@apache.org>.
zhztheplayer commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2229958181
@dcoliversun As a possible temporary solution, would you please try with [this method](https://github.com/apache/incubator-gluten/tree/main/tools/gluten-te/ubuntu/examples/buildhere-veloxbe-portable-libs) to create a build in docker container? It should be a more general way for building.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "majetideepak (via GitHub)" <gi...@apache.org>.
majetideepak commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2231859489
This issue is being fixed on the Velox side https://github.com/facebookincubator/velox/pull/10469
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2244318395
@NEUpanning Thanks for your share. And I get the old error with latest velox (https://github.com/oap-project/velox/tree/2024_07_19) and gluten (2a6a974d6fbaa38869eb9a0b91b2e796a578884c). Could you please tell me how to solve it?
command: `nohup ./build_velox.sh --build_tests=ON --build_test_utils=ON --build_benchmarks=ON --run_setup_script=OFF &> nohup.log &`
log: [nohup.log](https://github.com/user-attachments/files/16344322/nohup.log)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "dcoliversun (via GitHub)" <gi...@apache.org>.
dcoliversun commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2246695287
@NEUpanning @majetideepak Thanks for your help :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "NEUpanning (via GitHub)" <gi...@apache.org>.
NEUpanning commented on issue #6374:
URL: https://github.com/apache/incubator-gluten/issues/6374#issuecomment-2242730614
After updating gluten and velox to the latest code, I successfully built velox backend using `./build_velox.sh --build_tests=ON --build_test_utils=ON --build_benchmarks=ON --run_setup_script=OFF`. Thanks for your effort @majetideepak
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] Failed to build Velox backend [incubator-gluten]
Posted by "NEUpanning (via GitHub)" <gi...@apache.org>.
NEUpanning closed issue #6374: Failed to build Velox backend
URL: https://github.com/apache/incubator-gluten/issues/6374
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org