You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@arrow.apache.org by Lei Xu <le...@eto.ai> on 2022/12/13 21:19:41 UTC
[C++] Undefined reference during static link Arrow (10.0.1) in Manylinux2014 container
Hi, there,
I was trying to statically link Arrow / ArrowDataset into one of my
projects in Manylinux,
it reports missing the reference to standard C++ functions.
```
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
/usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
`arrow::dataset::(anonymous
namespace)::BasicFragmentEvolution::EvolveBatch(std::shared_ptr<arrow::RecordBatch>
const&, std::vector<arrow::FieldPath, std::allocator<arrow::FieldPath> >
const&, std::vector<arrow::dataset::FragmentSelectionColumn,
std::allocator<arrow::dataset::FragmentSelectionColumn> > const&) const':
(.text+0xe9b): undefined reference to `std::__throw_bad_array_new_length()'
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
/usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
`arrow::dataset::UnionDataset::ReplaceSchema(std::shared_ptr<arrow::Schema>)
const [clone .localalias]':
(.text+0x1cac): undefined reference to `std::__throw_bad_array_new_length()'
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
/usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
`arrow::Result<arrow::compute::Expression>
arrow::compute::ModifyExpression<arrow::dataset::(anonymous
namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
const&) const::{lambda(arrow::compute::Expression)#1},
arrow::dataset::(anonymous
namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
const&) const::{lambda(arrow::compute::Expression,
arrow::compute::Expression*)#2}>(arrow::compute::Expression,
arrow::dataset::(anonymous
namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
const&) const::{lambda(arrow::compute::Expression)#1} const&,
arrow::dataset::(anonymous
namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
const&) const::{lambda(arrow::compute::Expression,
arrow::compute::Expression*)#2} const&)':
(.text+0x384f): undefined reference to `std::__throw_bad_array_new_length()'
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
(.text+0x3867): undefined reference to `std::__throw_bad_array_new_length()'
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
/usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
`arrow::dataset::(anonymous
namespace)::BasicFragmentEvolution::DevolveSelection(std::vector<arrow::FieldPath,
std::allocator<arrow::FieldPath> > const&) const':
(.text+0x4ea6): undefined reference to `std::__throw_bad_array_new_length()'
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
/usr/lib64/libarrow_dataset.a(dataset.cc.o):(.text+0x4eb6): more undefined
references to `std::__throw_bad_array_new_length()' follow
/opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
/usr/lib64/libarrow_bundled_dependencies.a(regexp.cc.o): in function
`re2::ConvertRunesToBytes(bool, int*, int, std::string*)':
(.text+0x10d1): undefined reference to `std::string::reserve()'
```
My Manylinux2014 dockerfile
```
FROM quay.io/pypa/manylinux2014_x86_64
ENV LD_LIBRARY_PATH=/usr/local/lib
ENV ARROW_VERSION=10.0.1-1.el7
RUN yum update -y \
&& yum install -y epel-release || yum install -y
https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(cut -d:
-f5 /etc/system-release-cpe | cut -d. -f1).noarch.rpm \
&& yum install -y
https://apache.jfrog.io/artifactory/arrow/centos/7/x86_64/Packages/apache-arrow-release-${ARROW_VERSION}.noarch.rpm
\
&& yum install -y --enablerepo=epel \
arrow-devel-${ARROW_VERSION} \
arrow-dataset-devel-${ARROW_VERSION} \
```
Is the reason because "libarrow_dataset.a" was built by another version of
the toolchain?
Best,
--
Lei Xu
Eto.ai
Re: [C++] Undefined reference during static link Arrow (10.0.1) in Manylinux2014 container
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
Our CentOS 7 packages use devtoolset-11 not devtoolset-10:
https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/apache-arrow/yum/centos-7/Dockerfile#L22
Could you try devtoolset-11 not devtoolset-10 or use
binaries in pyarrow's manylinux wheel instead?
Thanks,
--
kou
In <CA...@mail.gmail.com>
"[C++] Undefined reference during static link Arrow (10.0.1) in Manylinux2014 container" on Tue, 13 Dec 2022 13:19:41 -0800,
Lei Xu <le...@eto.ai> wrote:
> Hi, there,
>
> I was trying to statically link Arrow / ArrowDataset into one of my
> projects in Manylinux,
>
> it reports missing the reference to standard C++ functions.
>
> ```
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> /usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
> `arrow::dataset::(anonymous
> namespace)::BasicFragmentEvolution::EvolveBatch(std::shared_ptr<arrow::RecordBatch>
> const&, std::vector<arrow::FieldPath, std::allocator<arrow::FieldPath> >
> const&, std::vector<arrow::dataset::FragmentSelectionColumn,
> std::allocator<arrow::dataset::FragmentSelectionColumn> > const&) const':
> (.text+0xe9b): undefined reference to `std::__throw_bad_array_new_length()'
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> /usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
> `arrow::dataset::UnionDataset::ReplaceSchema(std::shared_ptr<arrow::Schema>)
> const [clone .localalias]':
> (.text+0x1cac): undefined reference to `std::__throw_bad_array_new_length()'
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> /usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
> `arrow::Result<arrow::compute::Expression>
> arrow::compute::ModifyExpression<arrow::dataset::(anonymous
> namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
> const&) const::{lambda(arrow::compute::Expression)#1},
> arrow::dataset::(anonymous
> namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
> const&) const::{lambda(arrow::compute::Expression,
> arrow::compute::Expression*)#2}>(arrow::compute::Expression,
> arrow::dataset::(anonymous
> namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
> const&) const::{lambda(arrow::compute::Expression)#1} const&,
> arrow::dataset::(anonymous
> namespace)::BasicFragmentEvolution::DevolveFilter(arrow::compute::Expression
> const&) const::{lambda(arrow::compute::Expression,
> arrow::compute::Expression*)#2} const&)':
> (.text+0x384f): undefined reference to `std::__throw_bad_array_new_length()'
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> (.text+0x3867): undefined reference to `std::__throw_bad_array_new_length()'
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> /usr/lib64/libarrow_dataset.a(dataset.cc.o): in function
> `arrow::dataset::(anonymous
> namespace)::BasicFragmentEvolution::DevolveSelection(std::vector<arrow::FieldPath,
> std::allocator<arrow::FieldPath> > const&) const':
> (.text+0x4ea6): undefined reference to `std::__throw_bad_array_new_length()'
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> /usr/lib64/libarrow_dataset.a(dataset.cc.o):(.text+0x4eb6): more undefined
> references to `std::__throw_bad_array_new_length()' follow
> /opt/rh/devtoolset-10/root/usr/libexec/gcc/x86_64-redhat-linux/10/ld:
> /usr/lib64/libarrow_bundled_dependencies.a(regexp.cc.o): in function
> `re2::ConvertRunesToBytes(bool, int*, int, std::string*)':
> (.text+0x10d1): undefined reference to `std::string::reserve()'
> ```
>
> My Manylinux2014 dockerfile
>
> ```
>
> FROM quay.io/pypa/manylinux2014_x86_64
>
> ENV LD_LIBRARY_PATH=/usr/local/lib
>
> ENV ARROW_VERSION=10.0.1-1.el7
>
> RUN yum update -y \
> && yum install -y epel-release || yum install -y
> https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(cut -d:
> -f5 /etc/system-release-cpe | cut -d. -f1).noarch.rpm \
> && yum install -y
> https://apache.jfrog.io/artifactory/arrow/centos/7/x86_64/Packages/apache-arrow-release-${ARROW_VERSION}.noarch.rpm
> \
> && yum install -y --enablerepo=epel \
> arrow-devel-${ARROW_VERSION} \
> arrow-dataset-devel-${ARROW_VERSION} \
>
> ```
>
> Is the reason because "libarrow_dataset.a" was built by another version of
> the toolchain?
>
> Best,
>
> --
> Lei Xu
> Eto.ai