You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Rares Vernica <rv...@gmail.com> on 2018/12/12 04:47:45 UTC
C++ buildings and Regex issue
Hello,
We are using the C++ bindings of Arrow 0.9.0 on our system on CentOS. Once
we load the Arrow library, our regular regex calls (outside of Arrow)
misbehave and trigger some unknown crashes. We are still trying to figure
things out but I was wondering if there are any know issues regarding regex
and the C++ binding. Also, how can one turn on/off flags related to regex
when compiling Arrow? We are still trying to isolate the crash.
Thanks!
Rares
Re: C++ buildings and Regex issue
Posted by Rares Vernica <rv...@gmail.com>.
Worked like a charm. Thanks a lot! Much appreciated!
--
Rares
On Tue, Dec 11, 2018 at 11:20 PM Kouhei Sutou <ko...@clear-code.com> wrote:
> Hi,
>
> Can you try "-DBOOST_ROOT=${YOUR_BOOST_INSTALL_PREFIX}" option?
>
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: C++ buildings and Regex issue" on Tue, 11 Dec 2018 22:53:58 -0800,
> Rares Vernica <rv...@gmail.com> wrote:
>
> > Hi,
> >
> > Unfortunately we need to stay on CentOS 6 for now.
> >
> > We have a locally built libboost-devel-1.54 for CentOS 6 which installs
> in
> > a custom location. I added the installation steps at the end of
> >
> https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/centos-6/Dockerfile
> > and the library is in the Docker container now. How can I ask Arrow to
> pick
> > up this Boost library from its custom location?
> >
> > Right now I see this:
> >
> >
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> > boost::filesystem::path_traits>' has no member named 'make_preferred'
> > i.make_preferred();
> > ^~~~~~~~~~~~~~
> >
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> > boost::filesystem::path_traits>' has no member named 'native'
> > out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
> >
> > and I assume Arrow is picking up the default CentOS Boost, which as you
> > mention it won't work.
> >
> > Thanks!
> > Rares
> >
> >
> > On Tue, Dec 11, 2018 at 10:18 PM Kouhei Sutou <ko...@clear-code.com>
> wrote:
> >
> >> Hi,
> >>
> >> You can't use system Boost on CentOS 6. Because system Boost
> >> is old. It's better that you upgrade to CentOS 7.
> >>
> >> Thanks,
> >> --
> >> kou
> >>
> >> In <CA...@mail.gmail.com>
> >> "Re: C++ buildings and Regex issue" on Tue, 11 Dec 2018 22:07:20
> -0800,
> >> Rares Vernica <rv...@gmail.com> wrote:
> >>
> >> > Wes,
> >> >
> >> > Thanks! We do plan to upgrade, as soon as we put down the fire. We
> >> noticed
> >> > some API changes and we will have to get our code updated.
> >> >
> >> > It looks like it is boost::regex. In our application we link
> dynamically
> >> > against a locally compiled Boost. For Arrow we noticed this for CentOS
> >> >
> >>
> https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/arrow.spec.in#L69
> >> >
> >> > %if %{_centos_ver} == 6
> >> > -DARROW_BOOST_VENDORED=ON \
> >> > %endif
> >> >
> >> > I tried replacing it with
> >> >
> >> > -DARROW_BOOST_USE_SHARED=ON
> >> >
> >> > but it does not look like it is going to build
> >> >
> >> >
> >>
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
> >> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> >> > boost::filesystem::path_traits>' has no member named 'make_preferred'
> >> > i.make_preferred();
> >> > ^~~~~~~~~~~~~~
> >> >
> >>
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
> >> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> >> > boost::filesystem::path_traits>' has no member named 'native'
> >> > out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
> >> >
> >> > I remember we had a similar conflict with ProtocolBuffers. In that
> case,
> >> > changing Arrow to use the system provided version did the trick.
> >> >
> >> > Thanks,
> >> > Rares
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Tue, Dec 11, 2018 at 9:30 PM Wes McKinney <we...@gmail.com>
> >> wrote:
> >> >
> >> >> hi,
> >> >>
> >> >> Could you clarify what you mean by "regex calls"? Are you talking
> >> >> about boost::regex, std::regex, something else? How did you link the
> >> >> relevant libraries in each part of your application, and in the Arrow
> >> >> + Parquet libraries
> >> >>
> >> >> 0.9.0 is over 1000 patches ago. I'd recommend that you try to upgrade
> >> >>
> >> >> $ git hist apache-arrow-0.9.0..master | wc -l
> >> >> 1540
> >> >>
> >> >> - Wes
> >> >> On Tue, Dec 11, 2018 at 10:58 PM Rares Vernica <rv...@gmail.com>
> >> wrote:
> >> >> >
> >> >> > Hello,
> >> >> >
> >> >> > We are using the C++ bindings of Arrow 0.9.0 on our system on
> CentOS.
> >> >> Once
> >> >> > we load the Arrow library, our regular regex calls (outside of
> Arrow)
> >> >> > misbehave and trigger some unknown crashes. We are still trying to
> >> figure
> >> >> > things out but I was wondering if there are any know issues
> regarding
> >> >> regex
> >> >> > and the C++ binding. Also, how can one turn on/off flags related to
> >> regex
> >> >> > when compiling Arrow? We are still trying to isolate the crash.
> >> >> >
> >> >> > Thanks!
> >> >> > Rares
> >> >>
> >>
>
Re: C++ buildings and Regex issue
Posted by Kouhei Sutou <ko...@clear-code.com>.
Hi,
Can you try "-DBOOST_ROOT=${YOUR_BOOST_INSTALL_PREFIX}" option?
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: C++ buildings and Regex issue" on Tue, 11 Dec 2018 22:53:58 -0800,
Rares Vernica <rv...@gmail.com> wrote:
> Hi,
>
> Unfortunately we need to stay on CentOS 6 for now.
>
> We have a locally built libboost-devel-1.54 for CentOS 6 which installs in
> a custom location. I added the installation steps at the end of
> https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/centos-6/Dockerfile
> and the library is in the Docker container now. How can I ask Arrow to pick
> up this Boost library from its custom location?
>
> Right now I see this:
>
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
> error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> boost::filesystem::path_traits>' has no member named 'make_preferred'
> i.make_preferred();
> ^~~~~~~~~~~~~~
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
> error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> boost::filesystem::path_traits>' has no member named 'native'
> out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
>
> and I assume Arrow is picking up the default CentOS Boost, which as you
> mention it won't work.
>
> Thanks!
> Rares
>
>
> On Tue, Dec 11, 2018 at 10:18 PM Kouhei Sutou <ko...@clear-code.com> wrote:
>
>> Hi,
>>
>> You can't use system Boost on CentOS 6. Because system Boost
>> is old. It's better that you upgrade to CentOS 7.
>>
>> Thanks,
>> --
>> kou
>>
>> In <CA...@mail.gmail.com>
>> "Re: C++ buildings and Regex issue" on Tue, 11 Dec 2018 22:07:20 -0800,
>> Rares Vernica <rv...@gmail.com> wrote:
>>
>> > Wes,
>> >
>> > Thanks! We do plan to upgrade, as soon as we put down the fire. We
>> noticed
>> > some API changes and we will have to get our code updated.
>> >
>> > It looks like it is boost::regex. In our application we link dynamically
>> > against a locally compiled Boost. For Arrow we noticed this for CentOS
>> >
>> https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/arrow.spec.in#L69
>> >
>> > %if %{_centos_ver} == 6
>> > -DARROW_BOOST_VENDORED=ON \
>> > %endif
>> >
>> > I tried replacing it with
>> >
>> > -DARROW_BOOST_USE_SHARED=ON
>> >
>> > but it does not look like it is going to build
>> >
>> >
>> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
>> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
>> > boost::filesystem::path_traits>' has no member named 'make_preferred'
>> > i.make_preferred();
>> > ^~~~~~~~~~~~~~
>> >
>> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
>> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
>> > boost::filesystem::path_traits>' has no member named 'native'
>> > out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
>> >
>> > I remember we had a similar conflict with ProtocolBuffers. In that case,
>> > changing Arrow to use the system provided version did the trick.
>> >
>> > Thanks,
>> > Rares
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Tue, Dec 11, 2018 at 9:30 PM Wes McKinney <we...@gmail.com>
>> wrote:
>> >
>> >> hi,
>> >>
>> >> Could you clarify what you mean by "regex calls"? Are you talking
>> >> about boost::regex, std::regex, something else? How did you link the
>> >> relevant libraries in each part of your application, and in the Arrow
>> >> + Parquet libraries
>> >>
>> >> 0.9.0 is over 1000 patches ago. I'd recommend that you try to upgrade
>> >>
>> >> $ git hist apache-arrow-0.9.0..master | wc -l
>> >> 1540
>> >>
>> >> - Wes
>> >> On Tue, Dec 11, 2018 at 10:58 PM Rares Vernica <rv...@gmail.com>
>> wrote:
>> >> >
>> >> > Hello,
>> >> >
>> >> > We are using the C++ bindings of Arrow 0.9.0 on our system on CentOS.
>> >> Once
>> >> > we load the Arrow library, our regular regex calls (outside of Arrow)
>> >> > misbehave and trigger some unknown crashes. We are still trying to
>> figure
>> >> > things out but I was wondering if there are any know issues regarding
>> >> regex
>> >> > and the C++ binding. Also, how can one turn on/off flags related to
>> regex
>> >> > when compiling Arrow? We are still trying to isolate the crash.
>> >> >
>> >> > Thanks!
>> >> > Rares
>> >>
>>
Re: C++ buildings and Regex issue
Posted by Rares Vernica <rv...@gmail.com>.
Hi,
Unfortunately we need to stay on CentOS 6 for now.
We have a locally built libboost-devel-1.54 for CentOS 6 which installs in
a custom location. I added the installation steps at the end of
https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/centos-6/Dockerfile
and the library is in the Docker container now. How can I ask Arrow to pick
up this Boost library from its custom location?
Right now I see this:
/root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
error: 'class boost::filesystem::basic_path<std::basic_string<char>,
boost::filesystem::path_traits>' has no member named 'make_preferred'
i.make_preferred();
^~~~~~~~~~~~~~
/root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
error: 'class boost::filesystem::basic_path<std::basic_string<char>,
boost::filesystem::path_traits>' has no member named 'native'
out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
and I assume Arrow is picking up the default CentOS Boost, which as you
mention it won't work.
Thanks!
Rares
On Tue, Dec 11, 2018 at 10:18 PM Kouhei Sutou <ko...@clear-code.com> wrote:
> Hi,
>
> You can't use system Boost on CentOS 6. Because system Boost
> is old. It's better that you upgrade to CentOS 7.
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: C++ buildings and Regex issue" on Tue, 11 Dec 2018 22:07:20 -0800,
> Rares Vernica <rv...@gmail.com> wrote:
>
> > Wes,
> >
> > Thanks! We do plan to upgrade, as soon as we put down the fire. We
> noticed
> > some API changes and we will have to get our code updated.
> >
> > It looks like it is boost::regex. In our application we link dynamically
> > against a locally compiled Boost. For Arrow we noticed this for CentOS
> >
> https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/arrow.spec.in#L69
> >
> > %if %{_centos_ver} == 6
> > -DARROW_BOOST_VENDORED=ON \
> > %endif
> >
> > I tried replacing it with
> >
> > -DARROW_BOOST_USE_SHARED=ON
> >
> > but it does not look like it is going to build
> >
> >
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> > boost::filesystem::path_traits>' has no member named 'make_preferred'
> > i.make_preferred();
> > ^~~~~~~~~~~~~~
> >
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
> > error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> > boost::filesystem::path_traits>' has no member named 'native'
> > out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
> >
> > I remember we had a similar conflict with ProtocolBuffers. In that case,
> > changing Arrow to use the system provided version did the trick.
> >
> > Thanks,
> > Rares
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On Tue, Dec 11, 2018 at 9:30 PM Wes McKinney <we...@gmail.com>
> wrote:
> >
> >> hi,
> >>
> >> Could you clarify what you mean by "regex calls"? Are you talking
> >> about boost::regex, std::regex, something else? How did you link the
> >> relevant libraries in each part of your application, and in the Arrow
> >> + Parquet libraries
> >>
> >> 0.9.0 is over 1000 patches ago. I'd recommend that you try to upgrade
> >>
> >> $ git hist apache-arrow-0.9.0..master | wc -l
> >> 1540
> >>
> >> - Wes
> >> On Tue, Dec 11, 2018 at 10:58 PM Rares Vernica <rv...@gmail.com>
> wrote:
> >> >
> >> > Hello,
> >> >
> >> > We are using the C++ bindings of Arrow 0.9.0 on our system on CentOS.
> >> Once
> >> > we load the Arrow library, our regular regex calls (outside of Arrow)
> >> > misbehave and trigger some unknown crashes. We are still trying to
> figure
> >> > things out but I was wondering if there are any know issues regarding
> >> regex
> >> > and the C++ binding. Also, how can one turn on/off flags related to
> regex
> >> > when compiling Arrow? We are still trying to isolate the crash.
> >> >
> >> > Thanks!
> >> > Rares
> >>
>
Re: C++ buildings and Regex issue
Posted by Kouhei Sutou <ko...@clear-code.com>.
Hi,
You can't use system Boost on CentOS 6. Because system Boost
is old. It's better that you upgrade to CentOS 7.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: C++ buildings and Regex issue" on Tue, 11 Dec 2018 22:07:20 -0800,
Rares Vernica <rv...@gmail.com> wrote:
> Wes,
>
> Thanks! We do plan to upgrade, as soon as we put down the fire. We noticed
> some API changes and we will have to get our code updated.
>
> It looks like it is boost::regex. In our application we link dynamically
> against a locally compiled Boost. For Arrow we noticed this for CentOS
> https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/arrow.spec.in#L69
>
> %if %{_centos_ver} == 6
> -DARROW_BOOST_VENDORED=ON \
> %endif
>
> I tried replacing it with
>
> -DARROW_BOOST_USE_SHARED=ON
>
> but it does not look like it is going to build
>
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
> error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> boost::filesystem::path_traits>' has no member named 'make_preferred'
> i.make_preferred();
> ^~~~~~~~~~~~~~
> /root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
> error: 'class boost::filesystem::basic_path<std::basic_string<char>,
> boost::filesystem::path_traits>' has no member named 'native'
> out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
>
> I remember we had a similar conflict with ProtocolBuffers. In that case,
> changing Arrow to use the system provided version did the trick.
>
> Thanks,
> Rares
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Dec 11, 2018 at 9:30 PM Wes McKinney <we...@gmail.com> wrote:
>
>> hi,
>>
>> Could you clarify what you mean by "regex calls"? Are you talking
>> about boost::regex, std::regex, something else? How did you link the
>> relevant libraries in each part of your application, and in the Arrow
>> + Parquet libraries
>>
>> 0.9.0 is over 1000 patches ago. I'd recommend that you try to upgrade
>>
>> $ git hist apache-arrow-0.9.0..master | wc -l
>> 1540
>>
>> - Wes
>> On Tue, Dec 11, 2018 at 10:58 PM Rares Vernica <rv...@gmail.com> wrote:
>> >
>> > Hello,
>> >
>> > We are using the C++ bindings of Arrow 0.9.0 on our system on CentOS.
>> Once
>> > we load the Arrow library, our regular regex calls (outside of Arrow)
>> > misbehave and trigger some unknown crashes. We are still trying to figure
>> > things out but I was wondering if there are any know issues regarding
>> regex
>> > and the C++ binding. Also, how can one turn on/off flags related to regex
>> > when compiling Arrow? We are still trying to isolate the crash.
>> >
>> > Thanks!
>> > Rares
>>
Re: C++ buildings and Regex issue
Posted by Rares Vernica <rv...@gmail.com>.
Wes,
Thanks! We do plan to upgrade, as soon as we put down the fire. We noticed
some API changes and we will have to get our code updated.
It looks like it is boost::regex. In our application we link dynamically
against a locally compiled Boost. For Arrow we noticed this for CentOS
https://github.com/apache/arrow-dist/blob/master/cpp-linux/yum/arrow.spec.in#L69
%if %{_centos_ver} == 6
-DARROW_BOOST_VENDORED=ON \
%endif
I tried replacing it with
-DARROW_BOOST_USE_SHARED=ON
but it does not look like it is going to build
/root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:204:7:
error: 'class boost::filesystem::basic_path<std::basic_string<char>,
boost::filesystem::path_traits>' has no member named 'make_preferred'
i.make_preferred();
^~~~~~~~~~~~~~
/root/rpmbuild/BUILD/apache-arrow-0.9.0/cpp/src/arrow/io/hdfs-internal.cc:205:27:
error: 'class boost::filesystem::basic_path<std::basic_string<char>,
boost::filesystem::path_traits>' has no member named 'native'
out_handle = dlopen(i.native().c_str(), RTLD_NOW | RTLD_LOCAL);
I remember we had a similar conflict with ProtocolBuffers. In that case,
changing Arrow to use the system provided version did the trick.
Thanks,
Rares
On Tue, Dec 11, 2018 at 9:30 PM Wes McKinney <we...@gmail.com> wrote:
> hi,
>
> Could you clarify what you mean by "regex calls"? Are you talking
> about boost::regex, std::regex, something else? How did you link the
> relevant libraries in each part of your application, and in the Arrow
> + Parquet libraries
>
> 0.9.0 is over 1000 patches ago. I'd recommend that you try to upgrade
>
> $ git hist apache-arrow-0.9.0..master | wc -l
> 1540
>
> - Wes
> On Tue, Dec 11, 2018 at 10:58 PM Rares Vernica <rv...@gmail.com> wrote:
> >
> > Hello,
> >
> > We are using the C++ bindings of Arrow 0.9.0 on our system on CentOS.
> Once
> > we load the Arrow library, our regular regex calls (outside of Arrow)
> > misbehave and trigger some unknown crashes. We are still trying to figure
> > things out but I was wondering if there are any know issues regarding
> regex
> > and the C++ binding. Also, how can one turn on/off flags related to regex
> > when compiling Arrow? We are still trying to isolate the crash.
> >
> > Thanks!
> > Rares
>
Re: C++ buildings and Regex issue
Posted by Wes McKinney <we...@gmail.com>.
hi,
Could you clarify what you mean by "regex calls"? Are you talking
about boost::regex, std::regex, something else? How did you link the
relevant libraries in each part of your application, and in the Arrow
+ Parquet libraries
0.9.0 is over 1000 patches ago. I'd recommend that you try to upgrade
$ git hist apache-arrow-0.9.0..master | wc -l
1540
- Wes
On Tue, Dec 11, 2018 at 10:58 PM Rares Vernica <rv...@gmail.com> wrote:
>
> Hello,
>
> We are using the C++ bindings of Arrow 0.9.0 on our system on CentOS. Once
> we load the Arrow library, our regular regex calls (outside of Arrow)
> misbehave and trigger some unknown crashes. We are still trying to figure
> things out but I was wondering if there are any know issues regarding regex
> and the C++ binding. Also, how can one turn on/off flags related to regex
> when compiling Arrow? We are still trying to isolate the crash.
>
> Thanks!
> Rares