You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by "Teodorescu, Radu (WorldQuant)" <Ra...@worldquant.com> on 2021/11/12 03:58:00 UTC
RE: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static
lib
It builds fine if I add all the missing libraries manually but I haven't been able to test it with GCS enabled (that flag breaks for me for independent reasons)
-----Original Message-----
From: David Li <li...@apache.org>
Sent: Thursday, November 11, 2021 7:35 PM
To: dev@arrow.apache.org
Subject: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
I don't believe this was intentional. IIRC, we refactored how we deal with Abseil in order to support integrating the Google Cloud SDK (since both it and gRPC use Abseil) and that (or perhaps some other change?) might have changed which libraries we include. Is it workable to explicitly link Abseil into your final application?
In general it seems we have some work to do around dependencies, in both the Flight SQL C++ PR [1] and this GitHub issue [2] we've seen issues around linking gRPC and Protobuf as well (where even if all our tests build and pass, it may not be easy to consume the artifacts once installed).
[1]: https://github.com/apache/arrow/pull/11507
[2]: https://github.com/apache/arrow/issues/11657
-David
On Thu, Nov 11, 2021, at 18:50, Teodorescu, Radu (WorldQuant) wrote:
> Hi,
> We are building with arrow as a CMake external project and using arrow_bundled lib for the third party dependencies.
> As of 6.0 we noticed a number of link errors and I tracked it down to
> a number of absl libraries no longer being included (5.0 has 21 absl libraries in the bundle while 6.0 only 7) In particular absl::strings_internal missing is giving us missing symbol for absl::lts_20210324::str_format_internal::FormatPack.
> Is this a bug or a change in the inclusion process coming with 6.0
> Thank you Radu
>
>
> ######################################################################
> #############
>
> The information contained in this communication is confidential, may
> be
>
> subject to legal privilege, and is intended only for the individual named.
>
> If you are not the named addressee, please notify the sender
> immediately and
>
> delete this email from your system. The views expressed in this email
> are
>
> the views of the sender only. Outgoing and incoming electronic
> communications
>
> to this address are electronically archived and subject to review
> and/or disclosure
>
> to someone other than the recipient.
>
> ######################################################################
> #############
>
###################################################################################
The information contained in this communication is confidential, may be
subject to legal privilege, and is intended only for the individual named.
If you are not the named addressee, please notify the sender immediately and
delete this email from your system. The views expressed in this email are
the views of the sender only. Outgoing and incoming electronic communications
to this address are electronically archived and subject to review and/or disclosure
to someone other than the recipient.
###################################################################################
RE: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
Posted by "Teodorescu, Radu (WorldQuant)" <Ra...@worldquant.com>.
Hi Rok - sorry I missed this earlier. Let us try it and get back to you.
Radu
-----Original Message-----
From: Rok Mihevc <ro...@gmail.com>
Sent: Wednesday, December 8, 2021 6:03 AM
To: dev@arrow.apache.org
Subject: Re: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
Hi Radu,
Could you please check if this PR solves the linking issue?
https://github.com/apache/arrow/pull/11889
I'm able to compile David's example
(https://issues.apache.org/jira/browse/ARROW-14708) locally.
Rok
On Mon, Nov 15, 2021 at 7:46 PM Teodorescu, Radu (WorldQuant) <Ra...@worldquant.com> wrote:
>
> Thank you!
>
> -----Original Message-----
> From: David Li <li...@apache.org>
> Sent: Saturday, November 13, 2021 2:56 PM
> To: dev@arrow.apache.org
> Subject: Re: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow
> bundled static lib
>
> Hi Radu,
>
> I can reproduce, I filed ARROW-14708 with some observations:
> https://issues.apache.org/jira/browse/ARROW-14708
>
> -David
>
> On Thu, Nov 11, 2021, at 22:58, Teodorescu, Radu (WorldQuant) wrote:
> > It builds fine if I add all the missing libraries manually but I
> > haven't been able to test it with GCS enabled (that flag breaks for
> > me for independent reasons)
> >
> > -----Original Message-----
> > From: David Li <li...@apache.org>
> > Sent: Thursday, November 11, 2021 7:35 PM
> > To: dev@arrow.apache.org
> > Subject: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled
> > static lib
> >
> > I don't believe this was intentional. IIRC, we refactored how we deal with Abseil in order to support integrating the Google Cloud SDK (since both it and gRPC use Abseil) and that (or perhaps some other change?) might have changed which libraries we include. Is it workable to explicitly link Abseil into your final application?
> >
> > In general it seems we have some work to do around dependencies, in both the Flight SQL C++ PR [1] and this GitHub issue [2] we've seen issues around linking gRPC and Protobuf as well (where even if all our tests build and pass, it may not be easy to consume the artifacts once installed).
> >
> > [1]: https://github.com/apache/arrow/pull/11507
> > [2]: https://github.com/apache/arrow/issues/11657
> >
> > -David
> >
> > On Thu, Nov 11, 2021, at 18:50, Teodorescu, Radu (WorldQuant) wrote:
> > > Hi,
> > > We are building with arrow as a CMake external project and using arrow_bundled lib for the third party dependencies.
> > > As of 6.0 we noticed a number of link errors and I tracked it down
> > > to a number of absl libraries no longer being included (5.0 has 21 absl libraries in the bundle while 6.0 only 7) In particular absl::strings_internal missing is giving us missing symbol for absl::lts_20210324::str_format_internal::FormatPack.
> > > Is this a bug or a change in the inclusion process coming with 6.0
> > > Thank you Radu
> > >
> > >
> > > ##################################################################
> > > ##
> > > ##
> > > #############
> > >
> > > The information contained in this communication is confidential,
> > > may be
> > >
> > > subject to legal privilege, and is intended only for the individual named.
> > >
> > > If you are not the named addressee, please notify the sender
> > > immediately and
> > >
> > > delete this email from your system. The views expressed in this
> > > email are
> > >
> > > the views of the sender only. Outgoing and incoming electronic
> > > communications
> > >
> > > to this address are electronically archived and subject to review
> > > and/or disclosure
> > >
> > > to someone other than the recipient.
> > >
> > > ##################################################################
> > > ##
> > > ##
> > > #############
> > >
> >
> >
> > ####################################################################
> > ##
> > #############
> >
> > The information contained in this communication is confidential, may
> > be
> >
> > subject to legal privilege, and is intended only for the individual named.
> >
> > If you are not the named addressee, please notify the sender
> > immediately and
> >
> > delete this email from your system. The views expressed in this
> > email are
> >
> > the views of the sender only. Outgoing and incoming electronic
> > communications
> >
> > to this address are electronically archived and subject to review
> > and/or disclosure
> >
> > to someone other than the recipient.
> >
> > ####################################################################
> > ##
> > #############
> >
> >
>
>
> ######################################################################
> #############
>
> The information contained in this communication is confidential, may
> be
>
> subject to legal privilege, and is intended only for the individual named.
>
> If you are not the named addressee, please notify the sender
> immediately and
>
> delete this email from your system. The views expressed in this email
> are
>
> the views of the sender only. Outgoing and incoming electronic
> communications
>
> to this address are electronically archived and subject to review
> and/or disclosure
>
> to someone other than the recipient.
>
> ######################################################################
> #############
>
###################################################################################
The information contained in this communication is confidential, may be
subject to legal privilege, and is intended only for the individual named.
If you are not the named addressee, please notify the sender immediately and
delete this email from your system. The views expressed in this email are
the views of the sender only. Outgoing and incoming electronic communications
to this address are electronically archived and subject to review and/or disclosure
to someone other than the recipient.
###################################################################################
Re: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
Posted by Rok Mihevc <ro...@gmail.com>.
Hi Radu,
Could you please check if this PR solves the linking issue?
https://github.com/apache/arrow/pull/11889
I'm able to compile David's example
(https://issues.apache.org/jira/browse/ARROW-14708) locally.
Rok
On Mon, Nov 15, 2021 at 7:46 PM Teodorescu, Radu (WorldQuant)
<Ra...@worldquant.com> wrote:
>
> Thank you!
>
> -----Original Message-----
> From: David Li <li...@apache.org>
> Sent: Saturday, November 13, 2021 2:56 PM
> To: dev@arrow.apache.org
> Subject: Re: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
>
> Hi Radu,
>
> I can reproduce, I filed ARROW-14708 with some observations: https://issues.apache.org/jira/browse/ARROW-14708
>
> -David
>
> On Thu, Nov 11, 2021, at 22:58, Teodorescu, Radu (WorldQuant) wrote:
> > It builds fine if I add all the missing libraries manually but I
> > haven't been able to test it with GCS enabled (that flag breaks for me
> > for independent reasons)
> >
> > -----Original Message-----
> > From: David Li <li...@apache.org>
> > Sent: Thursday, November 11, 2021 7:35 PM
> > To: dev@arrow.apache.org
> > Subject: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled
> > static lib
> >
> > I don't believe this was intentional. IIRC, we refactored how we deal with Abseil in order to support integrating the Google Cloud SDK (since both it and gRPC use Abseil) and that (or perhaps some other change?) might have changed which libraries we include. Is it workable to explicitly link Abseil into your final application?
> >
> > In general it seems we have some work to do around dependencies, in both the Flight SQL C++ PR [1] and this GitHub issue [2] we've seen issues around linking gRPC and Protobuf as well (where even if all our tests build and pass, it may not be easy to consume the artifacts once installed).
> >
> > [1]: https://github.com/apache/arrow/pull/11507
> > [2]: https://github.com/apache/arrow/issues/11657
> >
> > -David
> >
> > On Thu, Nov 11, 2021, at 18:50, Teodorescu, Radu (WorldQuant) wrote:
> > > Hi,
> > > We are building with arrow as a CMake external project and using arrow_bundled lib for the third party dependencies.
> > > As of 6.0 we noticed a number of link errors and I tracked it down
> > > to a number of absl libraries no longer being included (5.0 has 21 absl libraries in the bundle while 6.0 only 7) In particular absl::strings_internal missing is giving us missing symbol for absl::lts_20210324::str_format_internal::FormatPack.
> > > Is this a bug or a change in the inclusion process coming with 6.0
> > > Thank you Radu
> > >
> > >
> > > ####################################################################
> > > ##
> > > #############
> > >
> > > The information contained in this communication is confidential, may
> > > be
> > >
> > > subject to legal privilege, and is intended only for the individual named.
> > >
> > > If you are not the named addressee, please notify the sender
> > > immediately and
> > >
> > > delete this email from your system. The views expressed in this
> > > email are
> > >
> > > the views of the sender only. Outgoing and incoming electronic
> > > communications
> > >
> > > to this address are electronically archived and subject to review
> > > and/or disclosure
> > >
> > > to someone other than the recipient.
> > >
> > > ####################################################################
> > > ##
> > > #############
> > >
> >
> >
> > ######################################################################
> > #############
> >
> > The information contained in this communication is confidential, may
> > be
> >
> > subject to legal privilege, and is intended only for the individual named.
> >
> > If you are not the named addressee, please notify the sender
> > immediately and
> >
> > delete this email from your system. The views expressed in this email
> > are
> >
> > the views of the sender only. Outgoing and incoming electronic
> > communications
> >
> > to this address are electronically archived and subject to review
> > and/or disclosure
> >
> > to someone other than the recipient.
> >
> > ######################################################################
> > #############
> >
> >
>
>
> ###################################################################################
>
> The information contained in this communication is confidential, may be
>
> subject to legal privilege, and is intended only for the individual named.
>
> If you are not the named addressee, please notify the sender immediately and
>
> delete this email from your system. The views expressed in this email are
>
> the views of the sender only. Outgoing and incoming electronic communications
>
> to this address are electronically archived and subject to review and/or disclosure
>
> to someone other than the recipient.
>
> ###################################################################################
>
RE: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
Posted by "Teodorescu, Radu (WorldQuant)" <Ra...@worldquant.com>.
Thank you!
-----Original Message-----
From: David Li <li...@apache.org>
Sent: Saturday, November 13, 2021 2:56 PM
To: dev@arrow.apache.org
Subject: Re: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
Hi Radu,
I can reproduce, I filed ARROW-14708 with some observations: https://issues.apache.org/jira/browse/ARROW-14708
-David
On Thu, Nov 11, 2021, at 22:58, Teodorescu, Radu (WorldQuant) wrote:
> It builds fine if I add all the missing libraries manually but I
> haven't been able to test it with GCS enabled (that flag breaks for me
> for independent reasons)
>
> -----Original Message-----
> From: David Li <li...@apache.org>
> Sent: Thursday, November 11, 2021 7:35 PM
> To: dev@arrow.apache.org
> Subject: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled
> static lib
>
> I don't believe this was intentional. IIRC, we refactored how we deal with Abseil in order to support integrating the Google Cloud SDK (since both it and gRPC use Abseil) and that (or perhaps some other change?) might have changed which libraries we include. Is it workable to explicitly link Abseil into your final application?
>
> In general it seems we have some work to do around dependencies, in both the Flight SQL C++ PR [1] and this GitHub issue [2] we've seen issues around linking gRPC and Protobuf as well (where even if all our tests build and pass, it may not be easy to consume the artifacts once installed).
>
> [1]: https://github.com/apache/arrow/pull/11507
> [2]: https://github.com/apache/arrow/issues/11657
>
> -David
>
> On Thu, Nov 11, 2021, at 18:50, Teodorescu, Radu (WorldQuant) wrote:
> > Hi,
> > We are building with arrow as a CMake external project and using arrow_bundled lib for the third party dependencies.
> > As of 6.0 we noticed a number of link errors and I tracked it down
> > to a number of absl libraries no longer being included (5.0 has 21 absl libraries in the bundle while 6.0 only 7) In particular absl::strings_internal missing is giving us missing symbol for absl::lts_20210324::str_format_internal::FormatPack.
> > Is this a bug or a change in the inclusion process coming with 6.0
> > Thank you Radu
> >
> >
> > ####################################################################
> > ##
> > #############
> >
> > The information contained in this communication is confidential, may
> > be
> >
> > subject to legal privilege, and is intended only for the individual named.
> >
> > If you are not the named addressee, please notify the sender
> > immediately and
> >
> > delete this email from your system. The views expressed in this
> > email are
> >
> > the views of the sender only. Outgoing and incoming electronic
> > communications
> >
> > to this address are electronically archived and subject to review
> > and/or disclosure
> >
> > to someone other than the recipient.
> >
> > ####################################################################
> > ##
> > #############
> >
>
>
> ######################################################################
> #############
>
> The information contained in this communication is confidential, may
> be
>
> subject to legal privilege, and is intended only for the individual named.
>
> If you are not the named addressee, please notify the sender
> immediately and
>
> delete this email from your system. The views expressed in this email
> are
>
> the views of the sender only. Outgoing and incoming electronic
> communications
>
> to this address are electronically archived and subject to review
> and/or disclosure
>
> to someone other than the recipient.
>
> ######################################################################
> #############
>
>
###################################################################################
The information contained in this communication is confidential, may be
subject to legal privilege, and is intended only for the individual named.
If you are not the named addressee, please notify the sender immediately and
delete this email from your system. The views expressed in this email are
the views of the sender only. Outgoing and incoming electronic communications
to this address are electronically archived and subject to review and/or disclosure
to someone other than the recipient.
###################################################################################
Re: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
Posted by David Li <li...@apache.org>.
Hi Radu,
I can reproduce, I filed ARROW-14708 with some observations: https://issues.apache.org/jira/browse/ARROW-14708
-David
On Thu, Nov 11, 2021, at 22:58, Teodorescu, Radu (WorldQuant) wrote:
> It builds fine if I add all the missing libraries manually but I haven't been able to test it with GCS enabled (that flag breaks for me for independent reasons)
>
> -----Original Message-----
> From: David Li <li...@apache.org>
> Sent: Thursday, November 11, 2021 7:35 PM
> To: dev@arrow.apache.org
> Subject: [EXTERNAL] Re: Missing absl symbols from 6.0 arrow bundled static lib
>
> I don't believe this was intentional. IIRC, we refactored how we deal with Abseil in order to support integrating the Google Cloud SDK (since both it and gRPC use Abseil) and that (or perhaps some other change?) might have changed which libraries we include. Is it workable to explicitly link Abseil into your final application?
>
> In general it seems we have some work to do around dependencies, in both the Flight SQL C++ PR [1] and this GitHub issue [2] we've seen issues around linking gRPC and Protobuf as well (where even if all our tests build and pass, it may not be easy to consume the artifacts once installed).
>
> [1]: https://github.com/apache/arrow/pull/11507
> [2]: https://github.com/apache/arrow/issues/11657
>
> -David
>
> On Thu, Nov 11, 2021, at 18:50, Teodorescu, Radu (WorldQuant) wrote:
> > Hi,
> > We are building with arrow as a CMake external project and using arrow_bundled lib for the third party dependencies.
> > As of 6.0 we noticed a number of link errors and I tracked it down to
> > a number of absl libraries no longer being included (5.0 has 21 absl libraries in the bundle while 6.0 only 7) In particular absl::strings_internal missing is giving us missing symbol for absl::lts_20210324::str_format_internal::FormatPack.
> > Is this a bug or a change in the inclusion process coming with 6.0
> > Thank you Radu
> >
> >
> > ######################################################################
> > #############
> >
> > The information contained in this communication is confidential, may
> > be
> >
> > subject to legal privilege, and is intended only for the individual named.
> >
> > If you are not the named addressee, please notify the sender
> > immediately and
> >
> > delete this email from your system. The views expressed in this email
> > are
> >
> > the views of the sender only. Outgoing and incoming electronic
> > communications
> >
> > to this address are electronically archived and subject to review
> > and/or disclosure
> >
> > to someone other than the recipient.
> >
> > ######################################################################
> > #############
> >
>
>
> ###################################################################################
>
> The information contained in this communication is confidential, may be
>
> subject to legal privilege, and is intended only for the individual named.
>
> If you are not the named addressee, please notify the sender immediately and
>
> delete this email from your system. The views expressed in this email are
>
> the views of the sender only. Outgoing and incoming electronic communications
>
> to this address are electronically archived and subject to review and/or disclosure
>
> to someone other than the recipient.
>
> ###################################################################################
>
>