You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by shyam narayan singh <sh...@gmail.com> on 2019/01/17 16:44:49 UTC
Export symbol guidelines
Hi
As part of the timestamp support <https://github.com/apache/arrow/pull/3352>,
we vendored the Howard hinnant date project.
I had to export the compiled symbols. But travis job for "manylinux" is
failing for it is expecting only two symbols to be exported. Based on the
logs it looks like it expects all the symbols to be part of arrow namespace.
What is the expectation with the vendored code? Any guide lines?
Any help appreciated.
Regards
Re: Export symbol guidelines
Posted by Wes McKinney <we...@gmail.com>.
It is probably safer to namespace those symbols
On Thu, Jan 17, 2019 at 11:51 AM shyam narayan singh
<sh...@gmail.com> wrote:
>
> Vendored code is part of the arrow. Gandiva is just the caller of the
> timezone apis.
>
> Regards
> Shyam
>
> On Thu, Jan 17, 2019 at 11:06 PM Antoine Pitrou <an...@python.org> wrote:
>
> >
> > Side note: why do those symbols appear in libarrow.so and not
> > libgandiva.so?
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 17/01/2019 à 18:35, Antoine Pitrou a écrit :
> > >
> > > Le 17/01/2019 à 18:29, shyam narayan singh a écrit :
> > >> Hi
> > >>
> > >> 1. The symbols are "locate_zone" and "to_sys" that are part of
> > >> cast_time.cc. These are invoked during casting timestamp with timezone
> > >> present.
> > >> 2. I am trying different things. I just made the symbols hidden to see
> > the
> > >> affect. "manylinux" passed while the others failed.
> > >> 3. Couple of approaches
> > >> a. relax the constraints
> > >> b.move the the vendored code to arrow/vendored namespace.
> > >
> > > Ideally we don't want to change the vendored code at all (except at the
> > > very beginning and end of the file). I'm not sure it's possible to
> > > implement b) under that constraint.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> >
Re: Export symbol guidelines
Posted by shyam narayan singh <sh...@gmail.com>.
Vendored code is part of the arrow. Gandiva is just the caller of the
timezone apis.
Regards
Shyam
On Thu, Jan 17, 2019 at 11:06 PM Antoine Pitrou <an...@python.org> wrote:
>
> Side note: why do those symbols appear in libarrow.so and not
> libgandiva.so?
>
> Regards
>
> Antoine.
>
>
> Le 17/01/2019 à 18:35, Antoine Pitrou a écrit :
> >
> > Le 17/01/2019 à 18:29, shyam narayan singh a écrit :
> >> Hi
> >>
> >> 1. The symbols are "locate_zone" and "to_sys" that are part of
> >> cast_time.cc. These are invoked during casting timestamp with timezone
> >> present.
> >> 2. I am trying different things. I just made the symbols hidden to see
> the
> >> affect. "manylinux" passed while the others failed.
> >> 3. Couple of approaches
> >> a. relax the constraints
> >> b.move the the vendored code to arrow/vendored namespace.
> >
> > Ideally we don't want to change the vendored code at all (except at the
> > very beginning and end of the file). I'm not sure it's possible to
> > implement b) under that constraint.
> >
> > Regards
> >
> > Antoine.
> >
>
Re: Export symbol guidelines
Posted by Antoine Pitrou <an...@python.org>.
Side note: why do those symbols appear in libarrow.so and not libgandiva.so?
Regards
Antoine.
Le 17/01/2019 à 18:35, Antoine Pitrou a écrit :
>
> Le 17/01/2019 à 18:29, shyam narayan singh a écrit :
>> Hi
>>
>> 1. The symbols are "locate_zone" and "to_sys" that are part of
>> cast_time.cc. These are invoked during casting timestamp with timezone
>> present.
>> 2. I am trying different things. I just made the symbols hidden to see the
>> affect. "manylinux" passed while the others failed.
>> 3. Couple of approaches
>> a. relax the constraints
>> b.move the the vendored code to arrow/vendored namespace.
>
> Ideally we don't want to change the vendored code at all (except at the
> very beginning and end of the file). I'm not sure it's possible to
> implement b) under that constraint.
>
> Regards
>
> Antoine.
>
Re: Export symbol guidelines
Posted by Antoine Pitrou <an...@python.org>.
Le 17/01/2019 à 18:29, shyam narayan singh a écrit :
> Hi
>
> 1. The symbols are "locate_zone" and "to_sys" that are part of
> cast_time.cc. These are invoked during casting timestamp with timezone
> present.
> 2. I am trying different things. I just made the symbols hidden to see the
> affect. "manylinux" passed while the others failed.
> 3. Couple of approaches
> a. relax the constraints
> b.move the the vendored code to arrow/vendored namespace.
Ideally we don't want to change the vendored code at all (except at the
very beginning and end of the file). I'm not sure it's possible to
implement b) under that constraint.
Regards
Antoine.
Re: Export symbol guidelines
Posted by shyam narayan singh <sh...@gmail.com>.
Hi
1. The symbols are "locate_zone" and "to_sys" that are part of
cast_time.cc. These are invoked during casting timestamp with timezone
present.
2. I am trying different things. I just made the symbols hidden to see the
affect. "manylinux" passed while the others failed.
3. Couple of approaches
a. relax the constraints
b.move the the vendored code to arrow/vendored namespace.
Regards
Shyam
On Thu, Jan 17, 2019 at 10:23 PM Antoine Pitrou <an...@python.org> wrote:
>
> Le 17/01/2019 à 17:44, shyam narayan singh a écrit :
> > Hi
> >
> > As part of the timestamp support <
> https://github.com/apache/arrow/pull/3352>,
> > we vendored the Howard hinnant date project.
> >
> > I had to export the compiled symbols.
>
> Can you elaborate on this? Is it because the symbols have to be seen at
> runtime by JIT-compiled code? What are those symbols and when are they
> invoked exactly?
>
> > But travis job for "manylinux" is
> > failing for it is expecting only two symbols to be exported. Based on the
> > logs it looks like it expects all the symbols to be part of arrow
> namespace.
>
> Where is the build failure? It seems that
> https://travis-ci.org/apache/arrow/builds/480928314 succeeded the
> manylinux build.
>
> > What is the expectation with the vendored code? Any guide lines?
>
> One possibility (to be discussed) would be to relax the checks in
> python/manylinux1/scripts/check_arrow_visibility.sh
>
> But perhaps we can find a better way.
>
> Regards
>
> Antoine.
>
Re: Export symbol guidelines
Posted by Antoine Pitrou <an...@python.org>.
Le 17/01/2019 à 17:44, shyam narayan singh a écrit :
> Hi
>
> As part of the timestamp support <https://github.com/apache/arrow/pull/3352>,
> we vendored the Howard hinnant date project.
>
> I had to export the compiled symbols.
Can you elaborate on this? Is it because the symbols have to be seen at
runtime by JIT-compiled code? What are those symbols and when are they
invoked exactly?
> But travis job for "manylinux" is
> failing for it is expecting only two symbols to be exported. Based on the
> logs it looks like it expects all the symbols to be part of arrow namespace.
Where is the build failure? It seems that
https://travis-ci.org/apache/arrow/builds/480928314 succeeded the
manylinux build.
> What is the expectation with the vendored code? Any guide lines?
One possibility (to be discussed) would be to relax the checks in
python/manylinux1/scripts/check_arrow_visibility.sh
But perhaps we can find a better way.
Regards
Antoine.