You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Aneesh Karve <an...@quiltdata.io> on 2017/05/23 22:42:55 UTC
pyarrow=0.3.0 now segfaults on parquet.write_table
Greetings.
Pyarrow 0.3.0 was stable and working for our data package manager
<https://quiltdata.com>. As of today we're seeing a segfault on
parquet.write_table in CI (Linux and Mac). Here are our before
<https://travis-ci.org/quiltdata/quilt/builds/234903477> and after
<https://travis-ci.org/quiltdata/quilt/builds/235324971> CI results. Both
use `conda install --yes -c conda-forge pyarrow=0.3` and use pyarrow in the
same ways.
Conda does not seem to be holding any other version of 0.3 that I can see
with `conda list`.
Did any second order dependencies change? It would make sense if the 0.4
binaries were broken or missing but I'm a bit mystified as to how 0.3 broke
:-)
Thanks in advance. Looking forward to using Arrow as it evolves.
ᐧ
Re: pyarrow=0.3.0 now segfaults on parquet.write_table
Posted by Wes McKinney <we...@gmail.com>.
Also, I wasn't clear, but: I have no plans to make "conda install
pyarrow=0.3" work again. conda-forge is not a platform well suited to
maintaining multiple versions of things
On Tue, May 23, 2017 at 7:10 PM, Wes McKinney <we...@gmail.com> wrote:
> hi Aneesh,
>
> I've been updating the conda packages for arrow-cpp, parquet-cpp, and
> pyarrow over the last 24 hours, so I suspect you've had bad luck with
> the package stack getting into an intermediate state. There was a
> build dependency problem on Windows that caused the delay.
>
> conda install pyarrow=0.4.0 is now fully up to date on
> Linux/macOS/Windows, so I recommend using that ASAP as it includes
> important bug fixes.
>
> If you are depending on these packages in your product, I would
> _really_ appreciate some help maintaining the packages on conda-forge.
> We probably need to do some work on the pinned versions in the
> meta.yaml files so that the upgrade process does not disrupt existing
> production applications.
>
> Thanks
> Wes
>
> On Tue, May 23, 2017 at 6:42 PM, Aneesh Karve <an...@quiltdata.io> wrote:
>> Greetings.
>>
>> Pyarrow 0.3.0 was stable and working for our data package manager
>> <https://quiltdata.com>. As of today we're seeing a segfault on
>> parquet.write_table in CI (Linux and Mac). Here are our before
>> <https://travis-ci.org/quiltdata/quilt/builds/234903477> and after
>> <https://travis-ci.org/quiltdata/quilt/builds/235324971> CI results. Both
>> use `conda install --yes -c conda-forge pyarrow=0.3` and use pyarrow in the
>> same ways.
>>
>> Conda does not seem to be holding any other version of 0.3 that I can see
>> with `conda list`.
>>
>> Did any second order dependencies change? It would make sense if the 0.4
>> binaries were broken or missing but I'm a bit mystified as to how 0.3 broke
>> :-)
>>
>> Thanks in advance. Looking forward to using Arrow as it evolves.
>> ᐧ
Re: pyarrow=0.3.0 now segfaults on parquet.write_table
Posted by Wes McKinney <we...@gmail.com>.
It's fine to discuss offline and bring back any discussions to dev@
that concern the project development itself (versus purely external
packaging issues; the conda-forge packages are unofficial packages and
not artifacts provided by the Arrow project). The problem with the
0.3.0 package metadata was that the parquet-cpp version was not pinned
in the right way. If you make pull requests into the feedstocks, we
can add you as a maintainer so that you can merge updates.
Thanks
On Tue, May 23, 2017 at 9:55 PM, Aneesh Karve <an...@quiltdata.io> wrote:
> OK. We are up and running with 0.4.0. (I noted some package propagation
> issues; even after the forge page <https://anaconda.org/conda-forge/pyarrow>
> showed 0.4.0 it took >30 min. before I could install it.)
>
> Happy to help with the conda feedstock. Feel free to email me so we can
> discuss details if dev is not the right place.
> ᐧ
>
> On Tue, May 23, 2017 at 4:10 PM, Wes McKinney <we...@gmail.com> wrote:
>
>> hi Aneesh,
>>
>> I've been updating the conda packages for arrow-cpp, parquet-cpp, and
>> pyarrow over the last 24 hours, so I suspect you've had bad luck with
>> the package stack getting into an intermediate state. There was a
>> build dependency problem on Windows that caused the delay.
>>
>> conda install pyarrow=0.4.0 is now fully up to date on
>> Linux/macOS/Windows, so I recommend using that ASAP as it includes
>> important bug fixes.
>>
>> If you are depending on these packages in your product, I would
>> _really_ appreciate some help maintaining the packages on conda-forge.
>> We probably need to do some work on the pinned versions in the
>> meta.yaml files so that the upgrade process does not disrupt existing
>> production applications.
>>
>> Thanks
>> Wes
>>
>> On Tue, May 23, 2017 at 6:42 PM, Aneesh Karve <an...@quiltdata.io> wrote:
>> > Greetings.
>> >
>> > Pyarrow 0.3.0 was stable and working for our data package manager
>> > <https://quiltdata.com>. As of today we're seeing a segfault on
>> > parquet.write_table in CI (Linux and Mac). Here are our before
>> > <https://travis-ci.org/quiltdata/quilt/builds/234903477> and after
>> > <https://travis-ci.org/quiltdata/quilt/builds/235324971> CI results.
>> Both
>> > use `conda install --yes -c conda-forge pyarrow=0.3` and use pyarrow in
>> the
>> > same ways.
>> >
>> > Conda does not seem to be holding any other version of 0.3 that I can see
>> > with `conda list`.
>> >
>> > Did any second order dependencies change? It would make sense if the 0.4
>> > binaries were broken or missing but I'm a bit mystified as to how 0.3
>> broke
>> > :-)
>> >
>> > Thanks in advance. Looking forward to using Arrow as it evolves.
>> > ᐧ
>>
>
>
>
> --
>
>
> Aneesh Karve | 415-323-0193 | LinkedIn <http://linkedin.com/in/aneeshkarve> |
> Twitter <https://twitter.com/akarve>
>
> <https://quiltdata.com>
> quiltdata.com | Manage data like code
> <https://blog.quiltdata.com/its-time-to-manage-data-like-source-code-3df04cd312b8>
Re: pyarrow=0.3.0 now segfaults on parquet.write_table
Posted by Aneesh Karve <an...@quiltdata.io>.
OK. We are up and running with 0.4.0. (I noted some package propagation
issues; even after the forge page <https://anaconda.org/conda-forge/pyarrow>
showed 0.4.0 it took >30 min. before I could install it.)
Happy to help with the conda feedstock. Feel free to email me so we can
discuss details if dev is not the right place.
ᐧ
On Tue, May 23, 2017 at 4:10 PM, Wes McKinney <we...@gmail.com> wrote:
> hi Aneesh,
>
> I've been updating the conda packages for arrow-cpp, parquet-cpp, and
> pyarrow over the last 24 hours, so I suspect you've had bad luck with
> the package stack getting into an intermediate state. There was a
> build dependency problem on Windows that caused the delay.
>
> conda install pyarrow=0.4.0 is now fully up to date on
> Linux/macOS/Windows, so I recommend using that ASAP as it includes
> important bug fixes.
>
> If you are depending on these packages in your product, I would
> _really_ appreciate some help maintaining the packages on conda-forge.
> We probably need to do some work on the pinned versions in the
> meta.yaml files so that the upgrade process does not disrupt existing
> production applications.
>
> Thanks
> Wes
>
> On Tue, May 23, 2017 at 6:42 PM, Aneesh Karve <an...@quiltdata.io> wrote:
> > Greetings.
> >
> > Pyarrow 0.3.0 was stable and working for our data package manager
> > <https://quiltdata.com>. As of today we're seeing a segfault on
> > parquet.write_table in CI (Linux and Mac). Here are our before
> > <https://travis-ci.org/quiltdata/quilt/builds/234903477> and after
> > <https://travis-ci.org/quiltdata/quilt/builds/235324971> CI results.
> Both
> > use `conda install --yes -c conda-forge pyarrow=0.3` and use pyarrow in
> the
> > same ways.
> >
> > Conda does not seem to be holding any other version of 0.3 that I can see
> > with `conda list`.
> >
> > Did any second order dependencies change? It would make sense if the 0.4
> > binaries were broken or missing but I'm a bit mystified as to how 0.3
> broke
> > :-)
> >
> > Thanks in advance. Looking forward to using Arrow as it evolves.
> > ᐧ
>
--
Aneesh Karve | 415-323-0193 | LinkedIn <http://linkedin.com/in/aneeshkarve> |
Twitter <https://twitter.com/akarve>
<https://quiltdata.com>
quiltdata.com | Manage data like code
<https://blog.quiltdata.com/its-time-to-manage-data-like-source-code-3df04cd312b8>
Re: pyarrow=0.3.0 now segfaults on parquet.write_table
Posted by Wes McKinney <we...@gmail.com>.
hi Aneesh,
I've been updating the conda packages for arrow-cpp, parquet-cpp, and
pyarrow over the last 24 hours, so I suspect you've had bad luck with
the package stack getting into an intermediate state. There was a
build dependency problem on Windows that caused the delay.
conda install pyarrow=0.4.0 is now fully up to date on
Linux/macOS/Windows, so I recommend using that ASAP as it includes
important bug fixes.
If you are depending on these packages in your product, I would
_really_ appreciate some help maintaining the packages on conda-forge.
We probably need to do some work on the pinned versions in the
meta.yaml files so that the upgrade process does not disrupt existing
production applications.
Thanks
Wes
On Tue, May 23, 2017 at 6:42 PM, Aneesh Karve <an...@quiltdata.io> wrote:
> Greetings.
>
> Pyarrow 0.3.0 was stable and working for our data package manager
> <https://quiltdata.com>. As of today we're seeing a segfault on
> parquet.write_table in CI (Linux and Mac). Here are our before
> <https://travis-ci.org/quiltdata/quilt/builds/234903477> and after
> <https://travis-ci.org/quiltdata/quilt/builds/235324971> CI results. Both
> use `conda install --yes -c conda-forge pyarrow=0.3` and use pyarrow in the
> same ways.
>
> Conda does not seem to be holding any other version of 0.3 that I can see
> with `conda list`.
>
> Did any second order dependencies change? It would make sense if the 0.4
> binaries were broken or missing but I'm a bit mystified as to how 0.3 broke
> :-)
>
> Thanks in advance. Looking forward to using Arrow as it evolves.
> ᐧ