Re: LZ4 support in Windows conda (py)arrow

[Joris Peeters <jo...@gmail.com>]

I created https://github.com/AnacondaRecipes/arrow-cpp-feedstock/issues/3
back then in June, but not much is happening there. The absence of LZ4 in
Arrow's conda default channel is biting us frequently.
Is there a contact inside Anaconda I could flag this to? Should be an easy
and valuable fix.

On Fri, Jun 11, 2021 at 10:08 AM Joris Van den Bossche <
jorisvandenbossche@gmail.com> wrote:

> The conda packages from the default channel are indeed managed by
> Anaconda (Continuum Analytics is their old name). For arrow-cpp, the
> reciple is here:
> https://github.com/AnacondaRecipes/arrow-cpp-feedstock
>
> And indeed, looking at the windows build script, LZ4 is not enabled:
>
> https://github.com/AnacondaRecipes/arrow-cpp-feedstock/blob/master/recipe/bld.bat
>
> You can open an issue there to try to reach out to them.
>
> Joris
>
> On Fri, 11 Jun 2021 at 10:38, Joris Peeters <jo...@gmail.com>
> wrote:
> >
> > In the matrix of pyarrow artifacts for [conda, conda-forge] x [linux,
> windows], it seems all of them support LZ4, except (conda, windows). Is
> there a known reason for this?
> >
> > As I understand it, the conda Arrow builds are managed externally (by
> Continuum?), so perhaps should get in touch with them - as might not have
> `-DARROW_WITH_LZ4=ON`?
> >
> > There errors at run-time look like,
> >
> > ...
> >     pa.CompressedInputStream(
> >   File "pyarrow\io.pxi", line 1208, in
> pyarrow.lib.CompressedInputStream.__init__
> >   File "pyarrow\io.pxi", line 1579, in pyarrow.lib.Codec.__init__
> >   File "pyarrow\error.pxi", line 141, in
> pyarrow.lib.pyarrow_internal_check_status
> >   File "pyarrow\error.pxi", line 118, in pyarrow.lib.check_status
> > pyarrow.lib.ArrowNotImplementedError: Support for codec 'lz4' not built
> >
> > Cheers,
> > -Joris.
> >
>

Re: LZ4 support in Windows conda (py)arrow

[Joris Van den Bossche <jo...@gmail.com>]

I pinged them on the issue you created (with a hint on how to fix it, they
just forgot to enable a build flag AFAIK).

Unfortunately that issue tracker is also our main point of contact. I agree
this is very annoying, but I can only advise to use conda-forge instead or
try to get someone that is a customer of their enterprise version who might
have a more direct line bring it up.

Joris

On Wed, 1 Dec 2021 at 09:37, Joris Peeters <jo...@gmail.com>
wrote:

> I created https://github.com/AnacondaRecipes/arrow-cpp-feedstock/issues/3
> back then in June, but not much is happening there. The absence of LZ4 in
> Arrow's conda default channel is biting us frequently.
> Is there a contact inside Anaconda I could flag this to? Should be an easy
> and valuable fix.
>
> On Fri, Jun 11, 2021 at 10:08 AM Joris Van den Bossche <
> jorisvandenbossche@gmail.com> wrote:
>
>> The conda packages from the default channel are indeed managed by
>> Anaconda (Continuum Analytics is their old name). For arrow-cpp, the
>> reciple is here:
>> https://github.com/AnacondaRecipes/arrow-cpp-feedstock
>>
>> And indeed, looking at the windows build script, LZ4 is not enabled:
>>
>> https://github.com/AnacondaRecipes/arrow-cpp-feedstock/blob/master/recipe/bld.bat
>>
>> You can open an issue there to try to reach out to them.
>>
>> Joris
>>
>> On Fri, 11 Jun 2021 at 10:38, Joris Peeters <jo...@gmail.com>
>> wrote:
>> >
>> > In the matrix of pyarrow artifacts for [conda, conda-forge] x [linux,
>> windows], it seems all of them support LZ4, except (conda, windows). Is
>> there a known reason for this?
>> >
>> > As I understand it, the conda Arrow builds are managed externally (by
>> Continuum?), so perhaps should get in touch with them - as might not have
>> `-DARROW_WITH_LZ4=ON`?
>> >
>> > There errors at run-time look like,
>> >
>> > ...
>> >     pa.CompressedInputStream(
>> >   File "pyarrow\io.pxi", line 1208, in
>> pyarrow.lib.CompressedInputStream.__init__
>> >   File "pyarrow\io.pxi", line 1579, in pyarrow.lib.Codec.__init__
>> >   File "pyarrow\error.pxi", line 141, in
>> pyarrow.lib.pyarrow_internal_check_status
>> >   File "pyarrow\error.pxi", line 118, in pyarrow.lib.check_status
>> > pyarrow.lib.ArrowNotImplementedError: Support for codec 'lz4' not built
>> >
>> > Cheers,
>> > -Joris.
>> >
>>
>