You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Ian Joiner <ia...@gmail.com> on 2022/02/03 07:35:45 UTC

[Python] Parquet CMake issue

Hi,

In order to prevent problematic PRs from happening again I’m cleaning up my local env. 

Here is my cmake:

cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
      -DCMAKE_INSTALL_LIBDIR=lib \
      -DCMAKE_BUILD_TYPE=debug \
      -DARROW_WITH_BZ2=ON \
      -DARROW_WITH_ZLIB=ON \
      -DARROW_WITH_ZSTD=ON \
      -DARROW_WITH_LZ4=ON \
      -DARROW_WITH_SNAPPY=ON \
      -DARROW_WITH_BROTLI=ON \
      -DARROW_PARQUET=ON \
      -DARROW_PYTHON=ON \
      -DARROW_ORC=ON \
      -DARROW_BUILD_TESTS=ON \
      ..

After running make -j4 as in https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst I ran the C++ unit tests all of which passed. Parquet clearly exists.

After running make install I began to build pyarrow. However unless turn off Parquet I got a Parquet error.

CMake Error at /Users/karlkatzen/anaconda3/envs/pyarrow-dev/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find Parquet (missing: PARQUET_INCLUDE_DIR PARQUET_LIB_DIR
  PARQUET_SO_VERSION)
Call Stack (most recent call first):
  /Users/karlkatzen/anaconda3/envs/pyarrow-dev/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
  cmake_modules/FindParquet.cmake:115 (find_package_handle_standard_args)
  CMakeLists.txt:447 (find_package)

What’s the cause of this problem?

Ian




Re: [Python] Parquet CMake issue

Posted by Ian Joiner <ia...@gmail.com>.
Hi Antoine,

Yes I did. It was installed in the correct directory in the correct conda
env. Moreover make can definitely find the other libraries under the same
directory such as libarrow.a & libarrow.dylib but oddly not libparquet.a or
libparquet.dylib.

Ian

On Thursday, February 3, 2022, Antoine Pitrou <an...@python.org> wrote:

>
> Hi Ian,
>
> Did you run "make install" as well after compiling Arrow C++? Perhaps
> PyArrow is picking up an old installed version?
>
> Regards
>
> Antoine.
>
> Le 03/02/2022 à 08:35, Ian Joiner a écrit :
>
>> Hi,
>>
>> In order to prevent problematic PRs from happening again I’m cleaning up
>> my local env.
>>
>> Here is my cmake:
>>
>> cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
>>        -DCMAKE_INSTALL_LIBDIR=lib \
>>        -DCMAKE_BUILD_TYPE=debug \
>>        -DARROW_WITH_BZ2=ON \
>>        -DARROW_WITH_ZLIB=ON \
>>        -DARROW_WITH_ZSTD=ON \
>>        -DARROW_WITH_LZ4=ON \
>>        -DARROW_WITH_SNAPPY=ON \
>>        -DARROW_WITH_BROTLI=ON \
>>        -DARROW_PARQUET=ON \
>>        -DARROW_PYTHON=ON \
>>        -DARROW_ORC=ON \
>>        -DARROW_BUILD_TESTS=ON \
>>        ..
>>
>> After running make -j4 as in https://github.com/apache/arro
>> w/blob/master/docs/source/developers/python.rst I ran the C++ unit tests
>> all of which passed. Parquet clearly exists.
>>
>> After running make install I began to build pyarrow. However unless turn
>> off Parquet I got a Parquet error.
>>
>> CMake Error at /Users/karlkatzen/anaconda3/en
>> vs/pyarrow-dev/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:230
>> (message):
>>    Could NOT find Parquet (missing: PARQUET_INCLUDE_DIR PARQUET_LIB_DIR
>>    PARQUET_SO_VERSION)
>> Call Stack (most recent call first):
>>    /Users/karlkatzen/anaconda3/envs/pyarrow-dev/share/cmake-3.
>> 21/Modules/FindPackageHandleStandardArgs.cmake:594
>> (_FPHSA_FAILURE_MESSAGE)
>>    cmake_modules/FindParquet.cmake:115 (find_package_handle_standard_
>> args)
>>    CMakeLists.txt:447 (find_package)
>>
>> What’s the cause of this problem?
>>
>> Ian
>>
>>
>>
>>

Re: [Python] Parquet CMake issue

Posted by Antoine Pitrou <an...@python.org>.
Hi Ian,

Did you run "make install" as well after compiling Arrow C++? Perhaps 
PyArrow is picking up an old installed version?

Regards

Antoine.

Le 03/02/2022 à 08:35, Ian Joiner a écrit :
> Hi,
> 
> In order to prevent problematic PRs from happening again I’m cleaning up my local env.
> 
> Here is my cmake:
> 
> cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
>        -DCMAKE_INSTALL_LIBDIR=lib \
>        -DCMAKE_BUILD_TYPE=debug \
>        -DARROW_WITH_BZ2=ON \
>        -DARROW_WITH_ZLIB=ON \
>        -DARROW_WITH_ZSTD=ON \
>        -DARROW_WITH_LZ4=ON \
>        -DARROW_WITH_SNAPPY=ON \
>        -DARROW_WITH_BROTLI=ON \
>        -DARROW_PARQUET=ON \
>        -DARROW_PYTHON=ON \
>        -DARROW_ORC=ON \
>        -DARROW_BUILD_TESTS=ON \
>        ..
> 
> After running make -j4 as in https://github.com/apache/arrow/blob/master/docs/source/developers/python.rst I ran the C++ unit tests all of which passed. Parquet clearly exists.
> 
> After running make install I began to build pyarrow. However unless turn off Parquet I got a Parquet error.
> 
> CMake Error at /Users/karlkatzen/anaconda3/envs/pyarrow-dev/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
>    Could NOT find Parquet (missing: PARQUET_INCLUDE_DIR PARQUET_LIB_DIR
>    PARQUET_SO_VERSION)
> Call Stack (most recent call first):
>    /Users/karlkatzen/anaconda3/envs/pyarrow-dev/share/cmake-3.21/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
>    cmake_modules/FindParquet.cmake:115 (find_package_handle_standard_args)
>    CMakeLists.txt:447 (find_package)
> 
> What’s the cause of this problem?
> 
> Ian
> 
> 
>