You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Yaron Gvili <rt...@hotmail.com> on 2022/09/11 13:57:53 UTC
PyArrow build problem
Hi All,
I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
Unknown CMake command "arrow_keep_backward_compatibility".
Call Stack (most recent call first):
/miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
/miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
CMakeLists.txt:123 (find_package)
Cheers,
Yaron.
Re: PyArrow build problem
Posted by Yaron Gvili <rt...@hotmail.com>.
After spending a bit more time on this, it looks like the problem may be related to the structure of PyArrow and its build system being changed since the previous time I synced. I found PyArrow C++ source has moved to `python/pyarrow/src` and has a separate `CMakeLists.txt` root. However, I couldn't get it to build nor find instructions for doing so, and I suspect the build problem I described earlier here would get fixed with this build.
Could anyone explain how the PyArrow build works now?
Cheers,
Yaron.
________________________________
From: Yaron Gvili <rt...@hotmail.com>
Sent: Sunday, September 11, 2022 9:57 AM
To: dev@arrow.apache.org <de...@arrow.apache.org>
Subject: PyArrow build problem
Hi All,
I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
Unknown CMake command "arrow_keep_backward_compatibility".
Call Stack (most recent call first):
/miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
/miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
CMakeLists.txt:123 (find_package)
Cheers,
Yaron.
Re: PyArrow build problem
Posted by Jeszy <je...@gmail.com>.
Hey,
Sorry for the confusion, VESA is an internal system. We're working
with Yaron to resolve this over a tighter loop, and will circle back
with the outcome.
Balazs
On Tue, Sep 13, 2022 at 8:48 AM Yaron Gvili <rt...@hotmail.com> wrote:
>
> OK, I posted in VESA-62.
>
>
> Yaron.
> ________________________________
> From: Sutou Kouhei <ko...@clear-code.com>
> Sent: Monday, September 12, 2022 8:26 PM
> To: dev@arrow.apache.org <de...@arrow.apache.org>
> Subject: Re: PyArrow build problem
>
> Hi,
>
> Ah, this mailing list doesn't accept attachments.
> Could you upload them to somewhere such as gist and share
> the URL? Or you can open a Jira issue and continue this
> because we can attach files on Jira.
>
> Thanks,
> --
> kou
>
> In
> <PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
> "Re: PyArrow build problem" on Mon, 12 Sep 2022 19:36:51 +0000,
> Yaron Gvili <rt...@hotmail.com> wrote:
>
> > Hi Kou,
> >
> > I'm attaching the cmake log files for the same command with both PYARROW_CMAKE_OPTIONS and CMAKE_PREFIX_PATH set and exported as requested.
> >
> >
> > Thanks,
> > Yaron.
> > ________________________________
> > From: Sutou Kouhei <ko...@clear-code.com>
> > Sent: Sunday, September 11, 2022 6:03 PM
> > To: dev@arrow.apache.org <de...@arrow.apache.org>
> > Subject: Re: PyArrow build problem
> >
> > Hi,
> >
> > It seems that
> > /miniconda3/envs/pyarrow-dev/lib/cmake/Arrow/ArrowConfig.cmake
> > isn't used. Could you share a build log with
> > PYARROW_CMAKE_OPTIONS="-DCMAKE_FIND_DEBUG_MODE=ON" environment
> > variable?
> >
> > You may need to export
> > CMAKE_PREFIX_PATH=/miniconda3/envs/pyarrow-dev environment
> > variable but the path should be included in
> > the CMAKE_SYSTEM_PREFIX_PATH CMake variable via the CONDA_PREFIX
> > environment variable...:
> > https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_PREFIX_PATH.html
> >
> >
> > Thanks,
> > --
> > kou
> >
> > In
> > <PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
> > "PyArrow build problem" on Sun, 11 Sep 2022 13:57:53 +0000,
> > Yaron Gvili <rt...@hotmail.com> wrote:
> >
> >> Hi All,
> >>
> >> I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
> >>
> >> CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
> >> Unknown CMake command "arrow_keep_backward_compatibility".
> >> Call Stack (most recent call first):
> >> /miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
> >> /miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
> >> CMakeLists.txt:123 (find_package)
> >>
> >>
> >> Cheers,
> >> Yaron.
Re: PyArrow build problem
Posted by Yaron Gvili <rt...@hotmail.com>.
OK, I posted in VESA-62.
Yaron.
________________________________
From: Sutou Kouhei <ko...@clear-code.com>
Sent: Monday, September 12, 2022 8:26 PM
To: dev@arrow.apache.org <de...@arrow.apache.org>
Subject: Re: PyArrow build problem
Hi,
Ah, this mailing list doesn't accept attachments.
Could you upload them to somewhere such as gist and share
the URL? Or you can open a Jira issue and continue this
because we can attach files on Jira.
Thanks,
--
kou
In
<PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
"Re: PyArrow build problem" on Mon, 12 Sep 2022 19:36:51 +0000,
Yaron Gvili <rt...@hotmail.com> wrote:
> Hi Kou,
>
> I'm attaching the cmake log files for the same command with both PYARROW_CMAKE_OPTIONS and CMAKE_PREFIX_PATH set and exported as requested.
>
>
> Thanks,
> Yaron.
> ________________________________
> From: Sutou Kouhei <ko...@clear-code.com>
> Sent: Sunday, September 11, 2022 6:03 PM
> To: dev@arrow.apache.org <de...@arrow.apache.org>
> Subject: Re: PyArrow build problem
>
> Hi,
>
> It seems that
> /miniconda3/envs/pyarrow-dev/lib/cmake/Arrow/ArrowConfig.cmake
> isn't used. Could you share a build log with
> PYARROW_CMAKE_OPTIONS="-DCMAKE_FIND_DEBUG_MODE=ON" environment
> variable?
>
> You may need to export
> CMAKE_PREFIX_PATH=/miniconda3/envs/pyarrow-dev environment
> variable but the path should be included in
> the CMAKE_SYSTEM_PREFIX_PATH CMake variable via the CONDA_PREFIX
> environment variable...:
> https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_PREFIX_PATH.html
>
>
> Thanks,
> --
> kou
>
> In
> <PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
> "PyArrow build problem" on Sun, 11 Sep 2022 13:57:53 +0000,
> Yaron Gvili <rt...@hotmail.com> wrote:
>
>> Hi All,
>>
>> I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
>>
>> CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
>> Unknown CMake command "arrow_keep_backward_compatibility".
>> Call Stack (most recent call first):
>> /miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
>> /miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
>> CMakeLists.txt:123 (find_package)
>>
>>
>> Cheers,
>> Yaron.
Re: PyArrow build problem
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
Ah, this mailing list doesn't accept attachments.
Could you upload them to somewhere such as gist and share
the URL? Or you can open a Jira issue and continue this
because we can attach files on Jira.
Thanks,
--
kou
In
<PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
"Re: PyArrow build problem" on Mon, 12 Sep 2022 19:36:51 +0000,
Yaron Gvili <rt...@hotmail.com> wrote:
> Hi Kou,
>
> I'm attaching the cmake log files for the same command with both PYARROW_CMAKE_OPTIONS and CMAKE_PREFIX_PATH set and exported as requested.
>
>
> Thanks,
> Yaron.
> ________________________________
> From: Sutou Kouhei <ko...@clear-code.com>
> Sent: Sunday, September 11, 2022 6:03 PM
> To: dev@arrow.apache.org <de...@arrow.apache.org>
> Subject: Re: PyArrow build problem
>
> Hi,
>
> It seems that
> /miniconda3/envs/pyarrow-dev/lib/cmake/Arrow/ArrowConfig.cmake
> isn't used. Could you share a build log with
> PYARROW_CMAKE_OPTIONS="-DCMAKE_FIND_DEBUG_MODE=ON" environment
> variable?
>
> You may need to export
> CMAKE_PREFIX_PATH=/miniconda3/envs/pyarrow-dev environment
> variable but the path should be included in
> the CMAKE_SYSTEM_PREFIX_PATH CMake variable via the CONDA_PREFIX
> environment variable...:
> https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_PREFIX_PATH.html
>
>
> Thanks,
> --
> kou
>
> In
> <PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
> "PyArrow build problem" on Sun, 11 Sep 2022 13:57:53 +0000,
> Yaron Gvili <rt...@hotmail.com> wrote:
>
>> Hi All,
>>
>> I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
>>
>> CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
>> Unknown CMake command "arrow_keep_backward_compatibility".
>> Call Stack (most recent call first):
>> /miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
>> /miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
>> CMakeLists.txt:123 (find_package)
>>
>>
>> Cheers,
>> Yaron.
Re: PyArrow build problem
Posted by Yaron Gvili <rt...@hotmail.com>.
Hi Kou,
I'm attaching the cmake log files for the same command with both PYARROW_CMAKE_OPTIONS and CMAKE_PREFIX_PATH set and exported as requested.
Thanks,
Yaron.
________________________________
From: Sutou Kouhei <ko...@clear-code.com>
Sent: Sunday, September 11, 2022 6:03 PM
To: dev@arrow.apache.org <de...@arrow.apache.org>
Subject: Re: PyArrow build problem
Hi,
It seems that
/miniconda3/envs/pyarrow-dev/lib/cmake/Arrow/ArrowConfig.cmake
isn't used. Could you share a build log with
PYARROW_CMAKE_OPTIONS="-DCMAKE_FIND_DEBUG_MODE=ON" environment
variable?
You may need to export
CMAKE_PREFIX_PATH=/miniconda3/envs/pyarrow-dev environment
variable but the path should be included in
the CMAKE_SYSTEM_PREFIX_PATH CMake variable via the CONDA_PREFIX
environment variable...:
https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_PREFIX_PATH.html
Thanks,
--
kou
In
<PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
"PyArrow build problem" on Sun, 11 Sep 2022 13:57:53 +0000,
Yaron Gvili <rt...@hotmail.com> wrote:
> Hi All,
>
> I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
>
> CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
> Unknown CMake command "arrow_keep_backward_compatibility".
> Call Stack (most recent call first):
> /miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
> /miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
> CMakeLists.txt:123 (find_package)
>
>
> Cheers,
> Yaron.
Re: PyArrow build problem
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
It seems that
/miniconda3/envs/pyarrow-dev/lib/cmake/Arrow/ArrowConfig.cmake
isn't used. Could you share a build log with
PYARROW_CMAKE_OPTIONS="-DCMAKE_FIND_DEBUG_MODE=ON" environment
variable?
You may need to export
CMAKE_PREFIX_PATH=/miniconda3/envs/pyarrow-dev environment
variable but the path should be included in
the CMAKE_SYSTEM_PREFIX_PATH CMake variable via the CONDA_PREFIX
environment variable...:
https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_PREFIX_PATH.html
Thanks,
--
kou
In
<PA...@PAXP190MB1565.EURP190.PROD.OUTLOOK.COM>
"PyArrow build problem" on Sun, 11 Sep 2022 13:57:53 +0000,
Yaron Gvili <rt...@hotmail.com> wrote:
> Hi All,
>
> I got the error below while running "cd python/ && python setup.py build_ext --inplace", after successfully building and installing a recent master version (a63e60bad89b41266d155bc496eb383765702492) of Arrow C++ under a pyarrow-dev Conda environment, as in the Python dev doc (https://arrow.apache.org/docs/developers/python.html). I didn't have a problem with this environment until now. Anyone have an idea how to fix the error?
>
> CMake Error at /miniconda3/envs/pyarrow-dev/lib/cmake/Parquet/ParquetConfig.cmake:65 (arrow_keep_backward_compatibility):
> Unknown CMake command "arrow_keep_backward_compatibility".
> Call Stack (most recent call first):
> /miniconda3/envs/pyarrow-dev/share/cmake-3.23/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
> /miniconda3/envs/pyarrow-dev/lib/cmake/ArrowDataset/ArrowDatasetConfig.cmake:55 (find_dependency)
> CMakeLists.txt:123 (find_package)
>
>
> Cheers,
> Yaron.