[GitHub] [arrow] prashantkhoje opened a new issue #12594: [Python][ppc64le][cuda] pytest failure - test_cuda.py/test_foreign_buffer

[GitBox <gi...@apache.org>]

prashantkhoje opened a new issue #12594:
URL: https://github.com/apache/arrow/issues/12594


   Hello everyone,
   I tried to build and test pyarrow on ppc64le platform with CUDA.
   I encounter segmentation fault at [test_cuda.py/test_foreign_buffer#L292](https://github.com/apache/arrow/blob/e90472e35b40f58b17d408438bb8de1641bfe6ef/python/pyarrow/tests/test_cuda.py#L292).
   If i comment out the test_foreign_buffer, then rest of the tests pass successfully.
   
   Has anyone experienced the same either on ppc or x86-64?
   I'm sharing steps i followed and log.
   
   **# STEPS**
   ________________________________________________________________________________
   ```
   mkdir p11-arrow-repo
   cd p11-arrow-repo
   git clone https://github.com/apache/arrow.git
   cd arrow
   git checkout apache-arrow-7.0.0
   # Not available: benchmark (conda_env_cpp.txt) - Removed for now.
   vim ci/conda_env_cpp.txt
   git diff ci/conda_env_cpp.txt
   cd ..
   
   conda create -y -n p11-arrow -c conda-forge
   conda activate p11-arrow
   
   conda install -y -c conda-forge \
       --file arrow/ci/conda_e
   [p11-arrow.txt](https://github.com/apache/arrow/files/8215513/p11-arrow.txt)
   nv_unix.txt \
       --file arrow/ci/conda_env_cpp.txt \
       --file arrow/ci/conda_env_python.txt \
       --file arrow/ci/conda_env_gandiva.txt \
       compilers \
       python=3.9 \
       pandas
   
   conda install -y -c conda-forge cudatoolkit=11.2
   
   pushd arrow
   git submodule init
   git submodule update
   export PARQUET_TEST_DATA="${PWD}/cpp/submodules/parquet-testing/data"
   export ARROW_TEST_DATA="${PWD}/testing/data"
   popd
   
   # CPP
   export ARROW_HOME=$CONDA_PREFIX
   export LD_LIBRARY_PATH=$CONDA_PREFIX/lib
   
   mkdir arrow/cpp/build
   pushd arrow/cpp/build
   
   cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
         -DCMAKE_INSTALL_LIBDIR=lib \
         -DCMAKE_BUILD_TYPE=release \
         -DARROW_WITH_BZ2=ON \
         -DARROW_WITH_ZLIB=ON \
         -DARROW_WITH_ZSTD=ON \
         -DARROW_WITH_LZ4=ON \
         -DARROW_WITH_SNAPPY=ON \
         -DARROW_WITH_BROTLI=ON \
         -DARROW_PARQUET=ON \
         -DARROW_PYTHON=ON \
         -DARROW_BUILD_TESTS=ON \
         -DARROW_ORC=ON \
         -DARROW_DATASET=ON \
         -DARROW_HDFS=ON \
         -DARROW_CUDA=ON -DARROW_EXTRA_ERROR_CONTEXT=ON ..
   
   make -j 4
   make install
   ctest
   popd
   
   # Python
   pushd arrow/python
   export PYARROW_WITH_PARQUET=1
   export PYARROW_WITH_CUDA=1
   export PYARROW_WITH_ORC=1
   export PYARROW_WITH_DATASET=1
   export PYARROW_WITH_HDFS=1
   python setup.py build_ext --inplace
   python setup.py install
   pip install -e . --no-build-isolation
   pytest -v
   popd
   ```
   
   Thank you,
   Prashant
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] prashantkhoje commented on issue #12594: [Python][ppc64le][cuda] pytest segfault - test_cuda.py/test_foreign_buffer

[GitBox <gi...@apache.org>]

prashantkhoje commented on issue #12594:
URL: https://github.com/apache/arrow/issues/12594#issuecomment-1067787181


   @leofang, @jakirkham, Have you encountered the failure mentioned here?
   I don't have x86 with gpu to compare.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] prashantkhoje commented on issue #12594: [Python][ppc64le][cuda] pytest failure - test_cuda.py/test_foreign_buffer

[GitBox <gi...@apache.org>]

prashantkhoje commented on issue #12594:
URL: https://github.com/apache/arrow/issues/12594#issuecomment-1062989489


   [p11-arrow.txt](https://github.com/apache/arrow/files/8215547/p11-arrow.txt)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org