You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "srinathtcs1993 (via GitHub)" <gi...@apache.org> on 2023/04/12 12:02:14 UTC

[GitHub] [arrow] srinathtcs1993 opened a new issue, #35072: Issue while try adding Pyarrow dependency to Docker file

srinathtcs1993 opened a new issue, #35072:
URL: https://github.com/apache/arrow/issues/35072

   ### Describe the bug, including details regarding any error messages, version, and platform.
   
   I am trying to add pyarrow to my docker file as a dependency. I am getting an issue as below
   
   CMake Error at thrift_ep-stamp/thrift_ep-download-RELEASE-impl.cmake:9 (message):
     Command failed (1):
   
      '/usr/bin/cmake' '-P' '/arrow/cpp/build/thrift_ep-prefix/src/thrift_ep-stamp/download-thrift_ep.cmake'
   
   make[2]: *** [CMakeFiles/thrift_ep.dir/build.make:99: thrift_ep-prefix/src/thrift_ep-stamp/thrift_ep-download] Error 1
   make[1]: *** [CMakeFiles/Makefile2:828: CMakeFiles/thrift_ep.dir/all] Error 2
   make[1]: *** Waiting for unfinished jobs....
   -- zstd_ep build command succeeded.  See also /arrow/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-build-*.log
   [ 16%] Performing install step for 'zstd_ep'
   -- zstd_ep install command succeeded.  See also /arrow/cpp/build/zstd_ep-prefix/src/zstd_ep-stamp/zstd_ep-install-*.log
   [ 16%] Completed 'zstd_ep'
   [ 16%] Built target zstd_ep
   make: *** [Makefile:146: all] Error 2
   The command '/bin/sh -c mkdir /arrow     && wget -q [https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz](https://github.com/apache/arrow/archive/apache-arrow-$%7BARROW_VERSION%7D.tar.gz) -O /tmp/apache-arrow.tar.gz     && echo "${ARROW_SHA1} *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz     && tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1     && mkdir -p /arrow/cpp/build     && cd /arrow/cpp/build     && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE         -DOPENSSL_ROOT_DIR=/usr/local/ssl         -DCMAKE_INSTALL_LIBDIR=lib         -DCMAKE_INSTALL_PREFIX=$ARROW_HOME         -DARROW_WITH_BZ2=ON         -DARROW_WITH_ZLIB=ON         -DARROW_WITH_ZSTD=ON         -DARROW_WITH_LZ4=ON         -DARROW_WITH_SNAPPY=ON         -DARROW_PARQUET=ON         -DARROW_PYTHON=ON         -DARROW_PLASMA=ON         -DARROW_BUILD_TESTS=OFF         ..     && make -j$(nproc)     && make install     && cd /arrow/python     && python setup.py build_e
 xt --build-type=$ARROW_BUILD_TYPE --with-parquet     && python setup.py install     && rm -rf /arrow /tmp/apache-arrow.tar.gz' returned a non-zero code: 2
   
   
   
   This is my docker build file code
   
   
   FROM python:3.8-alpine3.16
   
   RUN apk update \
       && apk upgrade \
       && apk add --no-cache build-base \
           autoconf \
           bash \
           bison \
           boost-dev \
           cmake \
           flex \
           libressl-dev \
           zlib-dev
   RUN apk upgrade expat
   
   RUN pip install --no-cache-dir six numpy cython
   
   ARG ARROW_VERSION=3.0.0
   ARG ARROW_SHA1=c1fed962cddfab1966a0e03461376ebb28cf17d3
   ARG ARROW_BUILD_TYPE=release
   
   ENV ARROW_HOME=/usr/local \
       PARQUET_HOME=/usr/local
   
   #Download and build apache-arrow
   RUN mkdir /arrow \
       && wget -q https://github.com/apache/arrow/archive/apache-arrow-${ARROW_VERSION}.tar.gz -O /tmp/apache-arrow.tar.gz \
       && echo "${ARROW_SHA1} *apache-arrow.tar.gz" | sha1sum /tmp/apache-arrow.tar.gz \
       && tar -xvf /tmp/apache-arrow.tar.gz -C /arrow --strip-components 1 \
       && mkdir -p /arrow/cpp/build \
       && cd /arrow/cpp/build \
       && cmake -DCMAKE_BUILD_TYPE=$ARROW_BUILD_TYPE \
           -DOPENSSL_ROOT_DIR=/usr/local/ssl \
           -DCMAKE_INSTALL_LIBDIR=lib \
           -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
           -DARROW_WITH_BZ2=ON \
           -DARROW_WITH_ZLIB=ON \
           -DARROW_WITH_ZSTD=ON \
           -DARROW_WITH_LZ4=ON \
           -DARROW_WITH_SNAPPY=ON \
           -DARROW_PARQUET=ON \
           -DARROW_PYTHON=ON \
           -DARROW_PLASMA=ON \
           -DARROW_BUILD_TESTS=OFF \
           .. \
       && make -j$(nproc) \
       && make install \
       && cd /arrow/python \
       && python setup.py build_ext --build-type=$ARROW_BUILD_TYPE --with-parquet \
       && python setup.py install \
       && rm -rf /arrow /tmp/apache-arrow.tar.gz
   
   
   kindly let me know what changes should I do to add pyarrow as a dependency to docker file
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1511052361

   Yes, thanks for your support


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506357769

   [Pyarrow_installation_full_log.txt](https://github.com/apache/arrow/files/11218371/Pyarrow_installation_full_log.txt)
   Attached the full log here.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506227062

   Or how about installing the `thrift` package by `apk`?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506412083

   Yes this is correct


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1510843342

   Thanks for your suggestion. thrift-dev is working fine, but need to understand why the hash comparison got failed for thrift, since that was working fine couple of months ago, 
   
   I have a doubt whether thrift-dev may also end up having the same hash comparison failure in near future.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1510955221

   > need to understand why the hash comparison got failed for thrift, since that was working fine couple of months ago,
   
   I think that we can't help you because I think that it's a your environment problem.
   
   > I have a doubt whether thrift-dev may also end up having the same hash comparison failure in near future.
   
   It must not happen. If you install `thrift-dev`, Apache Arrow C++ doesn't use bundled Apache Thrift. It means that you don't need to download Apache Thrift source archive and validate it by yourself.
   
   Can we close this issue?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1505835380

   3.0.0 is too old. Could you try recent version?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506414934

   Then why did `thrift-0.12.0.tar.gz` downloaded from your network have wrong MD5 hash (`ff9af01fec424b5a279fa8a3c9e95c0c` not `3deebbb4d1ca77dd9c9e009a1ea02183`)?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506225612

   Could you attach full error log?
   It seems that Apache Thrift download error is happen.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506362338

   Apache Thrift is downloaded, but after that it is trying compare the hash value and due to mismatch it is failing
   -- Using src='http://www.apache.org/dyn/closer.cgi?action=download&filename=/thrift/0.12.0/thrift-0.12.0.tar.gz'
   -- Using src='https://downloads.apache.org/thrift/0.12.0/thrift-0.12.0.tar.gz'
   -- Using src='https://github.com/apache/thrift/archive/v0.12.0.tar.gz'
   -- verifying file...
          file='/arrow/cpp/build/thrift_ep-prefix/src/thrift-0.12.0.tar.gz'
   -- MD5 hash of
       /arrow/cpp/build/thrift_ep-prefix/src/thrift-0.12.0.tar.gz
     does not match expected value
       expected: '3deebbb4d1ca77dd9c9e009a1ea02183'
         actual: 'ff9af01fec424b5a279fa8a3c9e95c0c'
   -- Hash mismatch, removing...
   -- Using src='https://apache.claz.org/thrift/0.12.0/thrift-0.12.0.tar.gz'
   -- Using src='https://github.com/apache/thrift/archive/v0.12.0.tar.gz'
   -- verifying file...
          file='/arrow/cpp/build/thrift_ep-prefix/src/thrift-0.12.0.tar.gz'
   -- MD5 hash of
       /arrow/cpp/build/thrift_ep-prefix/src/thrift-0.12.0.tar.gz
     does not match expected value
       expected: '3deebbb4d1ca77dd9c9e009a1ea02183'
         actual: 'ff9af01fec424b5a279fa8a3c9e95c0c'
   -- Hash mismatch, removing...
   -- Using src='https://apache.cs.utah.edu/thrift/0.12.0/thrift-0.12.0.tar.gz'
   -- Using src='https://github.com/apache/thrift/archive/v0.12.0.tar.gz'
   
   
   CMAKE is taking the hash value from this attribute, DARROW_THRIFT_BUILD_MD5_CHECKSUM, not sure how to skip this hash value comparison or override a value which we require to make that hash comparison successful


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506538743

   Ah, sorry. `thrift-dev` not `thrift`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506502111

   I have tried installing thrift using apk
   ![image](https://user-images.githubusercontent.com/34440654/231689109-60ddcacd-885c-45fb-9c9b-acb1a2bf402d.png)
   but getting error again


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506535243

   I can't help you without log.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506222236

   I tried with 4.0.0 till 11.0.0 version, but all throws the same error.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506383391

   Is your network correct?
   `3deebbb4d1ca77dd9c9e009a1ea02183` is the correct MD5 hash: https://archive.apache.org/dist//thrift/0.12.0/thrift-0.12.0.tar.gz.md5


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] kou commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506463907

   > Or how about installing the `thrift` package by `apk`?
   
   may help you.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] srinathtcs1993 commented on issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "srinathtcs1993 (via GitHub)" <gi...@apache.org>.
srinathtcs1993 commented on issue #35072:
URL: https://github.com/apache/arrow/issues/35072#issuecomment-1506438946

   Not sure of that


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [arrow] raulcd closed issue #35072: Issue while try adding Pyarrow dependency to Docker file

Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd closed issue #35072: Issue while try adding Pyarrow dependency to Docker file
URL: https://github.com/apache/arrow/issues/35072


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org