You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by we...@apache.org on 2019/07/09 14:22:50 UTC
[arrow] branch master updated: ARROW-5868: [Python] Correctly
remove liblz4 shared libraries from manylinux2010 image so lz4 is
statically linked
This is an automated email from the ASF dual-hosted git repository.
wesm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/master by this push:
new 7838886 ARROW-5868: [Python] Correctly remove liblz4 shared libraries from manylinux2010 image so lz4 is statically linked
7838886 is described below
commit 7838886bf6862d3bdc1225caf24e2c6150d6ef33
Author: Wes McKinney <we...@apache.org>
AuthorDate: Tue Jul 9 09:22:28 2019 -0500
ARROW-5868: [Python] Correctly remove liblz4 shared libraries from manylinux2010 image so lz4 is statically linked
I pushed the image to Docker Hub (it was on quay.io before). I'm not sure what's the best way to test for this -- I checked the produced wheels locally to verify that liblz4.so is no longer required
Author: Wes McKinney <we...@apache.org>
Closes #4828 from wesm/ARROW-5868 and squashes the following commits:
1fb1acb09 <Wes McKinney> Remove liblz4 shared libraries from /usr/local so static linking occurs
---
docker-compose.yml | 2 +-
python/manylinux2010/README.md | 46 ++++---------------------------
python/manylinux2010/scripts/build_lz4.sh | 2 +-
3 files changed, 8 insertions(+), 42 deletions(-)
diff --git a/docker-compose.yml b/docker-compose.yml
index 0b977b1..8d197c1 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -484,7 +484,7 @@ services:
# $ docker-compose pull python-manylinux2010
# an then run:
# $ docker-compose run -e PYTHON_VERSION=3.7 python-manylinux2010
- image: quay.io/ursa-labs/arrow_manylinux2010_x86_64_base:latest
+ image: ursalab/arrow_manylinux2010_x86_64_base:latest
build:
context: python/manylinux2010
dockerfile: Dockerfile-x86_64_base
diff --git a/python/manylinux2010/README.md b/python/manylinux2010/README.md
index db0e414..c5acb27 100644
--- a/python/manylinux2010/README.md
+++ b/python/manylinux2010/README.md
@@ -27,7 +27,7 @@ The process is split up in two parts:
1. There are base Docker images that contain the build dependencies for
Arrow. Those images do not need to be rebuilt frequently, and are hosted
- on the public quay.io service.
+ on the public Docker Hub service.
2. Based on on these images, there is a bash script (`build_arrow.sh`) that
the PyArrow wheels for all supported Python versions, and place them
@@ -42,7 +42,7 @@ use `PYTHON_VERSION="2.7"` with `UNICODE_WIDTH=32`):
```bash
# Build the python packages
-docker run --env PYTHON_VERSION="2.7" --env UNICODE_WIDTH=16 --shm-size=2g --rm -t -i -v $PWD:/io -v $PWD/../../:/arrow quay.io/ursa-labs/arrow_manylinux2010_x86_64_base:latest /io/build_arrow.sh
+docker run --env PYTHON_VERSION="2.7" --env UNICODE_WIDTH=16 --shm-size=2g --rm -t -i -v $PWD:/io -v $PWD/../../:/arrow ursalab/arrow_manylinux2010_x86_64_base:latest /io/build_arrow.sh
# Now the new packages are located in the dist/ folder
ls -l dist/
```
@@ -65,46 +65,12 @@ installation of a dependency is persisted in the Docker image.
### Publishing a new build image
-If you have write access to the `quay.io` Ursa Labs account, you can directly
+If you have write access to the Docker Hub Ursa Labs account, you can directly
publish a build image that you built locally.
-For that you need to first tag your image for quay.io upload:
```bash
-$ docker image tag arrow_manylinux2010_x86_64_base:latest quay.io/ursa-labs/arrow_manylinux2010_x86_64_base
-```
-
-Then you can push it:
-```bash
-$ docker image push quay.io/ursa-labs/arrow_manylinux2010_x86_64_base
-The push refers to repository [quay.io/ursa-labs/arrow_manylinux2010_x86_64_base]
+$ docker push ursalab/arrow_manylinux2010_x86_64_base
+The push refers to repository [ursalab/arrow_manylinux2010_x86_64_base]
a1ab88d27acc: Pushing [==============> ] 492.5MB/1.645GB
[... etc. ...]
-```
-
-### Using quay.io to trigger and build the docker image
-
-You can also create your own `quay.io` repository and trigger builds there from
-your Github fork of the Arrow repository.
-
-1. Make the change in the build scripts (eg. to modify the boost build, update `scripts/boost.sh`).
-
-2. Setup an account on quay.io and link to your GitHub account
-
-3. In quay.io, Add a new repository using :
-
- 1. Link to GitHub repository push
- 2. Trigger build on changes to a specific branch (eg. myquay) of the repo (eg. `pravindra/arrow`)
- 3. Set Dockerfile location to `/python/manylinux2010/Dockerfile-x86_64_base`
- 4. Set Context location to `/python/manylinux2010`
-
-4. Push change (in step 1) to the branch specified in step 3.ii
-
- * This should trigger a build in quay.io, the build takes about 2 hrs to finish.
-
-5. Add a tag `latest` to the build after step 4 finishes, save the build ID (eg. `quay.io/pravindra/arrow_manylinux2010_x86_64_base:latest`)
-
-6. In your arrow PR,
-
- * include the change from 1.
- * modify the `python-manylinux2010` entry in `docker-compose.yml`
- to switch to the location from step 5 for the docker image.
+```
\ No newline at end of file
diff --git a/python/manylinux2010/scripts/build_lz4.sh b/python/manylinux2010/scripts/build_lz4.sh
index 7b3d441..6d5d34c 100755
--- a/python/manylinux2010/scripts/build_lz4.sh
+++ b/python/manylinux2010/scripts/build_lz4.sh
@@ -30,4 +30,4 @@ make install PREFIX=${PREFIX}
popd
rm -rf lz4-${LZ4_VERSION}.tar.gz lz4-${LZ4_VERSION}
# We don't want to link against shared libs
-rm -rf /usr/lib/liblz4.so*
+rm -rf ${PREFIX}/lib/liblz4.so*