You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by si...@apache.org on 2020/10/01 14:56:39 UTC
[pulsar] branch master updated: [python] Made the script for
building Docker images for Python work (#8153)
This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new e65875b [python] Made the script for building Docker images for Python work (#8153)
e65875b is described below
commit e65875b99e3985630e3147c2579e91da4c3973bf
Author: Masahiro Sakamoto <ma...@yahoo-corp.jp>
AuthorDate: Thu Oct 1 23:56:19 2020 +0900
[python] Made the script for building Docker images for Python work (#8153)
### Motivation
Currently, enabling SNAPPY compression on the Python client causes a segmentation fault. I think this is because the Docker images for building the Python clients uploaded to Docker Hub do not include the SNAPPY library.
We need to run `pulsar-client-cpp/docker/create-images.sh` in order to build the latest Docker images that include the SNAPPY library, but this script fails for two reasons:
1. Log4cxx compilation fails. Currently the source of log4cxx is from the master branch, but if we get the source of tag `v0.11.0` and compile it, it will succeed.
```
In file included from logstream.cpp:24:0:
../../../src/main/include/log4cxx/private/log4cxx_private.h:43:31: error: token "@" is not valid in preprocessor expressions
#define LOG4CXX_INIT_IOS_BASE @INIT_IOS_BASE@
^
logstream.cpp:38:5: note: in expansion of macro ‘LOG4CXX_INIT_IOS_BASE’
#if LOG4CXX_INIT_IOS_BASE
^
```
2. Building the image `pulsar-build/manylinux-cp34-cp34m` fails. This is probably because Python 3.4 has already reached EOL in 2014 and is no longer included in the base image `quay.io/pypa/manylinux1_x86_64`.
```
Step 8/32 : RUN ln -s /opt/python/${PYTHON_SPEC}/include/python${PYTHON_VERSION}m /opt/python/${PYTHON_SPEC}/include/python${PYTHON_VERSION}
---> Running in 0e1d652c2c29
ln: creating symbolic link `/opt/python/cp34-cp34m/include/python3.4' to `/opt/python/cp34-cp34m/include/python3.4m': No such file or directory
The command '/bin/sh -c ln -s /opt/python/${PYTHON_SPEC}/include/python${PYTHON_VERSION}m /opt/python/${PYTHON_SPEC}/include/python${PYTHON_VERSION}' returned a non-zero code: 1
```
### Modifications
1. Get the source of Log4cxx from tag v0.11.0 when building the Docker images.
2. Do not build the Docker image for Python 3.4.
---
pulsar-client-cpp/docker/Dockerfile | 7 ++++---
pulsar-client-cpp/docker/build-wheels.sh | 1 -
pulsar-client-cpp/docker/create-images.sh | 1 -
pulsar-client-cpp/docker/push-images.sh | 1 -
pulsar-client-cpp/python/pulsar/__init__.py | 2 +-
5 files changed, 5 insertions(+), 7 deletions(-)
diff --git a/pulsar-client-cpp/docker/Dockerfile b/pulsar-client-cpp/docker/Dockerfile
index e3458f6..c158006 100644
--- a/pulsar-client-cpp/docker/Dockerfile
+++ b/pulsar-client-cpp/docker/Dockerfile
@@ -102,12 +102,13 @@ RUN curl -L -O https://ftp.gnu.org/gnu/libtool/libtool-2.4.6.tar.gz && \
rm -rf /libtool-2.4.6.tar.gz /libtool-2.4.6
# Compile log4cxx
-RUN git clone https://github.com/apache/logging-log4cxx.git && \
- cd logging-log4cxx && \
+RUN curl -O -L https://github.com/apache/logging-log4cxx/archive/v0.11.0.tar.gz && \
+ tar xvfz v0.11.0.tar.gz && \
+ cd logging-log4cxx-0.11.0 && \
./autogen.sh && \
CXXFLAGS=-fPIC ./configure && \
make && make install && \
- rm -rf /logging-log4cxx
+ rm -rf /v0.11.0.tar.gz /logging-log4cxx-0.11.0
# Compile expat
RUN curl -O -L https://github.com/libexpat/libexpat/archive/R_2_2_0.tar.gz && \
diff --git a/pulsar-client-cpp/docker/build-wheels.sh b/pulsar-client-cpp/docker/build-wheels.sh
index 1007b29..a281655 100755
--- a/pulsar-client-cpp/docker/build-wheels.sh
+++ b/pulsar-client-cpp/docker/build-wheels.sh
@@ -29,7 +29,6 @@ cd $ROOT_DIR
PYTHON_VERSIONS=(
'2.7 cp27-cp27mu'
'2.7 cp27-cp27m'
- '3.4 cp34-cp34m'
'3.5 cp35-cp35m'
'3.6 cp36-cp36m'
'3.7 cp37-cp37m'
diff --git a/pulsar-client-cpp/docker/create-images.sh b/pulsar-client-cpp/docker/create-images.sh
index 34136c1..883d7fe 100755
--- a/pulsar-client-cpp/docker/create-images.sh
+++ b/pulsar-client-cpp/docker/create-images.sh
@@ -26,7 +26,6 @@ set -e
PYTHON_VERSIONS=(
'2.7 cp27-cp27mu'
'2.7 cp27-cp27m'
- '3.4 cp34-cp34m'
'3.5 cp35-cp35m'
'3.6 cp36-cp36m'
'3.7 cp37-cp37m'
diff --git a/pulsar-client-cpp/docker/push-images.sh b/pulsar-client-cpp/docker/push-images.sh
index 155becd..5792dbf 100755
--- a/pulsar-client-cpp/docker/push-images.sh
+++ b/pulsar-client-cpp/docker/push-images.sh
@@ -28,7 +28,6 @@ DOCKER_ORG=apachepulsar
PYTHON_VERSIONS=(
'2.7 cp27-cp27mu'
'2.7 cp27-cp27m'
- '3.4 cp34-cp34m'
'3.5 cp35-cp35m'
'3.6 cp36-cp36m'
'3.7 cp37-cp37m'
diff --git a/pulsar-client-cpp/python/pulsar/__init__.py b/pulsar-client-cpp/python/pulsar/__init__.py
index 4912d8d..f1eec97 100644
--- a/pulsar-client-cpp/python/pulsar/__init__.py
+++ b/pulsar-client-cpp/python/pulsar/__init__.py
@@ -21,7 +21,7 @@
The Pulsar Python client library is based on the existing C++ client library.
All the same features are exposed through the Python interface.
-Currently, the supported Python versions are 2.7, 3.4, 3.5, 3.6 and 3.7.
+Currently, the supported Python versions are 2.7, 3.5, 3.6, 3.7 and 3.8.
## Install from PyPI