You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/12/22 15:54:36 UTC
[GitHub] [pulsar] samuelzimmermanphillips opened a new issue #13457: Pulsar branch-2.9 build failure in auditwheel running build-wheel-file-within-docker.sh
samuelzimmermanphillips opened a new issue #13457:
URL: https://github.com/apache/pulsar/issues/13457
**Description**
Docker build fails in `pulsar-client-cpp/docker/build-wheel-file-within-docker.sh` with error on `+ auditwheel repair dist/pulsar_client-2.8.2-cp38-cp38-linux_x86_64.whl dist/pulsar_client-2.9.1-cp38-cp38-linux_x86_64.whl`
**To Reproduce**
Steps to reproduce the behavior:
`git clone https://github.com/apache/pulsar.git`
`git checkout branch-2.9`
Follow steps [here](https://github.com/apache/pulsar#build-custom-docker-images):
```
mvn clean install -DskipTests
mvn package -Pdocker,-main -am -pl docker/pulsar-all -DskipTests
```
**Expected behavior**
Pulsar successfully builds docker images on latest version with Log4J 2.17+ to be tagged and pushed to a custom or local docker registry.
**Trace**
```
[100%] Built target _pulsar
make[2]: Leaving directory `/pulsar/pulsar-client-cpp'
/usr/local/bin/cmake -E cmake_progress_start /pulsar/pulsar-client-cpp/CMakeFiles 0
make[1]: Leaving directory `/pulsar/pulsar-client-cpp'
+ cd python
+ python setup.py bdist_wheel
2.9.1
pulsar-client
running bdist_wheel
running build
running build_py
copying pulsar/schema/definition.py -> build/lib.linux-x86_64-3.8/pulsar/schema
copying pulsar/schema/schema.py -> build/lib.linux-x86_64-3.8/pulsar/schema
running build_ext
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64/wheel
copying build/lib.linux-x86_64-3.8/_pulsar.cpython-38-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/pulsar
copying build/lib.linux-x86_64-3.8/pulsar/__init__.py -> build/bdist.linux-x86_64/wheel/pulsar
copying build/lib.linux-x86_64-3.8/pulsar/exceptions.py -> build/bdist.linux-x86_64/wheel/pulsar
creating build/bdist.linux-x86_64/wheel/pulsar/functions
copying build/lib.linux-x86_64-3.8/pulsar/functions/__init__.py -> build/bdist.linux-x86_64/wheel/pulsar/functions
copying build/lib.linux-x86_64-3.8/pulsar/functions/context.py -> build/bdist.linux-x86_64/wheel/pulsar/functions
copying build/lib.linux-x86_64-3.8/pulsar/functions/function.py -> build/bdist.linux-x86_64/wheel/pulsar/functions
copying build/lib.linux-x86_64-3.8/pulsar/functions/serde.py -> build/bdist.linux-x86_64/wheel/pulsar/functions
creating build/bdist.linux-x86_64/wheel/pulsar/schema
copying build/lib.linux-x86_64-3.8/pulsar/schema/__init__.py -> build/bdist.linux-x86_64/wheel/pulsar/schema
copying build/lib.linux-x86_64-3.8/pulsar/schema/definition.py -> build/bdist.linux-x86_64/wheel/pulsar/schema
copying build/lib.linux-x86_64-3.8/pulsar/schema/schema.py -> build/bdist.linux-x86_64/wheel/pulsar/schema
copying build/lib.linux-x86_64-3.8/pulsar/schema/schema_avro.py -> build/bdist.linux-x86_64/wheel/pulsar/schema
running install_egg_info
running egg_info
writing pulsar_client.egg-info/PKG-INFO
writing dependency_links to pulsar_client.egg-info/dependency_links.txt
writing requirements to pulsar_client.egg-info/requires.txt
writing top-level names to pulsar_client.egg-info/top_level.txt
reading manifest file 'pulsar_client.egg-info/SOURCES.txt'
writing manifest file 'pulsar_client.egg-info/SOURCES.txt'
Copying pulsar_client.egg-info to build/bdist.linux-x86_64/wheel/pulsar_client-2.9.1-py3.8.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/pulsar_client-2.9.1.dist-info/WHEEL
creating 'dist/pulsar_client-2.9.1-cp38-cp38-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding '_pulsar.cpython-38-x86_64-linux-gnu.so'
adding 'pulsar/__init__.py'
adding 'pulsar/exceptions.py'
adding 'pulsar/functions/__init__.py'
adding 'pulsar/functions/context.py'
adding 'pulsar/functions/function.py'
adding 'pulsar/functions/serde.py'
adding 'pulsar/schema/__init__.py'
adding 'pulsar/schema/definition.py'
adding 'pulsar/schema/schema.py'
adding 'pulsar/schema/schema_avro.py'
adding 'pulsar_client-2.9.1.dist-info/METADATA'
adding 'pulsar_client-2.9.1.dist-info/WHEEL'
adding 'pulsar_client-2.9.1.dist-info/top_level.txt'
adding 'pulsar_client-2.9.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
+ auditwheel repair dist/pulsar_client-2.8.2-cp38-cp38-linux_x86_64.whl dist/pulsar_client-2.9.1-cp38-cp38-linux_x86_64.whl
usage: auditwheel [-h] [-V] [-v] command ...
auditwheel: error: unrecognized arguments: dist/pulsar_client-2.9.1-cp38-cp38-linux_x86_64.whl
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 2 (Exit value: 2)
```
```
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Pulsar 2.9.1:
[INFO]
[INFO] Pulsar ............................................. SUCCESS [ 24.345 s]
[INFO] Apache Pulsar :: Docker Images ..................... SUCCESS [ 0.630 s]
[INFO] Apache Pulsar :: Docker Images :: Pulsar Latest Version FAILURE [27:17 min]
[INFO] Apache Pulsar :: Docker Images :: Pulsar Latest Version (Include All Components) SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27:43 min
[INFO] Finished at: 2021-12-22T09:50:54-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:exec (build-pulsar-clients-python-35) on project pulsar-docker-image: Command execution failed.: Process exited with an error: 2 (Exit value: 2) -> [Help 1]
```
**Desktop:**
- OS: MacOS 12.0 Monterey `Darwin <hostname removed>.local 21.0.1 Darwin Kernel Version 21.0.1: Tue Sep 14 20:56:24 PDT 2021; root:xnu-8019.30.61~4/RELEASE_ARM64_T6000 arm64`
However, the command is failing inside a container it seems.
**Additional context**
This error occurs on branch 2.9. I've also tried to build master and branch-2.8 with no success. I've also tried building following the steps to reproduce on a clean install of Linux Mint with openjdk-11 and maven 3.8.4, but that gives a different error about `Text file busy: 'build/bdist.linux-x86_64/wheel/pulsar_client-2.9.1-py3.8.egg-info'` - so should be a separate issue.
Maven:
```
➜ mvn --version
Apache Maven 3.8.4 (9b656c72d54e5bacbed989b64718c159fe39b537)
Maven home: /opt/homebrew/Cellar/maven/3.8.4/libexec
Java version: 11, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-11.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.16", arch: "x86_64", family: "mac"
```
Java:
```
➜ java --version
openjdk 11 2018-09-25
OpenJDK Runtime Environment 18.9 (build 11+28)
OpenJDK 64-Bit Server VM 18.9 (build 11+28, mixed mode)
```
Python (although it seems to be using a bundled version so this probably doesn't matter):
`Python 3.9.7`
Side note: Is there any timeline for when the Log4J patched (2.17) images will be released to the [Docker Registry](https://hub.docker.com/r/apachepulsar/pulsar-all/tags)? I feel like there are a lot of people struggling with issues like this and I'm surprised the patch has been completed in here but hasn't been released. There's still a lot of vulnerable Pulsar instances because of this. It's blocking development for us as we also can't seem to build pulsar images across different machines.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sarimak commented on issue #13457: Pulsar branch-2.9 build failure in auditwheel running build-wheel-file-within-docker.sh
Posted by GitBox <gi...@apache.org>.
sarimak commented on issue #13457:
URL: https://github.com/apache/pulsar/issues/13457#issuecomment-1003118572
I encountered the same issue when self-building manylinux2014 wheel for Pulsar Client 2.8.1 and Python 3.9 using backported Docker-based build procedure from https://github.com/apache/pulsar/pull/10954/files to v2.8.1 tag. The wheel gets built and works fine, just the auditwheel step failed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] eolivelli commented on issue #13457: Pulsar branch-2.9 build failure in auditwheel running build-wheel-file-within-docker.sh
Posted by GitBox <gi...@apache.org>.
eolivelli commented on issue #13457:
URL: https://github.com/apache/pulsar/issues/13457#issuecomment-999889941
Can you try to 'git clean -xdf' your repo?
IIRC that errors occurs if you have multiple builds locally
I have built the official images for 2.9.1 and 2.9.0 and it worked well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] kezhenxu94 commented on issue #13457: Pulsar branch-2.9 build failure in auditwheel running build-wheel-file-within-docker.sh
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on issue #13457:
URL: https://github.com/apache/pulsar/issues/13457#issuecomment-1003191436
@samuelzimmermanphillips please subscribe https://github.com/apache/pulsar/issues/12944 to follow the process on building under ARM architecture
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] samuelzimmermanphillips commented on issue #13457: Pulsar branch-2.9 build failure in auditwheel running build-wheel-file-within-docker.sh
Posted by GitBox <gi...@apache.org>.
samuelzimmermanphillips commented on issue #13457:
URL: https://github.com/apache/pulsar/issues/13457#issuecomment-999921950
> Can you try to 'git clean -xdf' your repo? IIRC that errors occurs if you have multiple builds locally
>
> I have built the official images for 2.9.1 and 2.9.0 and it worked well
That seemed to get rid of the auditwheel error, but now it's failing on the `java.security` file. This appears to be an issue in the docker container as it's using the amd64 version and I'm compiling on arm.
`[ERROR] Failed to execute goal com.spotify:dockerfile-maven-plugin:1.4.13:build (default) on project pulsar-docker-image: Could not build image: The command '/bin/sh -c echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-11-openjdk-amd64/conf/security/java.security' returned a non-zero code: 2 -> [Help 1]`
More trace:
```
[INFO] dockerfile: null
[INFO] contextDirectory: /Users/samwork/Code/pulsar/docker/pulsar
[INFO] Building Docker context /Users/samwork/Code/pulsar/docker/pulsar
[INFO] Path(dockerfile): null
[INFO] Path(contextDirectory): /Users/samwork/Code/pulsar/docker/pulsar
[INFO]
[INFO] Image will be built as apachepulsar/pulsar:2.9.2-SNAPSHOT
[INFO]
[INFO] Step 1/26 : FROM busybox as pulsar
```
[...]
```
[INFO] ---> Using cache
[INFO] ---> 19d52019bcaa
[INFO] Step 17/26 : RUN python3 get-pip.py
[INFO]
[INFO] ---> Using cache
[INFO] ---> 19e943035cac
[INFO] Step 18/26 : RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
[INFO]
[INFO] ---> Using cache
[INFO] ---> 0ac369505cc6
[INFO] Step 19/26 : ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
[INFO]
[INFO] ---> Using cache
[INFO] ---> 7d06df455d73
[INFO] Step 20/26 : RUN echo networkaddress.cache.ttl=1 >> /usr/lib/jvm/java-11-openjdk-amd64/conf/security/java.security
[INFO]
[INFO] ---> Running in 977017eabc3e
[INFO] /bin/sh: 1: cannot create /usr/lib/jvm/java-11-openjdk-amd64/conf/security/java.security: Directory nonexistent
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] github-actions[bot] commented on issue #13457: Pulsar branch-2.9 build failure in auditwheel running build-wheel-file-within-docker.sh
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #13457:
URL: https://github.com/apache/pulsar/issues/13457#issuecomment-1051438264
The issue had no activity for 30 days, mark with Stale label.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org