You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by ji...@apache.org on 2022/09/17 11:09:19 UTC
[pulsar] 02/02: [fix][cpp] Fix libcurl build failure when building deb package (#17614)
This is an automated email from the ASF dual-hosted git repository.
jianghaiting pushed a commit to branch branch-2.10
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 7162c3015d70ff8e7685c8d718bb5152b569fb38
Author: Yunze Xu <xy...@163.com>
AuthorDate: Tue Sep 13 22:18:56 2022 +0800
[fix][cpp] Fix libcurl build failure when building deb package (#17614)
* [fix][cpp] Fix libcurl build failure when building deb package
See https://github.com/apache/pulsar/pull/17538#issuecomment-1244898721
The root cause is when libcurl is built from source, it uses
[`ld`](https://linux.die.net/man/1/ld) to check if the `libcurl.so`
links to the correct dependencies in runtime. In Linux, a dynamic
library links to the paths of `/etc/ld.so.conf` by default. However,
different from other images like `centos:7` and `alpine`, this file
includes `/usr/lib/x86_64-linux-gnu` in `debian:9`.
```bash
$ cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
$ cat /etc/ld.so.conf.d/*.conf
/usr/lib/x86_64-linux-gnu/libfakeroot
/usr/local/lib
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu
```
When libcurl is compiled, it links to the install path of libopenssl via
the `--with-ssl` option:
https://github.com/apache/pulsar/blob/1f50366768e76f1a5f7084f7972167f989ddd0af/pulsar-client-cpp/pkg/deb/Dockerfile#L85
i.e. `/usr/local/ssl/lib/libopenssl.so`.
However, after the `libcurl.so` is built, it links to
`/usr/lib/x86_64-linux-gnu/libssl.so.1.1`, see the following output:
```bash
$ ldd /usr/local/lib/libcurl.so
/usr/local/lib/libcurl.so: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/local/lib/libcurl.so)
```
In `debian:9`, the default libopenssl version is 1.1.0:
```bash
$ strings /usr/lib/x86_64-linux-gnu/libssl.so.1.1 | grep OpenSSL
OpenSSL 1.1.0l 10 Sep 2019
```
The ABI compatibility is not guaranteed between 1.1.0l and 1.1.1n, see
https://abi-laboratory.pro/index.php?view=timeline&l=openssl.
Set the `LD_LIBRARY_PATH` to `/usr/local/ssl/lib` in the Dockerfile to
build deb package.
Actually it's not required for other images like `centos:7`, but it's
also good to add the `LD_LIBRARY_PATH` to them. So this PR set the
environment variable to them as well.
* Fix workflow so that cpp-tests isn't skipped
* Revisit workflow fix to cover doc only workflows too
* Fix multi-line condition
Co-authored-by: Lari Hotari <lh...@apache.org>
(cherry picked from commit 0754ea105eab8c356ba18b2f4862cb1de1f6f1b6)
---
pulsar-client-cpp/docker/alpine/Dockerfile | 2 ++
pulsar-client-cpp/docker/alpine/Dockerfile-alpine-3.8 | 2 ++
pulsar-client-cpp/docker/manylinux1/Dockerfile | 2 ++
pulsar-client-cpp/docker/manylinux2014/Dockerfile | 2 ++
pulsar-client-cpp/docker/manylinux_musl/Dockerfile | 2 ++
pulsar-client-cpp/pkg/deb/Dockerfile | 2 ++
pulsar-client-cpp/pkg/rpm/Dockerfile | 2 ++
7 files changed, 14 insertions(+)
diff --git a/pulsar-client-cpp/docker/alpine/Dockerfile b/pulsar-client-cpp/docker/alpine/Dockerfile
index fb3282d09bd..9527d8dcc5c 100644
--- a/pulsar-client-cpp/docker/alpine/Dockerfile
+++ b/pulsar-client-cpp/docker/alpine/Dockerfile
@@ -58,6 +58,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make -j8 && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# Download and copile protoubf
RUN curl -O -L https://github.com/google/protobuf/releases/download/v3.11.3/protobuf-cpp-3.11.3.tar.gz && \
tar xvfz protobuf-cpp-3.11.3.tar.gz && \
diff --git a/pulsar-client-cpp/docker/alpine/Dockerfile-alpine-3.8 b/pulsar-client-cpp/docker/alpine/Dockerfile-alpine-3.8
index 2873289545b..0f503be471d 100644
--- a/pulsar-client-cpp/docker/alpine/Dockerfile-alpine-3.8
+++ b/pulsar-client-cpp/docker/alpine/Dockerfile-alpine-3.8
@@ -58,6 +58,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make -j8 && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# Download and copile protoubf
RUN curl -O -L https://github.com/google/protobuf/releases/download/v3.11.3/protobuf-cpp-3.11.3.tar.gz && \
tar xvfz protobuf-cpp-3.11.3.tar.gz && \
diff --git a/pulsar-client-cpp/docker/manylinux1/Dockerfile b/pulsar-client-cpp/docker/manylinux1/Dockerfile
index ae7059afca1..8f5bcae77a1 100644
--- a/pulsar-client-cpp/docker/manylinux1/Dockerfile
+++ b/pulsar-client-cpp/docker/manylinux1/Dockerfile
@@ -61,6 +61,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# Download and compile boost
RUN curl -O -L https://boostorg.jfrog.io/artifactory/main/release/1.68.0/source/boost_1_68_0.tar.gz && \
tar xvfz boost_1_68_0.tar.gz && \
diff --git a/pulsar-client-cpp/docker/manylinux2014/Dockerfile b/pulsar-client-cpp/docker/manylinux2014/Dockerfile
index 5bc1b0315ed..f6ee8e0d21b 100644
--- a/pulsar-client-cpp/docker/manylinux2014/Dockerfile
+++ b/pulsar-client-cpp/docker/manylinux2014/Dockerfile
@@ -63,6 +63,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make -j8 && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# Download and compile boost
RUN curl -O -L https://boostorg.jfrog.io/artifactory/main/release/1.78.0/source/boost_1_78_0.tar.gz && \
tar xvfz boost_1_78_0.tar.gz && \
diff --git a/pulsar-client-cpp/docker/manylinux_musl/Dockerfile b/pulsar-client-cpp/docker/manylinux_musl/Dockerfile
index 89dd2d10755..23a081ab238 100644
--- a/pulsar-client-cpp/docker/manylinux_musl/Dockerfile
+++ b/pulsar-client-cpp/docker/manylinux_musl/Dockerfile
@@ -63,6 +63,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make -j8 && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# Download and compile boost
RUN curl -O -L https://boostorg.jfrog.io/artifactory/main/release/1.78.0/source/boost_1_78_0.tar.gz && \
tar xvfz boost_1_78_0.tar.gz && \
diff --git a/pulsar-client-cpp/pkg/deb/Dockerfile b/pulsar-client-cpp/pkg/deb/Dockerfile
index db33f71de38..52a209fddef 100644
--- a/pulsar-client-cpp/pkg/deb/Dockerfile
+++ b/pulsar-client-cpp/pkg/deb/Dockerfile
@@ -79,6 +79,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# LibCurl
RUN curl -O -L https://github.com/curl/curl/releases/download/curl-7_61_0/curl-7.61.0.tar.gz && \
tar xvfz curl-7.61.0.tar.gz && \
diff --git a/pulsar-client-cpp/pkg/rpm/Dockerfile b/pulsar-client-cpp/pkg/rpm/Dockerfile
index ee332c51fd7..642ede00f4b 100644
--- a/pulsar-client-cpp/pkg/rpm/Dockerfile
+++ b/pulsar-client-cpp/pkg/rpm/Dockerfile
@@ -79,6 +79,8 @@ RUN curl -O -L https://github.com/openssl/openssl/archive/OpenSSL_1_1_1n.tar.gz
make && make install && \
rm -rf /OpenSSL_1_1_1n.tar.gz /openssl-OpenSSL_1_1_1n
+ENV LD_LIBRARY_PATH /usr/local/ssl/lib/:
+
# LibCurl
RUN curl -O -L https://github.com/curl/curl/releases/download/curl-7_61_0/curl-7.61.0.tar.gz && \
tar xvfz curl-7.61.0.tar.gz && \