You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "BewareMyPower (via GitHub)" <gi...@apache.org> on 2024/12/12 03:40:10 UTC

[I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

BewareMyPower opened a new issue, #23717:
URL: https://github.com/apache/pulsar/issues/23717

   ### Search before asking
   
   - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
   
   
   ### Read release policy
   
   - [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
   
   
   ### Version
   
   ```
   apachepulsar/pulsar                                                latest      93b5506ac8eb   12 days ago     481MB
   ```
   
   ### Minimal reproduce step
   
   ```
   git clone https://github.com/apache/pulsar-client-python.git
   cd pulsar-client-python
   ./build-support/pulsar-test-service-start.sh
   ```
   
   The script is stuck at waiting for the 8080 port.
   
   ### What did you expect to see?
   
   The script should succeed
   
   ### What did you see instead?
   
   The Pulsar process in the container was crashed and the log from `/pulsar/logs` inside the container stopped at:
   
   ```
   2024-12-12T02:49:54,604+0000 [main] INFO
   org.apache.pulsar.broker.service.BrokerService - Started Pulsar Broker
   service
   ```
   
   https://github.com/apache/pulsar/blob/f571aa1e6c247bea54d7a10ed508991fee1ea71b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L608-L612
   
   As you can see, it didn't log the listened port.
   
   ### Anything else?
   
   [hs_err_pid296.log](https://github.com/user-attachments/files/18105163/hs_err_pid296.log)
   
   It crashed when loading the native SSL library:
   
   ```
   Current thread (0x0000ffffaa98c800):  JavaThread "main"             [_thread_in_native, id=349, stack(0x0000ffffadd2d000,0x0000ffffadf2ba80) (2042K)]
   
   Stack: [0x0000ffffadd2d000,0x0000ffffadf2ba80],  sp=0x0000ffffadf287f0,  free space=2029k
   Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
   C  [libnetty_tcnative_linux_aarch_6415896617181118050458.so+0x240ec]  init_have_lse_atomics+0xc
   C  [ld-musl-aarch64.so.1+0x6b2fc]
   C  [ld-musl-aarch64.so.1+0x6d058]  dlopen+0xc4
   V  [libjvm.so+0xbc2078]  os::Linux::dlopen_helper(char const*, char*, int)+0x28
   V  [libjvm.so+0xbc2364]  os::dll_load(char const*, char*, int)+0x74
   V  [libjvm.so+0x90704c]  JVM_LoadLibrary+0x88
   C  [libjava.so+0xf3cc]  Java_jdk_internal_loader_NativeLibraries_load+0x21c
   j  jdk.internal.loader.NativeLibraries.load(Ljdk/internal/loader/NativeLibraries$NativeLibraryImpl;Ljava/lang/String;ZZ)Z+0 java.base@21.0.5
   j  jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open()Z+57 java.base@21.0.5
   j  jdk.internal.loader.NativeLibraries.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)Ljdk/internal/loader/NativeLibrary;+254 java.base@21.0.5
   j  jdk.internal.loader.NativeLibraries.loadLibrary(Ljava/lang/Class;Ljava/io/File;)Ljdk/internal/loader/NativeLibrary;+51 java.base@21.0.5
   j  java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/io/File;)Ljdk/internal/loader/NativeLibrary;+31 java.base@21.0.5
   j  java.lang.Runtime.load0(Ljava/lang/Class;Ljava/lang/String;)V+61 java.base@21.0.5
   j  java.lang.System.load(Ljava/lang/String;)V+7 java.base@21.0.5
   j  io.netty.util.internal.NativeLibraryUtil.loadLibrary(Ljava/lang/String;Z)V+5
   j  java.lang.invoke.LambdaForm$DMH+0x0000008800430000.invokeStatic(Ljava/lang/Object;Ljava/lang/Object;I)V+11 java.base@21.0.5
   j  java.lang.invoke.LambdaForm$MH+0x0000008800431000.invoke(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+54 java.base@21.0.5
   j  java.lang.invoke.LambdaForm$MH+0x0000008800128400.invokeExact_MT(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;+22 java.base@21.0.5
   j  jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+72 java.base@21.0.5
   j  jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;+23 java.base@21.0.5
   J 3874 c1 java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; java.base@21.0.5 (108 bytes) @ 0x0000ffff9bf827e4 [0x0000ffff9bf82240+0x00000000000005a4]
   j  io.netty.util.internal.NativeLibraryLoader$1.run()Ljava/lang/Object;+53
   J 1613 c1 java.security.AccessController.doPrivileged(Ljava/security/PrivilegedAction;)Ljava/lang/Object; java.base@21.0.5 (9 bytes) @ 0x0000ffff9bb92fd0 [0x0000ffff9bb92e00+0x00000000000001d0]
   j  io.netty.util.internal.NativeLibraryLoader.loadLibraryByHelper(Ljava/lang/Class;Ljava/lang/String;Z)V+10
   j  io.netty.util.internal.NativeLibraryLoader.loadLibrary(Ljava/lang/ClassLoader;Ljava/lang/String;Z)V+15
   j  io.netty.util.internal.NativeLibraryLoader.load(Ljava/lang/String;Ljava/lang/ClassLoader;)V+352
   j  io.netty.util.internal.NativeLibraryLoader.loadFirstAvailable(Ljava/lang/ClassLoader;[Ljava/lang/String;)V+33
   j  io.netty.handler.ssl.OpenSsl.loadTcNative()V+297
   j  io.netty.handler.ssl.OpenSsl.<clinit>()V+170
   v  ~StubRoutines::call_stub 0x0000ffffa2d99130
   V  [libjvm.so+0x83f4e8]  JavaCalls::call_helper(JavaValue*, methodHandle const&, JavaCallArguments*, JavaThread*)+0x218
   V  [libjvm.so+0x819560]  InstanceKlass::call_class_initializer(JavaThread*)+0x280
   V  [libjvm.so+0x81b214]  InstanceKlass::initialize_impl(JavaThread*)+0x5a4
   V  [libjvm.so+0xa71540]  LinkResolver::resolve_static_call(CallInfo&, LinkInfo const&, bool, JavaThread*)+0x1a0
   V  [libjvm.so+0xa71cf0]  LinkResolver::resolve_invoke(CallInfo&, Handle, constantPoolHandle const&, int, Bytecodes::Code, JavaThread*)+0x1e0
   V  [libjvm.so+0x837358]  InterpreterRuntime::resolve_invoke(JavaThread*, Bytecodes::Code)+0x228
   V  [libjvm.so+0x8377b8]  InterpreterRuntime::resolve_from_cache(JavaThread*, Bytecodes::Code)+0x108
   j  io.netty.handler.ssl.SslContext.defaultProvider()Lio/netty/handler/ssl/SslProvider;+0
   j  io.netty.handler.ssl.SslContext.defaultServerProvider()Lio/netty/handler/ssl/SslProvider;+0
   j  io.netty.handler.ssl.SslContext.newServerContextInternal(Lio/netty/handler/ssl/SslProvider;Ljava/security/Provider;[Ljava/security/cert/X509Certificate;Ljavax/net/ssl/TrustManagerFactory;[Ljava/security/cert/X509Certificate;Ljava/security/PrivateKey;Ljava/lang/String;Ljavax/net/ssl/KeyManagerFactory;Ljava/lang/Iterable;Lio/netty/handler/ssl/CipherSuiteFilter;Lio/netty/handler/ssl/ApplicationProtocolConfig;JJLio/netty/handler/ssl/ClientAuth;[Ljava/lang/String;ZZLjava/security/SecureRandom;Ljava/lang/String;[Ljava/util/Map$Entry;)Lio/netty/handler/ssl/SslContext;+4
   j  io.netty.handler.ssl.SslContextBuilder.build()Lio/netty/handler/ssl/SslContext;+101
   j  org.apache.pulsar.common.util.SecurityUtility.createNettySslContextForServer(Lio/netty/handler/ssl/SslProvider;ZLjava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/util/Set;Ljava/util/Set;Z)Lio/netty/handler/ssl/SslContext;+123
   j  org.apache.pulsar.common.util.DefaultPulsarSslFactory.buildNettySslContext()Lio/netty/handler/ssl/SslContext;+279
   j  org.apache.pulsar.common.util.DefaultPulsarSslFactory.createInternalSslContext()V+60
   j  org.apache.pulsar.broker.service.PulsarChannelInitializer.<init>(Lorg/apache/pulsar/broker/PulsarService;Lorg/apache/pulsar/broker/service/PulsarChannelInitializer$PulsarChannelOptions;)V+87
   j  org.apache.pulsar.broker.service.PulsarChannelInitializer$$Lambda+0x00000088004fa480.newPulsarChannelInitializer(Lorg/apache/pulsar/broker/PulsarService;Lorg/apache/pulsar/broker/service/PulsarChannelInitializer$PulsarChannelOptions;)Lorg/apache/pulsar/broker/service/PulsarChannelInitializer;+6
   j  org.apache.pulsar.broker.service.BrokerService.start()V+201
   ```
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari closed issue #23717: [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library
URL: https://github.com/apache/pulsar/issues/23717


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2548188459

   @BewareMyPower It looks like the ldd test fails on linux/amd64 already with 2.0.66.Final version. I added a comment in https://github.com/netty/netty-tcnative/issues/907#issuecomment-2548185401


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537933636

   I tried to reproduce this issue inside an `alpine:3.20` container. It seems that the `netty-tcnative-boringssl-static` 2.0.66 can find the `__getauxval` symbol after installing the `gcompat` dependency, while 2.0.69 cannot.
   
   ```
   ~ # unzip -q netty-tcnative-boringssl-static-2.0.66.Final-linux-aarch_64.jar 
   ~ # ldd META-INF/native/libnetty_tcnative_linux_aarch_64.so 
   	/lib/ld-musl-aarch64.so.1 (0xffffa4a08000)
   	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffffa4a08000)
   Error loading shared library libcrypt.so.1: No such file or directory (needed by META-INF/native/libnetty_tcnative_linux_aarch_64.so)
   	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffffa4a08000)
   	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffffa4a08000)
   Error relocating META-INF/native/libnetty_tcnative_linux_aarch_64.so: __getauxval: symbol not found
   ~ # apk add gcompat
   (1/3) Installing musl-obstack (1.2.3-r2)
   (2/3) Installing libucontext (1.2-r3)
   (3/3) Installing gcompat (1.1.0-r4)
   OK: 138 MiB in 29 packages
   ~ # ldd META-INF/native/libnetty_tcnative_linux_aarch_64.so 
   	/lib/ld-musl-aarch64.so.1 (0xffffbac74000)
   	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffffbac74000)
   	libcrypt.so.1 => /lib/libcrypt.so.1 (0xffffbaa44000)
   	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffffbac74000)
   	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffffbac74000)
   	libucontext.so.1 => /lib/libucontext.so.1 (0xffffbaa23000)
   	libobstack.so.1 => /usr/lib/libobstack.so.1 (0xffffbaa02000)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2538150514

   @lhotari Your Dockerfile works so I think we can apply your fix for now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2548242000

   @BewareMyPower It looks like it's necessary to preload gcompat on Alpine (for example with `ENV LD_PRELOAD=/lib/lib/libgcompat.so.0` in Dockerfile). More details in https://github.com/netty/netty-tcnative/issues/907#issuecomment-2548228536 .
   The first source to mention this that I found is https://github.com/grpc/grpc-java/issues/8751#issuecomment-1112549548 .
   
   I doubt that this would play nicely with the glibc solution we have in Pulsar.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537957288

   @BewareMyPower We don't have a pure musl base image. glibc in https://github.com/apache/pulsar/tree/master/docker/glibc-package gets added to the Alpine base mixed, making it a mixed musl and glibc environment which is not generally recommended.
   
   https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/24647#note_176723 states it clearly:
   _"Combining glibc and musl runtimes is basically all but guaranteed to create an unstable environment, unless the system is appropriately configured (glibc side uses glibc binaries only, and vice versa)."_
   
   https://github.com/sgerrand/alpine-pkg-glibc/issues/194#issuecomment-1510140972 mentions:
   _"You really need everything pure musl, or everything musl (but works with gcompat or libc6-compat shim-like tools...) or everything glibc using this package)"_ 
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537966131

   > I opened an issue at the upstream side: [netty/netty-tcnative#907](https://github.com/netty/netty-tcnative/issues/907), which includes a minimum reproduced steps on `alpine:3.20`.
   > 
   > It seems that the missed `__getauxval` symbol can be found from the `gcompat` dependency for 2.0.66 but not for 2.0.69.
   
   For Alpine base images, I think that it's necessary to have `gcompat`, `libc6-compat`, `libuuid` and possibly also `libgcc` packages installed to load netty-tcnative. @BewareMyPower Does it reproduce with all those packages installed?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537961679

   I opened an issue at the upstream side: https://github.com/netty/netty-tcnative/issues/907
   
   It seems that the missed `__getauxval` symbol can be found from the `gcompat` dependency for 2.0.66 but not for 2.0.69.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537960440

   This gist contains a Dockerfile for building a minideb based Pulsar 4.0.1 image by copying the content from the Pulsar 4.0.1 image: https://gist.github.com/lhotari/3ffef8117743f7044e6bbdc3933bc029 . 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537860114

   The Netty JNI is incorrectly linked:
   
   ```bash
   cf43f09c95f1:/tmp$ ldd libnetty_tcnative_linux_aarch_6412030080574647118807.so 
   	/lib/ld-musl-aarch64.so.1 (0xffff9501c000)
   	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffff9501c000)
   	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffff9501c000)
   	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xffff94dbc000)
   	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff9501c000)
   Error relocating libnetty_tcnative_linux_aarch_6412030080574647118807.so: __getauxval: symbol not found
   ```
   
   From [here](https://github.com/grpc/grpc-java/issues/10096#issuecomment-1520411572),
   
   > musl refuses to tell you "symbol can't be found" and instead just crashes. With glibc these issues are caught and handled. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537979594

   > ```
   > ~ # apk add libc6-compat libuuid libgcc
   > (1/1) Installing libuuid (2.40.1-r1)
   > OK: 138 MiB in 30 packages
   > ~ # ldd META-INF/native/libnetty_tcnative_linux_aarch_64.so 
   > 	/lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   > 	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   > 	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   > 	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xffff9bdf3000)
   > 	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   > Error relocating META-INF/native/libnetty_tcnative_linux_aarch_64.so: __getauxval: symbol not found
   > ```
   > 
   > These extra dependencies don't work. Actually for 2.0.66, I only need to install `gcc` and `gcompact` to make it work
   
   Yes, the problem reproduces. I agree that this is a netty-tcnative issue with musl compatibility.
   
   
   btw. I guess instead of `gcc`, that `libgcc` is preferred for 2.0.66 so that the image doesn't have a compiler installed. 
   
   ```shell
   apk update
   apk add gcompat libgcc
   wget https://repo1.maven.org/maven2/io/netty/netty-tcnative-boringssl-static/2.0.66.Final/netty-tcnative-boringssl-static-2.0.66.Final-linux-aarch_64.jar
   unzip netty-tcnative-boringssl-static-2.0.66.Final-linux-aarch_64.jar
   ldd META-INF/native/libnetty_tcnative_linux_aarch_64.so
   ```
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537970852

   ```
    => WARN: InvalidDefaultArgInFrom: Default value for ARG ${PULSAR_IMAGE} results in empty or invalid base image name (line 10)                                                                                                                  0.0s
    => CANCELED [internal] load metadata for docker.io/bitnami/minideb:bookworm                                                                                                                                                                    0.0s
   Dockerfile:10
   --------------------
      8 |     
      9 |     # First create a stage with the original Pulsar image
     10 | >>> FROM ${PULSAR_IMAGE} AS pulsar
     11 |     
     12 |     ###  Create one stage to include JVM distribution
   --------------------
   ERROR: failed to solve: failed to parse stage name "apachepulsar/pulsar-all=4.0.1": invalid reference format
   ```
   
   I'm trying to build this Dockerfile but it failed: https://gist.github.com/lhotari/3ffef8117743f7044e6bbdc3933bc029


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2556537781

   @BewareMyPower I found out that adding `-e LD_PRELOAD=/lib/libgcompat.so.0` to the docker command line in pulsar-client-python's `build-support/pulsar-test-service-start.sh` script works around the issue. This is addressed in #23762 for future releases of the docker image. For existing docker images, it's a suitable workaround.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537969595

   ```
   ~ # apk add libc6-compat libuuid libgcc
   (1/1) Installing libuuid (2.40.1-r1)
   OK: 138 MiB in 30 packages
   ~ # ldd META-INF/native/libnetty_tcnative_linux_aarch_64.so 
   	/lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xffff9bdf3000)
   	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff9c053000)
   Error relocating META-INF/native/libnetty_tcnative_linux_aarch_64.so: __getauxval: symbol not found
   ```
   
   These extra dependencies don't work. Actually for 2.0.66, I only need to install `gcc` and `gcompact` to make it work


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537980537

   > I'm trying to build this Dockerfile but it failed: https://gist.github.com/lhotari/3ffef8117743f7044e6bbdc3933bc029
   
   I accidentially introduced a typo while updating the version to 4.0.1 . It's fixed now.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2538362554

   ```
   test                                                               latest      b8ab7278d588   2 hours ago     3.38GB
   apachepulsar/pulsar                                                4.0.1       93b5506ac8eb   12 days ago     481MB
   ```
   
   Though the image size is much bigger now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "BewareMyPower (via GitHub)" <gi...@apache.org>.
BewareMyPower commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537871642

   Yes, it's a bug of Netty but I'm not sure if it's the same issue. It should be an issue with the JNI library of netty-tcnative-boringssl-static.
   
   It can be verified by executing the following command after you start a container:
   
   ```bash
   cd /tmp
   unzip -q /pulsar/lib/io.netty-netty-tcnative-boringssl-static-*-aarch_64.jar 
   ldd META-INF/native/libnetty_tcnative_linux_aarch_64.so 
   ```
   
   With 4.0.0, the JAR version is 2.0.66 and the link is good:
   
   ```
   	/lib/ld-musl-aarch64.so.1 (0xffff8670a000)
   	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffff8670a000)
   	libcrypt.so.1 => /lib/libcrypt.so.1 (0xffff864da000)
   	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffff8670a000)
   	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffff8670a000)
   	libucontext.so.1 => /lib/libucontext.so.1 (0xffff864b9000)
   	libobstack.so.1 => /usr/lib/libobstack.so.1 (0xffff86498000)
   ```
   
   However, with 4.0.1, the JAR version is 2.0.69 and the link is broken so it crashes on musl Linux:
   
   ```
   	/lib/ld-musl-aarch64.so.1 (0xffffb161c000)
   	librt.so.1 => /lib/ld-musl-aarch64.so.1 (0xffffb161c000)
   	libdl.so.2 => /lib/ld-musl-aarch64.so.1 (0xffffb161c000)
   	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0xffffb13bc000)
   	libc.so.6 => /lib/ld-musl-aarch64.so.1 (0xffffb161c000)
   Error relocating META-INF/native/libnetty_tcnative_linux_aarch_64.so: __getauxval: symbol not found
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "mattisonchao (via GitHub)" <gi...@apache.org>.
mattisonchao commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2537863964

   That might be a netty bug.  https://github.com/netty/netty/issues/14479


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Bug] Pulsar 4.0.1 image crashed when loading the native SSL library [pulsar]

Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on issue #23717:
URL: https://github.com/apache/pulsar/issues/23717#issuecomment-2539439036

   > ```
   > test                                                               latest      b8ab7278d588   2 hours ago     3.38GB
   > apachepulsar/pulsar                                                4.0.1       93b5506ac8eb   12 days ago     481MB
   > ```
   > 
   > Though the image size is much bigger now.
   
   @BewareMyPower  The example Dockerfile is for pulsar-all image. There's a comment how to pass the Pulsar image as a build arg. Passing `--build-arg PULSAR_IMAGE=apachepulsar/pulsar:4.0.1` would solve this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org