You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "Richard Goodman (Jira)" <ji...@apache.org> on 2025/03/11 15:18:00 UTC
[jira] [Commented] (BIGTOP-4289) Fix Hadoop build on Fedora 40
[ https://issues.apache.org/jira/browse/BIGTOP-4289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17934252#comment-17934252 ]
Richard Goodman commented on BIGTOP-4289:
-----------------------------------------
I've been trying to build Hadoop 3.3.6 locally and was running into a similar issue. I have OpenSSL 3.4.1 on the device building hadoop. Looking into the code, and from researching into OpenSSL 3.4.1, the threading that `num_locks` is complaining about, is only relevant for OpenSSL >=1.1.0, more can be read [here|https://github.com/openssl/openssl/blob/OpenSSL_1_1_1d/include/openssl/crypto.h#L212]. I applied the following patch locally, and it did allow this part to successfully build. I'm not expert on C code or OpenSSL, and purely basing it off what I was reading on OpenSSL release notes.
[^openssl.patch]
> Fix Hadoop build on Fedora 40
> -----------------------------
>
> Key: BIGTOP-4289
> URL: https://issues.apache.org/jira/browse/BIGTOP-4289
> Project: Bigtop
> Issue Type: Bug
> Components: hadoop
> Reporter: Kengo Seki
> Priority: Major
> Attachments: openssl.patch
>
>
> I tried to build Hadoop 3.3.6 on Fedora 40 and came across the following errors.
> {code}
> $ cat /etc/redhat-release
> Fedora release 40 (Forty)
> $ gcc --version
> gcc (GCC) 14.2.1 20240912 (Red Hat 14.2.1-3)
> Copyright (C) 2024 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> $ openssl version
> OpenSSL 3.2.2 4 Jun 2024 (Library: OpenSSL 3.2.2 4 Jun 2024)
> $ ./gradlew hadoop-clean hadoop-pkg
> ...
> [WARNING] /home/vagrant/bigtop/build/hadoop/rpm/BUILD/hadoop-3.3.6-src/hadoop-common-project/hadoop-common/src/main/native/src/test/org/apache/hadoop/io/erasurecode/erasure_code_test.c: In function ‘main’:
> [WARNING] /home/vagrant/bigtop/build/hadoop/rpm/BUILD/hadoop-3.3.6-src/hadoop-common-project/hadoop-common/src/main/native/src/test/org/apache/hadoop/io/erasurecode/erasure_code_test.c:116:7: error: implicit declaration of function ‘dumpDecoder’ [-Wimplicit-function-declaration]
> [WARNING] 116 | dumpDecoder(pDecoder);
> [WARNING] | ^~~~~~~~~~~
> [WARNING] make[2]: *** [CMakeFiles/erasure_code_test.dir/build.make:146: CMakeFiles/erasure_code_test.dir/main/native/src/test/org/apache/hadoop/io/erasurecode/erasure_code_test.c.o] Error 1
> [WARNING] make[1]: *** [CMakeFiles/Makefile2:89: CMakeFiles/erasure_code_test.dir/all] Error 2
> [WARNING] make[1]: *** Waiting for unfinished jobs....
> [WARNING] /home/vagrant/bigtop/build/hadoop/rpm/BUILD/hadoop-3.3.6-src/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c: In function ‘locks_setup’:
> [WARNING] /home/vagrant/bigtop/build/hadoop/rpm/BUILD/hadoop-3.3.6-src/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:250:33: error: implicit declaration of function ‘dlsym_CRYPTO_num_locks’; did you mean ‘dlsym_CRYPTO_malloc’? [-Wimplicit-function-declaration]
> [WARNING] 250 | lock_cs = dlsym_CRYPTO_malloc(dlsym_CRYPTO_num_locks() * \
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~
> [WARNING] | dlsym_CRYPTO_malloc
> [WARNING] /home/vagrant/bigtop/build/hadoop/rpm/BUILD/hadoop-3.3.6-src/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:257:3: error: implicit declaration of function ‘dlsym_CRYPTO_set_id_callback’; did you mean ‘CRYPTO_set_id_callback’? [-Wimplicit-function-declaration]
> [WARNING] 257 | dlsym_CRYPTO_set_id_callback((unsigned long (*)())pthreads_thread_id);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] | CRYPTO_set_id_callback
> [WARNING] /home/vagrant/bigtop/build/hadoop/rpm/BUILD/hadoop-3.3.6-src/hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/crypto/random/OpensslSecureRandom.c:258:3: error: implicit declaration of function ‘dlsym_CRYPTO_set_locking_callback’; did you mean ‘CRYPTO_set_locking_callback’? [-Wimplicit-function-declaration]
> [WARNING] 258 | dlsym_CRYPTO_set_locking_callback((void (*)())pthreads_locking_callback);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] | CRYPTO_set_locking_callback
> ...
> {code}
> HADOOP-19180 includes the fix for the first dumpDecoder thing, but others don't seem to be addressed in upstream, at least as of the 3.4.1 release. [~iwasakims] Could you take a look at it?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)