You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "AiBe Gee (JIRA)" <ji...@apache.org> on 2019/05/13 05:26:00 UTC
[jira] [Updated] (HADOOP-16311) Hadoop build failure - natively on
ARM (armv7) - oom_listener_main.c issues
[ https://issues.apache.org/jira/browse/HADOOP-16311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
AiBe Gee updated HADOOP-16311:
------------------------------
Summary: Hadoop build failure - natively on ARM (armv7) - oom_listener_main.c issues (was: Hadoop build failure - natively on ARM (armv7) - oom_listener_test_main.cc issues)
> Hadoop build failure - natively on ARM (armv7) - oom_listener_main.c issues
> ---------------------------------------------------------------------------
>
> Key: HADOOP-16311
> URL: https://issues.apache.org/jira/browse/HADOOP-16311
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
> Affects Versions: 3.2.0
> Environment: Linux ARM (armv7)
> Reporter: AiBe Gee
> Priority: Major
>
> After failing to build Hadoop 3.3.0 - latest git snapshot, described in:
> https://issues.apache.org/jira/browse/HADOOP-16309
> I went on and tried my luck with the 3.2.0 release, only to find out that I'm unable to build this version too. Environment is the same as described in:
> https://issues.apache.org/jira/browse/HADOOP-16309
> Just the source archive is different:
> https://archive.apache.org/dist/hadoop/common/hadoop-3.2.0/hadoop-3.2.0-src.tar.gz
> Hadoop build failure log snippet:
> [WARNING] make[2]: Leaving directory '/kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native'
> [WARNING] [ 44%] Built target container
> [WARNING] [ 47%] Linking CXX static library libgtest.a
> [WARNING] /usr/bin/cmake -P CMakeFiles/gtest.dir/cmake_clean_target.cmake
> [WARNING] /usr/bin/cmake -E cmake_link_script CMakeFiles/gtest.dir/link.txt --verbose=1
> [WARNING] /usr/bin/ar qc libgtest.a CMakeFiles/gtest.dir/kit/hadoop-3.2.0-src/hadoop-common-project/hadoop-common/src/main/native/gtest/gtest-all.cc.o
> [WARNING] /usr/bin/ranlib libgtest.a
> [WARNING] make[2]: Leaving directory '/kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native'
> [WARNING] [ 47%] Built target gtest
> [WARNING] make[1]: Leaving directory '/kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native'
> [WARNING] /usr/bin/ld: CMakeFiles/oom-listener.dir/main/native/oom-listener/impl/oom_listener_main.c.o: in function `main':
> *[WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c:89: undefined reference to `cleanup'*
> [WARNING] collect2: error: ld returned 1 exit status
> [WARNING] make[2]: *** [CMakeFiles/oom-listener.dir/build.make:99: target/usr/local/bin/oom-listener] Error 1
> [WARNING] make[1]: *** [CMakeFiles/Makefile2:222: CMakeFiles/oom-listener.dir/all] Error 2
> [WARNING] make[1]: *** Waiting for unfinished jobs....
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.c: In function ‘handle_gpu_request’:
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.c:196:9: warning: ‘strncpy’ specified bound 128 equals destination size [-Wstringop-truncation]
> [WARNING] 196 | strncpy(container_id, optarg, MAX_CONTAINER_ID_LEN);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/fpga/fpga-module.c: In function ‘handle_fpga_request’:
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/fpga/fpga-module.c:196:9: warning: ‘strncpy’ specified bound 128 equals destination size [-Wstringop-truncation]
> [WARNING] 196 | strncpy(container_id, optarg, MAX_CONTAINER_ID_LEN);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c: In function ‘normalize_mount’:
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:1041:9: warning: ‘strncpy’ output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
> [WARNING] 1041 | strncpy(ret_ptr, real_mount, len);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:1034:20: note: length computed here
> [WARNING] 1034 | size_t len = strlen(real_mount);
> [WARNING] | ^~~~~~~~~~~~~~~~~~
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c: In function ‘check_trusted_image’:
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:128:9: warning: ‘strncpy’ output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
> [WARNING] 128 | strncpy(registry_ptr, privileged_registry[i], len);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/utils/docker-util.c:125:17: note: length computed here
> [WARNING] 125 | int len = strlen(privileged_registry[i]);
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c: In function ‘mount_cgroup’:
> [WARNING] /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c:2445:19: warning: ‘stpncpy’ output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
> [WARNING] 2445 | char *buf = stpncpy(hier_path, mount_path, strlen(mount_path));
> [WARNING] | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> [WARNING] make: *** [Makefile:84: all] Error 2
> [INFO] ------------------------------------------------------------------------
> [INFO] Reactor Summary for Apache Hadoop Main 3.2.0:
> [INFO]
> [INFO] Apache Hadoop Main ................................. SUCCESS [ 9.998 s]
> ...
> [INFO] Apache Hadoop YARN Server Common ................... SUCCESS [03:56 min]
> [INFO] Apache Hadoop YARN NodeManager ..................... FAILURE [04:08 min]
> [INFO] Apache Hadoop YARN Web Proxy ....................... SKIPPED
> ...
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 03:03 h
> [INFO] Finished at: 2019-05-13T05:06:13+03:00
> [INFO] ------------------------------------------------------------------------
> [ERROR] Failed to execute goal org.apache.hadoop:hadoop-maven-plugins:3.2.0:cmake-compile (cmake-compile) on project hadoop-yarn-server-nodemanager: make failed with error code 2 -> [Help 1]
> Did some investigation, went for a manual compilation of the affected source tree:
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native# make -j 4 V=1
> Scanning dependencies of target oom-listener
> [ 5%] Built target gtest
> [ 38%] Built target container
> [ 41%] Building C object CMakeFiles/oom-listener.dir/main/native/oom-listener/impl/oom_listener_main.c.o
> [ 44%] Building CXX object CMakeFiles/test-oom-listener.dir/main/native/oom-listener/test/oom_listener_test_main.cc.o
> [ 50%] Built target container-executor
> [ 52%] Building CXX object CMakeFiles/cetest.dir/main/native/container-executor/test/utils/test_docker_util.cc.o
> [ 58%] Built target test-container-executor
> [ 61%] Linking C executable target/usr/local/bin/oom-listener
> /usr/bin/ld: CMakeFiles/oom-listener.dir/main/native/oom-listener/impl/oom_listener_main.c.o: in function `main':
> */kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c:89: undefined reference to `cleanup'*
> collect2: error: ld returned 1 exit status
> make[2]: *** [CMakeFiles/oom-listener.dir/build.make:99: target/usr/local/bin/oom-listener] Error 1
> make[1]: *** [CMakeFiles/Makefile2:222: CMakeFiles/oom-listener.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc: In member function ‘virtual void OOMListenerTest_test_oom_Test::TestBody()’:
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7: error: ‘__WAIT_STATUS’ was not declared in this scope; did you mean ‘ADJ_STATUS’?
> 256 | __WAIT_STATUS mem_hog_status = {};
> | ^~~~~~~~~~~~~
> | ADJ_STATUS
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30: error: ‘mem_hog_status’ was not declared in this scope
> 257 | __pid_t exited0 = wait(mem_hog_status);
> | ^~~~~~~~~~~~~~
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:20: error: expected ‘;’ before ‘oom_listener_status’
> 275 | __WAIT_STATUS oom_listener_status = {};
> | ^~~~~~~~~~~~~~~~~~~~
> | ;
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30: error: ‘oom_listener_status’ was not declared in this scope; did you mean ‘oom_listener’?
> 276 | __pid_t exited1 = wait(oom_listener_status);
> | ^~~~~~~~~~~~~~~~~~~
> | oom_listener
> make[2]: *** [CMakeFiles/test-oom-listener.dir/build.make:76: CMakeFiles/test-oom-listener.dir/main/native/oom-listener/test/oom_listener_test_main.cc.o] Error 1
> make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/test-oom-listener.dir/all] Error 2
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc: In member function ‘char* ContainerExecutor::TestDockerUtil::flatten(args*)’:
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/utils/test_docker_util.cc:107:20 warning: ‘char* strncpy(char*, const char*, size_t)’ output truncated before terminating nul copying 1 byte from a string of the same length [-Wstringop-truncation]
> 107 | strncpy(buffer + current_len, " ", 1);
> | ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> make[2]: *** wait: No child processes. Stop.
> make[2]: *** Waiting for unfinished jobs....
> Then at line 89 in
> hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> - commented:
> /* cleanup(&descriptors); */
> Tried a manual compilation again and got into more errors:
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/native# make -j 4 V=1
> Scanning dependencies of target oom-listener
> [ 5%] Built target gtest
> [ 8%] Building C object CMakeFiles/oom-listener.dir/main/native/oom-listener/impl/oom_listener_main.c.o
> [ 41%] Built target container
> [ 44%] Building CXX object CMakeFiles/test-oom-listener.dir/main/native/oom-listener/test/oom_listener_test_main.cc.o
> [ 50%] Built target container-executor
> [ 52%] Building CXX object CMakeFiles/cetest.dir/main/native/container-executor/test/utils/test_docker_util.cc.o
> [ 58%] Built target test-container-executor
> [ 61%] Linking C executable target/usr/local/bin/oom-listener
> [ 63%] Built target oom-listener
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc: In member function ‘virtual void OOMListenerTest_test_oom_Test::TestBody()’:
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7: error: ‘__WAIT_STATUS’ was not declared in this scope; did you mean ‘ADJ_STATUS’?
> 256 | __WAIT_STATUS mem_hog_status = {};
> | ^~~~~~~~~~~~~
> | ADJ_STATUS
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30: error: ‘mem_hog_status’ was not declared in this scope
> 257 | __pid_t exited0 = wait(mem_hog_status);
> | ^~~~~~~~~~~~~~
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:20: error: expected ‘;’ before ‘oom_listener_status’
> 275 | __WAIT_STATUS oom_listener_status = {};
> | ^~~~~~~~~~~~~~~~~~~~
> | ;
> /kit/hadoop-3.2.0-src/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30: error: ‘oom_listener_status’ was not declared in this scope; did you mean ‘oom_listener’?
> 276 | __pid_t exited1 = wait(oom_listener_status);
> | ^~~~~~~~~~~~~~~~~~~
> | oom_listener
> make[2]: *** [CMakeFiles/test-oom-listener.dir/build.make:76: CMakeFiles/test-oom-listener.dir/main/native/oom-listener/test/oom_listener_test_main.cc.o] Error 1
> make[1]: *** [CMakeFiles/Makefile2:73: CMakeFiles/test-oom-listener.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org