You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by Mark Jens <ma...@gmail.com> on 2021/12/09 08:31:09 UTC
Build of NativeMap broken ?!
Hi,
I've noticed in the logs that Accumulo native library is not available:
TabletServer_1410060415.out:307:2021-12-08T14:09:24,257 [tserver.NativeMap]
ERROR: Tried and failed to load Accumulo native library from
/home/ubuntu/git/accumulo/server/native/target/accumulo-native-2.1.0-SNAPSHOT/accumulo-native-2.1.0-SNAPSHOT:/usr/java/packages/lib:/usr/lib/aarch64-linux-gnu/jni:/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
Initially I thought that it might be an issue related to ARM64 and I
checked the build of server/native:
]
[INFO] --- exec-maven-plugin:3.0.0:exec (test-native-libs) @
accumulo-native ---
g++ -g -fPIC -shared -O3 -Wall
-I'/usr/lib/jvm/java-11-openjdk-arm64'/include
-I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
nativeMap/org_apache_accumulo_tserver_NativeMap.cc:19:10: fatal error:
org_apache_accumulo_tserver_NativeMap.h: No such file or directory
19 | #include "org_apache_accumulo_tserver_NativeMap.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make: *** [Makefile:61: libaccumulo.so] Error 1
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 2
(Exit value: 2)
at org.apache.commons.exec.DefaultExecutor.executeInternal
(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute
(DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
(ExecMojo.java:982)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
(ExecMojo.java:929)
https://github.com/apache/accumulo/blob/2fd40dd8fd68c94bad6fb887d6b1cd99d39481c3/server/native/src/main/c%2B%2B/nativeMap/org_apache_accumulo_tserver_NativeMap.cc#L19
says: #include "org_apache_accumulo_tserver_NativeMap.h"
but there is no such file in the same folder!
There is
https://github.com/apache/accumulo/blob/2fd40dd8fd68c94bad6fb887d6b1cd99d39481c3/server/native/src/main/c%2B%2B/nativeMap/NativeMap.h
i.e. the include must be just "NativeMap.h", right ?!
But even after fixing this it still fails:
$ make
g++ -g -fPIC -shared -O3 -Wall
-I'/usr/lib/jvm/java-11-openjdk-arm64'/include
-I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
nativeMap/org_apache_accumulo_tserver_NativeMap.cc: In function ‘void
Java_org_apache_accumulo_tserver_NativeMap_singleUpdate(JNIEnv*, jclass,
jlong, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong, jboolean,
jbyteArray, jint)’:
nativeMap/org_apache_accumulo_tserver_NativeMap.cc:47:15: error:
‘Java_org_apache_accumulo_tserver_NativeMap_startUpdate’ was not declared
in this scope; did you mean
‘Java_org_apache_accumulo_tserver_NativeMap_singleUpdate’?
47 | jlong uid =
Java_org_apache_accumulo_tserver_NativeMap_startUpdate(env, cls, nm, r);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
Java_org_apache_accumulo_tserver_NativeMap_singleUpdate
nativeMap/org_apache_accumulo_tserver_NativeMap.cc:48:3: error:
‘Java_org_apache_accumulo_tserver_NativeMap_update’ was not declared in
this scope; did you mean
‘Java_org_apache_accumulo_tserver_NativeMap_createNM’?
48 | Java_org_apache_accumulo_tserver_NativeMap_update(env, cls, nm,
uid, cf, cq, cv, ts, del, val, mutationCount);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Java_org_apache_accumulo_tserver_NativeMap_createNM
make: *** [Makefile:61: libaccumulo.so] Error 1
The method names are JNI namespaced.
How is this supposed to work ?
Cheers,
Mark
Re: Build of NativeMap broken ?!
Posted by Mark Jens <ma...@gmail.com>.
All is fine!Sorry for the noise!
".../server/native$ mvn clean package" passed fine! It seems without
"clean" it was using old assembly without
org_apache_accumulo_tserver_NativeMap.h
$ file
target/accumulo-native-2.1.0-SNAPSHOT/accumulo-native-2.1.0-SNAPSHOT/libaccumulo.so
target/accumulo-native-2.1.0-SNAPSHOT/accumulo-native-2.1.0-SNAPSHOT/libaccumulo.so:
ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically
linked, BuildID[sha1]=78645cdb0614469bd8b04a74db85fb552e3475ff, with
debug_info, not stripped
On Thu, 9 Dec 2021 at 10:46, Mark Jens <ma...@gmail.com> wrote:
> OK, I see that the missing org_apache_accumulo_tserver_NativeMap.h is
> created by tserver module.
> component.xml refers to by relative path and I have 'mvn clean'-ed the
> whole project before trying to build the native module.
>
> Now I built tserver module and re-run 'mvn package' for native. The new
> error is:
>
> INFO] --- exec-maven-plugin:3.0.0:exec (test-native-libs) @
> accumulo-native ---
> g++ -g -fPIC -shared -O3 -Wall
> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include
> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
> libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc: In function ‘void
> Java_org_apache_accumulo_tserver_NativeMap_singleUpdate(JNIEnv*, jclass,
> jlong, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong, jboolean,
> jbyteArray, jint)’:
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:46:15: error:
> ‘Java_org_apache_accumulo_tserver_NativeMap_startUpdate’ was not declared
> in this scope; did you mean
> ‘Java_org_apache_accumulo_tserver_NativeMap_singleUpdate’?
> 46 | jlong uid =
> Java_org_apache_accumulo_tserver_NativeMap_startUpdate(env, cls, nm, r);
> |
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> |
> Java_org_apache_accumulo_tserver_NativeMap_singleUpdate
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:47:3: error:
> ‘Java_org_apache_accumulo_tserver_NativeMap_update’ was not declared in
> this scope; did you mean
> ‘Java_org_apache_accumulo_tserver_NativeMap_createNM’?
> 47 | Java_org_apache_accumulo_tserver_NativeMap_update(env, cls, nm,
> uid, cf, cq, cv, ts, del, val, mutationCount);
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> | Java_org_apache_accumulo_tserver_NativeMap_createNM
> make: *** [Makefile:61: libaccumulo.so] Error 1
> [ERROR] Command execution failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 2
> (Exit value: 2)
> at org.apache.commons.exec.DefaultExecutor.executeInternal
> (DefaultExecutor.java:404)
> at org.apache.commons.exec.DefaultExecutor.execute
> (DefaultExecutor.java:166)
> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
> (ExecMojo.java:982)
> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
> (ExecMojo.java:929)
>
> I will investigate further!
>
>
> On Thu, 9 Dec 2021 at 10:31, Mark Jens <ma...@gmail.com> wrote:
>
>> Hi,
>>
>> I've noticed in the logs that Accumulo native library is not available:
>>
>> TabletServer_1410060415.out:307:2021-12-08T14:09:24,257
>> [tserver.NativeMap] ERROR: Tried and failed to load Accumulo native library
>> from
>> /home/ubuntu/git/accumulo/server/native/target/accumulo-native-2.1.0-SNAPSHOT/accumulo-native-2.1.0-SNAPSHOT:/usr/java/packages/lib:/usr/lib/aarch64-linux-gnu/jni:/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
>>
>> Initially I thought that it might be an issue related to ARM64 and I
>> checked the build of server/native:
>>
>> ]
>> [INFO] --- exec-maven-plugin:3.0.0:exec (test-native-libs) @
>> accumulo-native ---
>> g++ -g -fPIC -shared -O3 -Wall
>> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include
>> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
>> libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
>> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:19:10: fatal error:
>> org_apache_accumulo_tserver_NativeMap.h: No such file or directory
>> 19 | #include "org_apache_accumulo_tserver_NativeMap.h"
>> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> compilation terminated.
>> make: *** [Makefile:61: libaccumulo.so] Error 1
>> [ERROR] Command execution failed.
>> org.apache.commons.exec.ExecuteException: Process exited with an error: 2
>> (Exit value: 2)
>> at org.apache.commons.exec.DefaultExecutor.executeInternal
>> (DefaultExecutor.java:404)
>> at org.apache.commons.exec.DefaultExecutor.execute
>> (DefaultExecutor.java:166)
>> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
>> (ExecMojo.java:982)
>> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
>> (ExecMojo.java:929)
>>
>>
>>
>> https://github.com/apache/accumulo/blob/2fd40dd8fd68c94bad6fb887d6b1cd99d39481c3/server/native/src/main/c%2B%2B/nativeMap/org_apache_accumulo_tserver_NativeMap.cc#L19
>>
>> says: #include "org_apache_accumulo_tserver_NativeMap.h"
>>
>> but there is no such file in the same folder!
>>
>> There is
>> https://github.com/apache/accumulo/blob/2fd40dd8fd68c94bad6fb887d6b1cd99d39481c3/server/native/src/main/c%2B%2B/nativeMap/NativeMap.h
>> i.e. the include must be just "NativeMap.h", right ?!
>>
>> But even after fixing this it still fails:
>>
>> $ make
>> g++ -g -fPIC -shared -O3 -Wall
>> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include
>> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
>> libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
>> nativeMap/org_apache_accumulo_tserver_NativeMap.cc: In function ‘void
>> Java_org_apache_accumulo_tserver_NativeMap_singleUpdate(JNIEnv*, jclass,
>> jlong, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong, jboolean,
>> jbyteArray, jint)’:
>> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:47:15: error:
>> ‘Java_org_apache_accumulo_tserver_NativeMap_startUpdate’ was not declared
>> in this scope; did you mean
>> ‘Java_org_apache_accumulo_tserver_NativeMap_singleUpdate’?
>> 47 | jlong uid =
>> Java_org_apache_accumulo_tserver_NativeMap_startUpdate(env, cls, nm, r);
>> |
>> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> |
>> Java_org_apache_accumulo_tserver_NativeMap_singleUpdate
>> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:48:3: error:
>> ‘Java_org_apache_accumulo_tserver_NativeMap_update’ was not declared in
>> this scope; did you mean
>> ‘Java_org_apache_accumulo_tserver_NativeMap_createNM’?
>> 48 | Java_org_apache_accumulo_tserver_NativeMap_update(env, cls, nm,
>> uid, cf, cq, cv, ts, del, val, mutationCount);
>> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>> | Java_org_apache_accumulo_tserver_NativeMap_createNM
>> make: *** [Makefile:61: libaccumulo.so] Error 1
>>
>> The method names are JNI namespaced.
>>
>> How is this supposed to work ?
>>
>> Cheers,
>> Mark
>>
>
Re: Build of NativeMap broken ?!
Posted by Mark Jens <ma...@gmail.com>.
OK, I see that the missing org_apache_accumulo_tserver_NativeMap.h is
created by tserver module.
component.xml refers to by relative path and I have 'mvn clean'-ed the
whole project before trying to build the native module.
Now I built tserver module and re-run 'mvn package' for native. The new
error is:
INFO] --- exec-maven-plugin:3.0.0:exec (test-native-libs) @ accumulo-native
---
g++ -g -fPIC -shared -O3 -Wall
-I'/usr/lib/jvm/java-11-openjdk-arm64'/include
-I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
nativeMap/org_apache_accumulo_tserver_NativeMap.cc: In function ‘void
Java_org_apache_accumulo_tserver_NativeMap_singleUpdate(JNIEnv*, jclass,
jlong, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong, jboolean,
jbyteArray, jint)’:
nativeMap/org_apache_accumulo_tserver_NativeMap.cc:46:15: error:
‘Java_org_apache_accumulo_tserver_NativeMap_startUpdate’ was not declared
in this scope; did you mean
‘Java_org_apache_accumulo_tserver_NativeMap_singleUpdate’?
46 | jlong uid =
Java_org_apache_accumulo_tserver_NativeMap_startUpdate(env, cls, nm, r);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
Java_org_apache_accumulo_tserver_NativeMap_singleUpdate
nativeMap/org_apache_accumulo_tserver_NativeMap.cc:47:3: error:
‘Java_org_apache_accumulo_tserver_NativeMap_update’ was not declared in
this scope; did you mean
‘Java_org_apache_accumulo_tserver_NativeMap_createNM’?
47 | Java_org_apache_accumulo_tserver_NativeMap_update(env, cls, nm,
uid, cf, cq, cv, ts, del, val, mutationCount);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Java_org_apache_accumulo_tserver_NativeMap_createNM
make: *** [Makefile:61: libaccumulo.so] Error 1
[ERROR] Command execution failed.
org.apache.commons.exec.ExecuteException: Process exited with an error: 2
(Exit value: 2)
at org.apache.commons.exec.DefaultExecutor.executeInternal
(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.execute
(DefaultExecutor.java:166)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
(ExecMojo.java:982)
at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
(ExecMojo.java:929)
I will investigate further!
On Thu, 9 Dec 2021 at 10:31, Mark Jens <ma...@gmail.com> wrote:
> Hi,
>
> I've noticed in the logs that Accumulo native library is not available:
>
> TabletServer_1410060415.out:307:2021-12-08T14:09:24,257
> [tserver.NativeMap] ERROR: Tried and failed to load Accumulo native library
> from
> /home/ubuntu/git/accumulo/server/native/target/accumulo-native-2.1.0-SNAPSHOT/accumulo-native-2.1.0-SNAPSHOT:/usr/java/packages/lib:/usr/lib/aarch64-linux-gnu/jni:/lib/aarch64-linux-gnu:/usr/lib/aarch64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
>
> Initially I thought that it might be an issue related to ARM64 and I
> checked the build of server/native:
>
> ]
> [INFO] --- exec-maven-plugin:3.0.0:exec (test-native-libs) @
> accumulo-native ---
> g++ -g -fPIC -shared -O3 -Wall
> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include
> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
> libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:19:10: fatal error:
> org_apache_accumulo_tserver_NativeMap.h: No such file or directory
> 19 | #include "org_apache_accumulo_tserver_NativeMap.h"
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> compilation terminated.
> make: *** [Makefile:61: libaccumulo.so] Error 1
> [ERROR] Command execution failed.
> org.apache.commons.exec.ExecuteException: Process exited with an error: 2
> (Exit value: 2)
> at org.apache.commons.exec.DefaultExecutor.executeInternal
> (DefaultExecutor.java:404)
> at org.apache.commons.exec.DefaultExecutor.execute
> (DefaultExecutor.java:166)
> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
> (ExecMojo.java:982)
> at org.codehaus.mojo.exec.ExecMojo.executeCommandLine
> (ExecMojo.java:929)
>
>
>
> https://github.com/apache/accumulo/blob/2fd40dd8fd68c94bad6fb887d6b1cd99d39481c3/server/native/src/main/c%2B%2B/nativeMap/org_apache_accumulo_tserver_NativeMap.cc#L19
>
> says: #include "org_apache_accumulo_tserver_NativeMap.h"
>
> but there is no such file in the same folder!
>
> There is
> https://github.com/apache/accumulo/blob/2fd40dd8fd68c94bad6fb887d6b1cd99d39481c3/server/native/src/main/c%2B%2B/nativeMap/NativeMap.h
> i.e. the include must be just "NativeMap.h", right ?!
>
> But even after fixing this it still fails:
>
> $ make
> g++ -g -fPIC -shared -O3 -Wall
> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include
> -I'/usr/lib/jvm/java-11-openjdk-arm64'/include/linux -Ijavah -o
> libaccumulo.so nativeMap/org_apache_accumulo_tserver_NativeMap.cc
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc: In function ‘void
> Java_org_apache_accumulo_tserver_NativeMap_singleUpdate(JNIEnv*, jclass,
> jlong, jbyteArray, jbyteArray, jbyteArray, jbyteArray, jlong, jboolean,
> jbyteArray, jint)’:
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:47:15: error:
> ‘Java_org_apache_accumulo_tserver_NativeMap_startUpdate’ was not declared
> in this scope; did you mean
> ‘Java_org_apache_accumulo_tserver_NativeMap_singleUpdate’?
> 47 | jlong uid =
> Java_org_apache_accumulo_tserver_NativeMap_startUpdate(env, cls, nm, r);
> |
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> |
> Java_org_apache_accumulo_tserver_NativeMap_singleUpdate
> nativeMap/org_apache_accumulo_tserver_NativeMap.cc:48:3: error:
> ‘Java_org_apache_accumulo_tserver_NativeMap_update’ was not declared in
> this scope; did you mean
> ‘Java_org_apache_accumulo_tserver_NativeMap_createNM’?
> 48 | Java_org_apache_accumulo_tserver_NativeMap_update(env, cls, nm,
> uid, cf, cq, cv, ts, del, val, mutationCount);
> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> | Java_org_apache_accumulo_tserver_NativeMap_createNM
> make: *** [Makefile:61: libaccumulo.so] Error 1
>
> The method names are JNI namespaced.
>
> How is this supposed to work ?
>
> Cheers,
> Mark
>