You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/01/30 23:54:08 UTC
[GitHub] [incubator-pinot] dongxiaoman opened a new issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
dongxiaoman opened a new issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509
Our Pinot hosts were upgraded from Ubuntu 16.0 to 20.0, and some of the segments are having trouble loading up. Below are the error logs:
```
2021/01/28 19:06:55.356 ERROR [HelixStateTransitionHandler] [HelixTaskExecutor-message_handle_thread] Exception while executing a state transition task pe_71_2018-03-02_2018-05-30_9
java.lang.reflect.InvocationTargetException: null
at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_275]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_275]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331) [pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
Caused by: java.lang.UnsatisfiedLinkError: /tmp/larray-7aadb555.lib: /tmp/larray-7aadb555.lib: failed to map segment from shared object
at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[?:1.8.0_275]
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934) ~[?:1.8.0_275]
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817) ~[?:1.8.0_275]
at java.lang.Runtime.load0(Runtime.java:810) ~[?:1.8.0_275]
at java.lang.System.load(System.java:1088) ~[?:1.8.0_275]
at xerial.larray.impl.LArrayLoader.load(LArrayLoader.java:86) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at xerial.larray.impl.LArrayNative.<clinit>(LArrayNative.java:26) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at xerial.larray.mmap.MMapBuffer.<init>(MMapBuffer.java:94) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.memory.PinotNonNativeOrderLBuffer.mapFile(PinotNonNativeOrderLBuffer.java:49) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.memory.PinotDataBuffer.mapFile(PinotDataBuffer.java:197) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SingleFileIndexDirectory.mapAndSliceFile(SingleFileIndexDirectory.java:288) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SingleFileIndexDirectory.mapBufferEntries(SingleFileIndexDirectory.java:259) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SingleFileIndexDirectory.load(SingleFileIndexDirectory.java:192) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SingleFileIndexDirectory.<init>(SingleFileIndexDirectory.java:92) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SegmentLocalFSDirectory.loadData(SegmentLocalFSDirectory.java:204) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SegmentLocalFSDirectory.load(SegmentLocalFSDirectory.java:185) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SegmentLocalFSDirectory.<init>(SegmentLocalFSDirectory.java:74) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.store.SegmentDirectory.createFromLocalFS(SegmentDirectory.java:108) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.segment.index.loader.SegmentPreProcessor.<init>(SegmentPreProcessor.java:75) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.indexsegment.immutable.ImmutableSegmentLoader.load(ImmutableSegmentLoader.java:98) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.pinot.core.data.manager.offline.OfflineTableDataManager.addSegment(OfflineTableDataManager.java:52) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
```
Another error is like this:
```
2021/01/30 23:38:43.091 ERROR [HelixStateTransitionHandler] [HelixTaskExecutor-message_handle_thread] Exception while executing a state transition task pe_21_2018-03-02_2018-05-30_9
java.lang.reflect.InvocationTargetException: null
at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source) ~[?:?]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_275]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_275]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331) [pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_275]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_275]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_275]
Caused by: java.lang.NoClassDefFoundError: Could not initialize class xerial.larray.impl.LArrayNative
at xerial.larray.mmap.MMapBuffer.<init>(MMapBuffer.java:94) ~[pinot-all-0.6.0-2020-11-17-30bc38177-SNAPSHOT-jar-with-dependencies.jar:0.6.0-2020-11-17-30bc38177-SNAPSHOT-30bc38177850c6349709c0d87b3b3ac334bf1078]
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] dongxiaoman commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-774823322
@fx19880617 Just verified, still the same error with java 11
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-775467156
@dongxiaoman Find a similar error here, can you please try the solution: https://support.datastax.com/hc/en-us/articles/115000550123-Graph-loader-throws-failed-to-map-segment-from-shared-object-warning
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] dongxiaoman commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-775479500
@Jackie-Jiang Awesome finding! It solved the problem in our QA host.
Will roll out to prod and verify in a larger scale
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-771222590
hmm, does it work with java 11?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-771222590
hmm, does it work with java 11?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] dongxiaoman closed issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
dongxiaoman closed issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] dongxiaoman commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-778474811
Close it now since it is running fine now.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] dongxiaoman commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
dongxiaoman commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-771006968
Tried upgrade to `xerial.larray` latest version (4.2.1) , still not working
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [incubator-pinot] fx19880617 commented on issue #6509: Mmap lib issue after upgrading from Ubuntu 16.0 to 20.0
Posted by GitBox <gi...@apache.org>.
fx19880617 commented on issue #6509:
URL: https://github.com/apache/incubator-pinot/issues/6509#issuecomment-770522403
@Jackie-Jiang ^^
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org