You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "ankitsultana (via GitHub)" <gi...@apache.org> on 2023/06/12 07:05:03 UTC
[GitHub] [pinot] ankitsultana opened a new issue, #10894: JDK Bug Causes Error Segment after Server Restart
ankitsultana opened a new issue, #10894:
URL: https://github.com/apache/pinot/issues/10894
I have been looking into issues with realtime tables and server restarts. On looking into a recent issue where one of the segments ran into the error state, I found the following stack trace.
This looks like a JDK bug. The exception was thrown within ~7 seconds of server process start-up, so I suspect that if we put in a retry with some delay things may work. We could potentially put the retry in `buildCloseableHttpClient`.
```
2023-06-12 05:54:01.122 [HelixTaskExecutor-message_handle_thread_17] ERROR o.a.helix.messaging.handling.HelixStateTransitionHandler - Exception while executing a state transition task table__134__1339__20230611T0357Z
--
java.lang.reflect.InvocationTargetException: null
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:350)
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:278)
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97)
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NullPointerException: null
at java.base/sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:133)
at java.base/sun.net.www.protocol.jar.JarURLConnection.getInputStream(JarURLConnection.java:155)
at java.base/java.net.URL.openStream(URL.java:1165)
at java.base/java.lang.ClassLoader.getResourceAsStream(ClassLoader.java:1739)
at org.apache.http.util.VersionInfo.loadVersionInfo(VersionInfo.java:233)
at org.apache.http.util.VersionInfo.getUserAgent(VersionInfo.java:319)
at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:1057)
at org.apache.pinot.common.utils.http.HttpClient.buildCloseableHttpClient(HttpClient.java:482)
at org.apache.pinot.common.utils.http.HttpClient.<init>(HttpClient.java:101)
at org.apache.pinot.common.utils.FileUploadDownloadClient.<init>(FileUploadDownloadClient.java:138)
at org.apache.pinot.server.realtime.ServerSegmentCompletionProtocolHandler.<init>(ServerSegmentCompletionProtocolHandler.java:82)
at org.apache.pinot.core.data.manager.realtime.SegmentBuildTimeLeaseExtender.<init>(SegmentBuildTimeLeaseExtender.java:97)
at org.apache.pinot.core.data.manager.realtime.SegmentBuildTimeLeaseExtender.lambda$getOrCreate$0(SegmentBuildTimeLeaseExtender.java:83)
at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1947)
at org.apache.pinot.core.data.manager.realtime.SegmentBuildTimeLeaseExtender.getOrCreate(SegmentBuildTimeLeaseExtender.java:81)
at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.doInit(RealtimeTableDataManager.java:140)
at org.apache.pinot.core.data.manager.BaseTableDataManager.init(BaseTableDataManager.java:167)
at org.apache.pinot.core.data.manager.offline.TableDataManagerProvider.getTableDataManager(TableDataManagerProvider.java:93)
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.createTableDataManager(HelixInstanceDataManager.java:225)
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.lambda$addRealtimeSegment$3(HelixInstanceDataManager.java:216)
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1737)
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addRealtimeSegment(HelixInstanceDataManager.java:216)
at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:168)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana closed issue #10894: JDK Bug Causes Error Segment after Server Restart
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana closed issue #10894: JDK Bug Causes Error Segment after Server Restart
URL: https://github.com/apache/pinot/issues/10894
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana commented on issue #10894: JDK Bug Causes Error Segment after Server Restart
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on issue #10894:
URL: https://github.com/apache/pinot/issues/10894#issuecomment-1586744150
The error is coming from the `VersionInfo.getUserAgent` call.
One option to fix this is to set `defaultUserAgentDisabled` to true or set the system property `http.agent`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] ankitsultana commented on issue #10894: JDK Bug Causes Error Segment after Server Restart
Posted by "ankitsultana (via GitHub)" <gi...@apache.org>.
ankitsultana commented on issue #10894:
URL: https://github.com/apache/pinot/issues/10894#issuecomment-1592085776
Fixed by #10895.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org