You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/02/02 14:52:49 UTC

[GitHub] [incubator-pinot] pabrahamusa commented on issue #6522: Failed to send lease extension for table - Pinot Stopped

pabrahamusa commented on issue #6522:
URL: https://github.com/apache/incubator-pinot/issues/6522#issuecomment-771689647


   After restart I can see different error also , seems like something not correct with segements. This happened when I accessed the table via controller UI.
   
   ```
   my-pinot-controller-0 controller 2021/02/02 14:48:42.466 WARN [MultiGetRequest] [restapi-multiget-thread-1] Caught 'java.net.SocketTimeoutException: Read timed out' while executing GET on URL: http://my-pinot-server-2.my-pinot-server-headless.logging.svc.cluster.local:8097/table/mylog_REALTIME/size
   my-pinot-controller-0 controller 2021/02/02 14:48:42.467 ERROR [CompletionServiceHelper] [grizzly-http-server-2] Connection error
   my-pinot-controller-0 controller java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: Read timed out
   my-pinot-controller-0 controller 	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.util.concurrent.FutureTask.get(FutureTask.java:192) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at org.apache.pinot.controller.util.CompletionServiceHelper.doMultiGetRequest(CompletionServiceHelper.java:65) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.pinot.controller.api.resources.ServerTableSizeReader.getSegmentSizeInfoFromServers(ServerTableSizeReader.java:69) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.pinot.controller.util.TableSizeReader.getTableSubtypeSize(TableSizeReader.java:156) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.pinot.controller.util.TableSizeReader.getTableSizeDetails(TableSizeReader.java:100) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.pinot.controller.api.resources.TableSize.getTableSize(TableSize.java:75) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:391) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:80) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:253) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:679) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:353) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549) [pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
   my-pinot-controller-0 controller Caused by: java.net.SocketTimeoutException: Read timed out
   my-pinot-controller-0 controller 	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.io.BufferedInputStream.read(BufferedInputStream.java:265) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:78) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:106) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1116) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1413) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1973) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at org.apache.pinot.common.http.MultiGetRequest.lambda$execute$0(MultiGetRequest.java:106) ~[pinot-all-0.7.0-SNAPSHOT-jar-with-dependencies.jar:0.7.0-SNAPSHOT-b4be817b1f078a4644995ad2be9ccb0eccbf6b29]
   my-pinot-controller-0 controller 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_282]
   my-pinot-controller-0 controller 	... 1 more
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org