You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "Nicholas DiPiazza (Jira)" <ji...@apache.org> on 2020/08/31 15:40:00 UTC
[jira] [Resolved] (TIKA-3173) Tika server with spawnChild - server
does not recover from OOM until an additional file comes in
[ https://issues.apache.org/jira/browse/TIKA-3173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nicholas DiPiazza resolved TIKA-3173.
-------------------------------------
Resolution: Won't Fix
> Tika server with spawnChild - server does not recover from OOM until an additional file comes in
> ------------------------------------------------------------------------------------------------
>
> Key: TIKA-3173
> URL: https://issues.apache.org/jira/browse/TIKA-3173
> Project: Tika
> Issue Type: Improvement
> Reporter: Nicholas DiPiazza
> Priority: Major
>
> On latest Tika 1.x latest branch, on Windows 10 with latest patches.
> First start a Tika server with spawnChild enabled.
> Next run the following test:
> (This test is available for run here https://github.com/nddipiazza/tika/blob/4a094ab310691a68d1c8d270cee4e138eb43a1ed/tika-server/src/test/java/org/apache/tika/server/RecoverFromOutOfMemoryTest.java)
> Step 1) Run a normal word doc through the /rmeta/text endpoint.
> Step 2) Run a file that is known to crash the tika parser.
> Step 3) Pause, give it several minutes to recover.
> Step 4) Run another normal word doc through the /rmeta/text endpoint.
> Step 5) Run another normal word doc through the /rmeta/text endpoint.
> The Step (2) will hang and finally fail with a socket timeout.
> The logs at this point show this:
> {code}
> INFO 2020-08-17 22:53:27,255 [pool-1-thread-1] (TikaServerCli.java:131) - Starting Apache Tika server
> INFO 2020-08-17 22:53:27,276 [pool-1-thread-1] (TikaServerWatchDog.java:69) - server watch dog is starting up
> Aug 17, 2020 10:53:28 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
> WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
> See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
> for optional dependencies.
> Aug 17, 2020 10:53:28 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
> WARNING: org.xerial's sqlite-jdbc is not loaded.
> Please provide the jar on your classpath to parse sqlite files.
> See tika-parsers/pom.xml for the correct version.
> INFO 2020-08-17 22:53:28,766 [main] (RecoverFromOutOfMemoryTest.java:125) - Still waiting for Tika server on port 54976 to start. Retries remaining: 9
> INFO 2020-08-17 22:53:29,148 [main] (TikaServerCli.java:131) - Starting Apache Tika server
> INFO 2020-08-17 22:53:29,845 [main] (ServerImpl.java:94) - Setting the server's publish address to be http://192.168.1.39:54976/
> INFO 2020-08-17 22:53:29,952 [main] (Log.java:169) - Logging initialized @2608ms to org.eclipse.jetty.util.log.Slf4jLog
> INFO 2020-08-17 22:53:30,082 [main] (Server.java:359) - jetty-9.4.27.v20200227; built: 2020-02-27T18:37:21.340Z; git: a304fd9f351f337e7c0e2a7c28878dd536149c6c; jvm 1.8.0_242-b08
> INFO 2020-08-17 22:53:30,406 [main] (AbstractConnector.java:331) - Started ServerConnector@46271dd6{HTTP/1.1, (http/1.1)}{192.168.1.39:54976}
> INFO 2020-08-17 22:53:30,406 [main] (Server.java:399) - Started @3063ms
> WARN 2020-08-17 22:53:30,419 [main] (ContextHandler.java:1534) - Empty contextPath
> INFO 2020-08-17 22:53:30,443 [main] (ContextHandler.java:828) - Started o.e.j.s.h.ContextHandler@54eb2b70{/,null,AVAILABLE}
> INFO 2020-08-17 22:53:30,443 [main] (TikaServerCli.java:352) - Started Apache Tika server at http://192.168.1.39:54976/
> INFO 2020-08-17 22:53:31,525 [qtp1923999715-18] (TikaResource.java:439) - rmeta/text (application/msword)
> INFO 2020-08-17 22:53:31,797 [main] (RecoverFromOutOfMemoryTest.java:111) - Normal doc response: [{"Author":"Maxim Valyanskiy","Content-Type":"application/msword","Creation-Date":"2009-06-11T13:53:04Z","Last-Modified":"1601-01-01T00:00:00Z","Last-Printed":"1601-01-01T00:00:00Z","Last-Save-Date":"1601-01-01T00:00:00Z","Revision-Number":"0","X-Parsed-By":["org.apache.tika.parser.DefaultParser","org.apache.tika.parser.microsoft.OfficeParser"],"X-TIKA:content":"\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\ntest\n\n","X-TIKA:content_handler":"ToTextContentHandler","X-TIKA:embedded_depth":"0","X-TIKA:parse_time_millis":"210","cp:revision":"0","creator":"Maxim Valyanskiy","date":"1601-01-01T00:00:00Z","dc:creator":"Maxim Valyanskiy","dcterms:created":"2009-06-11T13:53:04Z","dcterms:modified":"1601-01-01T00:00:00Z","meta:author":"Maxim Valyanskiy","meta:creation-date":"2009-06-11T13:53:04Z","meta:print-date":"1601-01-01T00:00:00Z","meta:save-date":"1601-01-01T00:00:00Z","modified":"1601-01-01T00:00:00Z"}]
> INFO 2020-08-17 22:53:31,814 [qtp1923999715-16] (TikaResource.java:439) - rmeta/text (application/vnd.ms-excel)
> WARN 2020-08-17 22:54:32,204 [main] (LogUtils.java:475) - Interceptor for {http://192.168.1.39:54976/rmeta/text}WebClient has thrown exception, unwinding now
> org.apache.cxf.interceptor.Fault: Could not receive Message.
> at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:65)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.jaxrs.client.AbstractClient.doRunInterceptorChain(AbstractClient.java:701)
> at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1086)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:932)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:901)
> at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:364)
> at org.apache.cxf.jaxrs.client.WebClient.put(WebClient.java:382)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.parseBombFile(RecoverFromOutOfMemoryTest.java:96)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.testOutOfMemory(RecoverFromOutOfMemoryTest.java:79)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
> at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> Caused by: java.net.SocketTimeoutException: SocketTimeoutException invoking http://192.168.1.39:54976/rmeta/text: Read timed out
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1400)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1384)
> at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77)
> at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
> at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:671)
> at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63)
> ... 35 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:377)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:373)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.getResponseCode(URLConnectionHTTPConduit.java:373)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.doProcessResponseCode(HTTPConduit.java:1597)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1625)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1570)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1371)
> ... 39 more
> INFO 2020-08-17 22:54:36,186 [main] (RecoverFromOutOfMemoryTest.java:82) - Could not parse the bomb file!
> javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: SocketTimeoutException invoking http://192.168.1.39:54976/rmeta/text: Read timed out
> at org.apache.cxf.jaxrs.client.AbstractClient.checkClientException(AbstractClient.java:629)
> at org.apache.cxf.jaxrs.client.AbstractClient.preProcessResult(AbstractClient.java:605)
> at org.apache.cxf.jaxrs.client.WebClient.doResponse(WebClient.java:1150)
> at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1087)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:932)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:901)
> at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:364)
> at org.apache.cxf.jaxrs.client.WebClient.put(WebClient.java:382)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.parseBombFile(RecoverFromOutOfMemoryTest.java:96)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.testOutOfMemory(RecoverFromOutOfMemoryTest.java:79)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
> at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> Caused by: java.net.SocketTimeoutException: SocketTimeoutException invoking http://192.168.1.39:54976/rmeta/text: Read timed out
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1400)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1384)
> at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77)
> at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
> at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:671)
> at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.jaxrs.client.AbstractClient.doRunInterceptorChain(AbstractClient.java:701)
> at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1086)
> ... 32 more
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> at java.net.SocketInputStream.read(SocketInputStream.java:171)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:377)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:373)
> at java.security.AccessController.doPrivileged(Native Method)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.getResponseCode(URLConnectionHTTPConduit.java:373)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.doProcessResponseCode(HTTPConduit.java:1597)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1625)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1570)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1371)
> ... 39 more
> WARN 2020-08-17 22:54:36,595 [qtp1923999715-16] (TikaResource.java:419) - rmeta/text: OOM (null)
> java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3664)
> at java.lang.String.<init>(String.java:207)
> at java.lang.String.substring(String.java:1969)
> at java.lang.String.trim(String.java:2878)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.addTextCell(ExcelExtractor.java:594)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:469)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:367)
> at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener$TikaFormatTrackingHSSFListener.processRecord(ExcelExtractor.java:689)
> at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:106)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:172)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:129)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:343)
> at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:172)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:183)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at org.apache.tika.parser.RecursiveParserWrapper.parse(RecursiveParserWrapper.java:234)
> at org.apache.tika.server.resource.TikaResource.parse(TikaResource.java:409)
> at org.apache.tika.server.resource.RecursiveMetadataResource.parseMetadata(RecursiveMetadataResource.java:158)
> at org.apache.tika.server.resource.RecursiveMetadataResource.getMetadata(RecursiveMetadataResource.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> WARN 2020-08-17 22:54:36,619 [qtp1923999715-16] (LogUtils.java:475) - Application {http://resource.server.tika.apache.org/}MetadataResource has thrown exception, unwinding now
> org.apache.cxf.interceptor.Fault: Java heap space
> at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
> at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
> at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247)
> at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3664)
> at java.lang.String.<init>(String.java:207)
> at java.lang.String.substring(String.java:1969)
> at java.lang.String.trim(String.java:2878)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.addTextCell(ExcelExtractor.java:594)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:469)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:367)
> at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener$TikaFormatTrackingHSSFListener.processRecord(ExcelExtractor.java:689)
> at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:106)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:172)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:129)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:343)
> at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:172)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:183)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at org.apache.tika.parser.RecursiveParserWrapper.parse(RecursiveParserWrapper.java:234)
> at org.apache.tika.server.resource.TikaResource.parse(TikaResource.java:409)
> at org.apache.tika.server.resource.RecursiveMetadataResource.parseMetadata(RecursiveMetadataResource.java:158)
> at org.apache.tika.server.resource.RecursiveMetadataResource.getMetadata(RecursiveMetadataResource.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> WARN 2020-08-17 22:54:36,623 [qtp1923999715-16] (PhaseInterceptorChain.java:501) - Exception in handleFault on interceptor org.apache.cxf.jaxrs.interceptor.JAXRSDefaultFaultOutInterceptor@5eee8836
> org.apache.cxf.interceptor.Fault: Java heap space
> at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
> at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
> at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247)
> at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3664)
> at java.lang.String.<init>(String.java:207)
> at java.lang.String.substring(String.java:1969)
> at java.lang.String.trim(String.java:2878)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.addTextCell(ExcelExtractor.java:594)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:469)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:367)
> at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener$TikaFormatTrackingHSSFListener.processRecord(ExcelExtractor.java:689)
> at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:106)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:172)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:129)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:343)
> at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:172)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:183)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at org.apache.tika.parser.RecursiveParserWrapper.parse(RecursiveParserWrapper.java:234)
> at org.apache.tika.server.resource.TikaResource.parse(TikaResource.java:409)
> at org.apache.tika.server.resource.RecursiveMetadataResource.parseMetadata(RecursiveMetadataResource.java:158)
> at org.apache.tika.server.resource.RecursiveMetadataResource.getMetadata(RecursiveMetadataResource.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> ERROR 2020-08-17 22:54:36,624 [qtp1923999715-16] (AbstractFaultChainInitiatorObserver.java:114) - An unexpected error occurred during error handling. No further error processing will occur.
> org.apache.cxf.interceptor.Fault: Java heap space
> at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
> at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
> at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247)
> at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3664)
> at java.lang.String.<init>(String.java:207)
> at java.lang.String.substring(String.java:1969)
> at java.lang.String.trim(String.java:2878)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.addTextCell(ExcelExtractor.java:594)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:469)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:367)
> at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener$TikaFormatTrackingHSSFListener.processRecord(ExcelExtractor.java:689)
> at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:106)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:172)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:129)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:343)
> at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:172)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:183)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at org.apache.tika.parser.RecursiveParserWrapper.parse(RecursiveParserWrapper.java:234)
> at org.apache.tika.server.resource.TikaResource.parse(TikaResource.java:409)
> at org.apache.tika.server.resource.RecursiveMetadataResource.parseMetadata(RecursiveMetadataResource.java:158)
> at org.apache.tika.server.resource.RecursiveMetadataResource.getMetadata(RecursiveMetadataResource.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> WARN 2020-08-17 22:54:36,624 [qtp1923999715-16] (HttpChannel.java:591) - /rmeta/text
> org.apache.cxf.interceptor.Fault: Java heap space
> at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
> at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267)
> at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:247)
> at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:79)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:190)
> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
> at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.OutOfMemoryError: Java heap space
> at java.util.Arrays.copyOfRange(Arrays.java:3664)
> at java.lang.String.<init>(String.java:207)
> at java.lang.String.substring(String.java:1969)
> at java.lang.String.trim(String.java:2878)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.addTextCell(ExcelExtractor.java:594)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:469)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:367)
> at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener$TikaFormatTrackingHSSFListener.processRecord(ExcelExtractor.java:689)
> at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:106)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:172)
> at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:129)
> at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:343)
> at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:172)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:183)
> at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:131)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
> at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
> at org.apache.tika.parser.RecursiveParserWrapper.parse(RecursiveParserWrapper.java:234)
> at org.apache.tika.server.resource.TikaResource.parse(TikaResource.java:409)
> at org.apache.tika.server.resource.RecursiveMetadataResource.parseMetadata(RecursiveMetadataResource.java:158)
> at org.apache.tika.server.resource.RecursiveMetadataResource.getMetadata(RecursiveMetadataResource.java:123)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179)
> at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201)
> at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104)
> at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
> WARN 2020-08-17 22:54:36,765 [Thread-1] (ServerStatusWatcher.java:196) - child process observed ERROR and is shutting down.
> INFO 2020-08-17 22:54:36,767 [Thread-1] (ServerStatusWatcher.java:181) - Shutting down child process with status: ERROR
> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
> {code}
> So far so good. The logs indicate the child process has recognized the issue.
> So after the delay, we perform step (4) and run another working document. Unfortunatley, this parse fails. Here are the logs from that attempt:
> {code}
> WARN 2020-08-17 22:57:00,543 [Thread-1] (TikaServerWatchDog.java:128) - 148488 ms have elapsed since last successful ping. Destroying child now
> WARN 2020-08-17 22:57:00,544 [pool-1-thread-1] (TikaServerWatchDog.java:290) - Received non-operating status from child: ERROR
> Aug 17, 2020 10:57:01 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
> WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
> See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
> for optional dependencies.
> Aug 17, 2020 10:57:01 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
> WARNING: org.xerial's sqlite-jdbc is not loaded.
> Please provide the jar on your classpath to parse sqlite files.
> See tika-parsers/pom.xml for the correct version.
> INFO 2020-08-17 22:57:02,300 [main] (TikaServerCli.java:131) - Starting Apache Tika server
> INFO 2020-08-17 22:57:02,373 [pool-1-thread-1] (TikaServerWatchDog.java:83) - Successfully restarted child process -- 0 restarts so far)
> ERROR 2020-08-17 22:57:02,588 [main] (JAXRSUtils.java:1889) - Problem with writing the data, class java.io.BufferedInputStream, ContentType: application/msword
> WARN 2020-08-17 22:57:02,588 [main] (LogUtils.java:475) - Interceptor for {http://192.168.1.39:54976/rmeta/text}WebClient has thrown exception, unwinding now
> org.apache.cxf.interceptor.Fault: Problem with writing the data, class java.io.BufferedInputStream, ContentType: application/msword
> at org.apache.cxf.jaxrs.client.WebClient$BodyWriter.doWriteBody(WebClient.java:1228)
> at org.apache.cxf.jaxrs.client.AbstractClient$AbstractBodyWriter.handleMessage(AbstractClient.java:1233)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.jaxrs.client.AbstractClient.doRunInterceptorChain(AbstractClient.java:701)
> at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1086)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:932)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:901)
> at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:364)
> at org.apache.cxf.jaxrs.client.WebClient.put(WebClient.java:382)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.assertCanParseNormalDoc(RecoverFromOutOfMemoryTest.java:102)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.testOutOfMemory(RecoverFromOutOfMemoryTest.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
> at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> Caused by: javax.ws.rs.ProcessingException: Problem with writing the data, class java.io.BufferedInputStream, ContentType: application/msword
> at org.apache.cxf.jaxrs.client.AbstractClient.reportMessageHandlerProblem(AbstractClient.java:844)
> at org.apache.cxf.jaxrs.client.AbstractClient.writeBody(AbstractClient.java:529)
> at org.apache.cxf.jaxrs.client.WebClient$BodyWriter.doWriteBody(WebClient.java:1223)
> ... 36 more
> Caused by: java.net.ConnectException: Connection refused: connect
> at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
> at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:607)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
> at sun.net.www.http.HttpClient.New(HttpClient.java:339)
> at sun.net.www.http.HttpClient.New(HttpClient.java:357)
> at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
> at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1205)
> at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
> at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
> at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1340)
> at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1315)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.setupWrappedStream(URLConnectionHTTPConduit.java:274)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleHeadersTrustCaching(HTTPConduit.java:1343)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.onFirstWrite(HTTPConduit.java:1304)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.onFirstWrite(URLConnectionHTTPConduit.java:307)
> at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:47)
> at org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
> at org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:63)
> at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:51)
> at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:193)
> at org.apache.cxf.helpers.IOUtils.copyAndCloseInput(IOUtils.java:159)
> at org.apache.cxf.jaxrs.provider.BinaryDataProvider.copyInputToOutput(BinaryDataProvider.java:205)
> at org.apache.cxf.jaxrs.provider.BinaryDataProvider.writeTo(BinaryDataProvider.java:159)
> at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody(JAXRSUtils.java:1472)
> at org.apache.cxf.jaxrs.client.AbstractClient.writeBody(AbstractClient.java:515)
> ... 37 more
> INFO 2020-08-17 22:57:02,590 [pool-1-thread-1] (TikaServerWatchDog.java:99) - about to shutdown process
> javax.ws.rs.ProcessingException: Problem with writing the data, class java.io.BufferedInputStream, ContentType: application/msword
> at org.apache.cxf.jaxrs.client.AbstractClient.reportMessageHandlerProblem(AbstractClient.java:844)
> at org.apache.cxf.jaxrs.client.AbstractClient.writeBody(AbstractClient.java:529)
> at org.apache.cxf.jaxrs.client.WebClient$BodyWriter.doWriteBody(WebClient.java:1223)
> at org.apache.cxf.jaxrs.client.AbstractClient$AbstractBodyWriter.handleMessage(AbstractClient.java:1233)
> at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
> at org.apache.cxf.jaxrs.client.AbstractClient.doRunInterceptorChain(AbstractClient.java:701)
> at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1086)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:932)
> at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:901)
> at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:364)
> at org.apache.cxf.jaxrs.client.WebClient.put(WebClient.java:382)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.assertCanParseNormalDoc(RecoverFromOutOfMemoryTest.java:102)
> at org.apache.tika.server.RecoverFromOutOfMemoryTest.testOutOfMemory(RecoverFromOutOfMemoryTest.java:89)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
> at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:230)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:58)
> Caused by: java.net.ConnectException: Connection refused: connect
> at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
> at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:607)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
> at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
> at sun.net.www.http.HttpClient.New(HttpClient.java:339)
> at sun.net.www.http.HttpClient.New(HttpClient.java:357)
> at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
> at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1205)
> at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
> at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
> at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1340)
> at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1315)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.setupWrappedStream(URLConnectionHTTPConduit.java:274)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleHeadersTrustCaching(HTTPConduit.java:1343)
> at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.onFirstWrite(HTTPConduit.java:1304)
> at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.onFirstWrite(URLConnectionHTTPConduit.java:307)
> at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:47)
> at org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
> at org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:63)
> at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:51)
> at org.apache.cxf.helpers.IOUtils.copy(IOUtils.java:193)
> at org.apache.cxf.helpers.IOUtils.copyAndCloseInput(IOUtils.java:159)
> at org.apache.cxf.jaxrs.provider.BinaryDataProvider.copyInputToOutput(BinaryDataProvider.java:205)
> at org.apache.cxf.jaxrs.provider.BinaryDataProvider.writeTo(BinaryDataProvider.java:159)
> at org.apache.cxf.jaxrs.utils.JAXRSUtils.writeMessageBody(JAXRSUtils.java:1472)
> at org.apache.cxf.jaxrs.client.AbstractClient.writeBody(AbstractClient.java:515)
> ... 37 more
> {code}
> The tika server saw the issue, took action by restarting, but we get an extra connection refused in the meantime.
> Finally, we run step (5) happens and the server is back up and able to parse again at this point.
> Should the Watchdog have detected tika was dead in the water and restarted automatically to avoid the need to have additional doc fail parse before getting back online?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)