You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by "Karl Wright (JIRA)" <ji...@apache.org> on 2018/07/27 00:30:00 UTC

[jira] [Commented] (CONNECTORS-1518) MCF shutting down when Tika is used

    [ https://issues.apache.org/jira/browse/CONNECTORS-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16559082#comment-16559082 ] 

Karl Wright commented on CONNECTORS-1518:
-----------------------------------------

Hi [~svanschalkwyk], the memory usage for the ElasticSearch connector seemingly depends on whether the mapper attachment is used or not.  Here's the code.  Note that everything is properly streamed when the mapper attachment is used, but is dumped into a string buffer when not:

{code}
        if (useMapperAttachments && inputStream != null) {
          if(needComma){
            pw.print(",");
          }
          // I'm told this is not necessary: see CONNECTORS-690
          //pw.print("\"type\" : \"attachment\",");
          pw.print("\"file\" : {");
          String contentType = document.getMimeType();
          if (contentType != null)
            pw.print("\"_content_type\" : "+jsonStringEscape(contentType)+",");
          String fileName = document.getFileName();
          if (fileName != null)
            pw.print("\"_name\" : "+jsonStringEscape(fileName)+",");
          // Since ES 1.0
          pw.print(" \"_content\" : \"");
          Base64 base64 = new Base64();
          base64.encodeStream(inputStream, pw);
          pw.print("\"}");
        }
        
        if (!useMapperAttachments && inputStream != null) {
          if (contentAttributeName != null)
          {
            Reader r = new InputStreamReader(inputStream, Consts.UTF_8);
            StringBuilder sb = new StringBuilder((int)document.getBinaryLength());
            char[] buffer = new char[65536];
            while (true)
            {
              int amt = r.read(buffer,0,buffer.length);
              if (amt == -1)
                break;
              sb.append(buffer,0,amt);
            }
            needComma = writeField(pw, needComma, contentAttributeName, new String[]{sb.toString()});
          }
        }
{code}

The second clause therefore needs to be reworked to properly stream the content rather than going via the StringBuffer in that situation.

So is it correct to assume you're not using the mapper attachment?


> MCF shutting down when Tika is used
> -----------------------------------
>
>                 Key: CONNECTORS-1518
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1518
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Tika extractor
>    Affects Versions: ManifoldCF 2.10
>         Environment: Centos 7
> Prior to crash:
> $free -h
>  total used free shared buff/cache available
> Mem: 15G 1.8G 12G 98M 1.1G 13G
> Swap: 2.0G 0B 2.0G
> After crash:
> $free -h
>  total used free shared buff/cache available
> Mem: 15G 10G 4.0G 98M 1.1G 4.4G
> Swap: 2.0G 0B 2.0G
>  
> {{start-options.env.unix :}}
> {{-Xss500m}}
> {{-Xms1g}}
> {{-Xmx8g}}
> {{-Dorg.apache.manifoldcf.configfile=./properties.xml}}
> {{-Dorg.apache.manifoldcf.jettyshutdowntoken=secret_token}}
> {{-cp}}
> {{.:./lib/mcf-core.jar:./lib/mcf-agents.jar:./lib/mcf-pull-agent.jar:./lib/mcf-ui-core.jar:./lib/mcf-jetty-runner.jar:./lib/jetty-continuation-9.2.3.v20140905.jar:./lib/jetty-http-9.2.3.v20140905.jar:./lib/jetty-io-9.2.3.v20140905.jar:./lib/jetty-jndi-9.2.3.v20140905.jar:./lib/jetty-jsp-jdt-2.3.3.jar:./lib/jetty-plus-9.2.3.v20140905.jar:./lib/jetty-schemas-3.1.M0.jar:./lib/jetty-security-9.2.3.v20140905.jar:./lib/jetty-server-9.2.3.v20140905.jar:./lib/jetty-servlet-9.2.3.v20140905.jar:./lib/jetty-util-9.2.3.v20140905.jar:./lib/jetty-webapp-9.2.3.v20140905.jar:./lib/jetty-xml-9.2.3.v20140905.jar:./lib/hsqldb-2.3.2.jar:./lib/postgresql-42.1.3.jar:./lib/commons-codec-1.10.jar:./lib/commons-collections-3.2.1.jar:./lib/commons-collections4-4.1.jar:./lib/commons-discovery-0.5.jar:./lib/commons-el-1.0.jar:./lib/commons-exec-1.3.jar:./lib/commons-fileupload-1.2.2.jar:./lib/commons-io-2.5.jar:./lib/commons-lang-2.6.jar:./lib/commons-lang3-3.6.jar:./lib/commons-logging-1.2.jar:./lib/ecj-4.3.1.jar:./lib/gson-2.8.0.jar:./lib/guava-21.0.jar:./lib/httpclient-4.5.3.jar:./lib/httpcore-4.4.6.jar:./lib/jasper-6.0.35.jar:./lib/jasper-el-6.0.35.jar:./lib/javax.servlet-api-3.1.0.jar:./lib/jna-4.1.0.jar:./lib/jna-platform-4.1.0.jar:./lib/json-simple-1.1.1.jar:./lib/jsp-api-2.1-glassfish-2.1.v20091210.jar:./lib/juli-6.0.35.jar:./lib/log4j-1.2-api-2.4.1.jar:./lib/log4j-api-2.4.1.jar:./lib/log4j-core-2.4.1.jar:./lib/mail-1.4.5.jar:./lib/serializer-2.7.1.jar:./lib/slf4j-api-1.7.24.jar:./lib/slf4j-simple-1.7.24.jar:./lib/velocity-1.7.jar:./lib/xalan-2.7.1.jar:./lib/xercesImpl-2.10.0.jar:./lib/xml-apis-1.4.01.jar:./lib/zookeeper-3.4.10.jar:}}
>            Reporter: Steph van Schalkwyk
>            Assignee: Karl Wright
>            Priority: Major
>             Fix For: ManifoldCF 2.11
>
>
>   ```Jul 26, 2018 1:21:51 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
>  WARNING: org.xerial's sqlite-jdbc is not loaded.
>  Please provide the jar on your classpath to parse sqlite files.
>  See tika-parsers/pom.xml for the correct version.
>  agents process ran out of memory - shutting down
>  java.lang.OutOfMemoryError: Java heap space
>  \{{ {{ at java.base/java.util.Arrays.copyOf(Arrays.java:3816)}}}}
>  \{{ {{ at java.base/java.util.BitSet.ensureCapacity(BitSet.java:338)}}}}
>  \{{ {{ at java.base/java.util.BitSet.expandTo(BitSet.java:353)}}}}
>  \{{ {{ at java.base/java.util.BitSet.set(BitSet.java:448)}}}}
>  \{{ {{ at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.characters(BoilerpipeHTMLContentHandler.java:267)}}}}
>  \{{ {{ at org.apache.tika.parser.html.BoilerpipeContentHandler.characters(BoilerpipeContentHandler.java:155)}}}}
>  \{{ {{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}}}
>  \{{ {{ at org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)}}}}
>  \{{ {{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}}}
>  \{{ {{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}}}
>  \{{ {{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}}}
>  \{{ {{ at org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)}}}}
>  \{{ {{ at org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)}}}}
>  \{{ {{ at org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)}}}}
>  \{{ {{ at org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)}}}}
>  \{{ {{ at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:279)}}}}
>  \{{ {{ at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:306)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.TextCell.render(TextCell.java:34)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processSheet(ExcelExtractor.java:609)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:392)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:343)}}}}
>  \{{ {{ at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)}}}}
>  \{{ {{ at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:109)}}}}
>  \{{ {{ at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:179)}}}}
>  \{{ {{ at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:136)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:319)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:170)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:184)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:132)}}}}
>  \{{ {{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}}}
>  \{{ {{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}}}
>  \{{ {{ at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)}}}}
>  {{ [Thread-475] INFO org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@37095ded\{HTTP/1.1}{{
> {0.0.0.0:8345}
> }}}}
>  {{ {{[Thread-475] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@5a6d5a8f
> {/mcf-api-service,[file:/tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-14189461872304124764.dir/webapp/,UNAVAILABLE|file:///tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-14189461872304124764.dir/webapp/,UNAVAILABLE]}
> }}{{
> {/opt/manifoldcf/manifoldcf_single/././web/war/mcf-api-service.war}}}}}
>  {{ [Thread-475] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@6979efad{/mcf-authority-service,[file:/tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-11619445383548662284.dir/webapp/,UNAVAILABLE|file:///tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-11619445383548662284.dir/webapp/,UNAVAILABLE]}\{/opt/manifoldcf/manifoldcf_single/././web/war/mcf-authority-service.war}}}
>  2018-07-26 13:22:47,170 qtp2061226112-492 FATAL Unable to register shutdown hook because JVM is shutting down. java.lang.IllegalStateException: Cannot add new shutdown hook as this is not started. Current state: STOPPED
>  \{{ {{ at org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry.addShutdownCallback(DefaultShutdownCallbackRegistry.java:113)}}}}
>  \{{ {{ at org.apache.logging.log4j.core.impl.Log4jContextFactory.addShutdownCallback(Log4jContextFactory.java:271)}}}}
>  \{{ {{ at org.apache.logging.log4j.core.LoggerContext.setUpShutdownHook(LoggerContext.java:256)}}}}
>  \{{ {{ at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:216)}}}}
>  \{{ {{ at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:146)}}}}
>  \{{ {{ at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:41)}}}}
>  \{{ {{ at org.apache.logging.log4j.LogManager.getContext(LogManager.java:270)}}}}
>  \{{ {{ at org.apache.log4j.Logger$PrivateManager.getContext(Logger.java:59)}}}}
>  \{{ {{ at org.apache.log4j.Logger.getLogger(Logger.java:37)}}}}
>  \{{ {{ at org.apache.velocity.runtime.log.Log4JLogChute.init(Log4JLogChute.java:72)}}}}
>  \{{ {{ at org.apache.velocity.runtime.log.LogManager.createLogChute(LogManager.java:157)}}}}
>  \{{ {{ at org.apache.velocity.runtime.log.LogManager.updateLog(LogManager.java:269)}}}}
>  \{{ {{ at org.apache.velocity.runtime.RuntimeInstance.initializeLog(RuntimeInstance.java:871)}}}}
>  \{{ {{ at org.apache.velocity.runtime.RuntimeInstance.init(RuntimeInstance.java:262)}}}}
>  \{{ {{ at org.apache.velocity.runtime.RuntimeInstance.requireInitialization(RuntimeInstance.java:302)}}}}
>  \{{ {{ at org.apache.velocity.runtime.RuntimeInstance.getTemplate(RuntimeInstance.java:1531)}}}}
>  \{{ {{ at org.apache.velocity.app.VelocityEngine.mergeTemplate(VelocityEngine.java:343)}}}}
>  \{{ {{ at org.apache.manifoldcf.ui.i18n.Messages.outputResourceWithVelocity(Messages.java:159)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.transformation.tika.Messages.outputResourceWithVelocity(Messages.java:136)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.outputSpecificationBody(TikaExtractor.java:544)}}}}
>  \{{ {{ at org.apache.jsp.editjob_jsp._jspService(editjob_jsp.java:3002)}}}}
>  \{{ {{ at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)}}}}
>  \{{ {{ at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)}}}}
>  \{{ {{ at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)}}}}
>  \{{ {{ at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)}}}}
>  \{{ {{ at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)}}}}
>  \{{ {{ at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)}}}}
>  \{{ {{ at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)}}}}
>  \{{ {{ at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)}}}}
>  \{{ {{ at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)}}}}
>  \{{ {{ at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)}}}}
>  \{{ {{ at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)}}}}
>  \{{ {{ at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)}}}}
>  \{{ {{ at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)}}}}
>  \{{ {{ at org.eclipse.jetty.server.Server.handle(Server.java:497)}}}}
>  \{{ {{ at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)}}}}
>  \{{ {{ at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)}}}}
>  \{{ {{ at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)}}}}
>  \{{ {{ at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)}}}}
>  \{{ {{ at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)}}}}
>  \{{ {{ at java.base/java.lang.Thread.run(Thread.java:844)}}}}[Worker thread '35'] WARN org.apache.tika.parser.microsoft.AbstractPOIFSExtractor - Ignoring unexpected exception while parsing summary entry SummaryInformation
>  java.lang.RuntimeException: java.nio.channels.ClosedByInterruptException
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.<init>(NPOIFSStream.java:151)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream.getBlockIterator(NPOIFSStream.java:95)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSDocument.getBlockIterator(NPOIFSDocument.java:179)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NDocumentInputStream.<init>(NDocumentInputStream.java:82)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.DocumentInputStream.<init>(DocumentInputStream.java:65)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaryEntryIfExists(SummaryExtractor.java:83)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaries(SummaryExtractor.java:73)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:156)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:132)}}}}
>  \{{ {{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}}}
>  \{{ {{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}}}
>  \{{ {{ at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:74)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:235)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:448)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)}}}}
>  Caused by: java.nio.channels.ClosedByInterruptException
>  \{{ {{ at java.base/java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:199)}}}}
>  \{{ {{ at java.base/sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:388)}}}}
>  \{{ {{ at org.apache.poi.poifs.nio.FileBackedDataSource.size(FileBackedDataSource.java:137)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getChainLoopDetector(NPOIFSFileSystem.java:627)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.<init>(NPOIFSStream.java:149)}}}}
>  \{{ {{ ... 21 more}}}}
>  [Worker thread '35'] WARN org.apache.tika.parser.microsoft.AbstractPOIFSExtractor - Ignoring unexpected exception while parsing summary entry DocumentSummaryInformation
>  java.lang.RuntimeException: java.nio.channels.ClosedChannelException
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.<init>(NPOIFSStream.java:151)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream.getBlockIterator(NPOIFSStream.java:95)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSMiniStore.getBlockAt(NPOIFSMiniStore.java:67)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:169)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.next(NPOIFSStream.java:142)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NDocumentInputStream.readFully(NDocumentInputStream.java:264)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NDocumentInputStream.read(NDocumentInputStream.java:162)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.DocumentInputStream.read(DocumentInputStream.java:127)}}}}
>  \{{ {{ at org.apache.poi.util.BoundedInputStream.read(BoundedInputStream.java:121)}}}}
>  \{{ {{ at org.apache.poi.util.BoundedInputStream.read(BoundedInputStream.java:103)}}}}
>  \{{ {{ at org.apache.poi.util.IOUtils.copy(IOUtils.java:312)}}}}
>  \{{ {{ at org.apache.poi.util.IOUtils.peekFirstNBytes(IOUtils.java:70)}}}}
>  \{{ {{ at org.apache.poi.hpsf.PropertySet.isPropertySetStream(PropertySet.java:393)}}}}
>  \{{ {{ at org.apache.poi.hpsf.PropertySet.<init>(PropertySet.java:191)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaryEntryIfExists(SummaryExtractor.java:83)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.SummaryExtractor.parseSummaries(SummaryExtractor.java:74)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:156)}}}}
>  \{{ {{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:132)}}}}
>  \{{ {{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}}}
>  \{{ {{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}}}
>  \{{ {{ at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:74)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:235)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)}}}}
>  \{{ {{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:448)}}}}
>  \{{ {{ at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)}}}}
>  Caused by: java.nio.channels.ClosedChannelException
>  \{{ {{ at java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:158)}}}}
>  \{{ {{ at java.base/sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:373)}}}}
>  \{{ {{ at org.apache.poi.poifs.nio.FileBackedDataSource.size(FileBackedDataSource.java:137)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSFileSystem.getChainLoopDetector(NPOIFSFileSystem.java:627)}}}}
>  \{{ {{ at org.apache.poi.poifs.filesystem.NPOIFSStream$StreamBlockByteBufferIterator.<init>(NPOIFSStream.java:149)}}}}
>  \{{ {{ ... 30 more}}}} ```}}{{Following up:When these exceptions occur, the heap runs out:13:39:39.856 [Worker thread '49'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:39.970 [Worker thread '43'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:40.415 [Worker thread '34'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:40.469 [Worker thread '1'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:43.739 [Worker thread '32'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:44.697 [Worker thread '43'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:45.756 [Worker thread '33'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:45.775 [Worker thread '36'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:46.751 [Worker thread '35'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:46.753 [Worker thread '40'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:47.536 [Worker thread '45'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:48.734 [Worker thread '44'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:50.922 [Worker thread '30'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:54.930 [Worker thread '28'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:40:33.660 [Worker thread '29'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  agents process ran out of memory - shutting down
>  java.lang.OutOfMemoryError: Java heap space
>  \{{ at java.base/java.lang.StringLatin1.newString(StringLatin1.java:549)}}
>  \{{ at java.base/java.lang.StringBuilder.toString(StringBuilder.java:415)}}
>  \{{ at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.flushBlock(BoilerpipeHTMLContentHandler.java:341)}}
>  \{{ at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.characters(BoilerpipeHTMLContentHandler.java:198)}}
>  \{{ at org.apache.tika.parser.html.BoilerpipeContentHandler.characters(BoilerpipeContentHandler.java:155)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}
>  \{{ at org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)}}
>  \{{ at org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)}}
>  \{{ at org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)}}
>  \{{ at org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)}}
>  \{{ at org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)}}
>  \{{ at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:279)}}
>  \{{ at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:306)}}
>  \{{ at org.apache.tika.parser.microsoft.TextCell.render(TextCell.java:34)}}
>  \{{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processSheet(ExcelExtractor.java:609)}}
>  \{{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:392)}}
>  \{{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:343)}}
>  \{{ at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)}}
>  \{{ at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:109)}}
>  \{{ at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:179)}}
>  \{{ at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:136)}}
>  \{{ at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:319)}}
>  \{{ at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:170)}}
>  \{{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:184)}}
>  \{{ at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:132)}}
>  \{{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}
>  \{{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}
>  \{{ at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)}}
>  \{{ at org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:74)}}
>  agents process ran out of memory - shutting down
>  java.lang.OutOfMemoryError: Java heap space
>  \{{ at java.base/java.util.Arrays.copyOf(Arrays.java:3744)}}
>  \{{ at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:146)}}
>  \{{ at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:531)}}
>  \{{ at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:550)}}
>  \{{ at java.base/java.lang.StringBuilder.append(StringBuilder.java:171)}}
>  \{{ at java.base/java.util.regex.Matcher.appendReplacement(Matcher.java:1002)}}
>  \{{ at java.base/java.util.regex.Matcher.replaceAll(Matcher.java:1181)}}
>  \{{ at de.l3s.boilerpipe.util.UnicodeTokenizer.tokenize(UnicodeTokenizer.java:40)}}
>  \{{ at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.flushBlock(BoilerpipeHTMLContentHandler.java:296)}}
>  \{{ at de.l3s.boilerpipe.sax.CommonTagActions$3.end(CommonTagActions.java:143)}}
>  \{{ at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.endElement(BoilerpipeHTMLContentHandler.java:183)}}
>  \{{ at org.apache.tika.parser.html.BoilerpipeContentHandler.endElement(BoilerpipeContentHandler.java:175)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)}}
>  \{{ at org.apache.tika.sax.SecureContentHandler.endElement(SecureContentHandler.java:256)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)}}
>  \{{ at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)}}
>  \{{ at org.apache.tika.sax.SafeContentHandler.endElement(SafeContentHandler.java:273)}}
>  \{{ at org.apache.tika.sax.XHTMLContentHandler.endDocument(XHTMLContentHandler.java:224)}}
>  \{{ at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:109)}}
>  \{{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}
>  \{{ at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)}}
>  \{{ at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)}}
>  \{{ at org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:74)}}
>  \{{ at org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:235)}}
>  \{{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)}}
>  \{{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)}}
>  \{{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)}}
>  \{{ at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)}}
>  \{{ at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)}}
>  \{{ at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)}}
>  \{{ at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:448)}}
>  13:40:33.995 [Worker thread '42'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  [Thread-475] INFO org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@5d235104\{HTTP/1.1}{0.0.0.0:8345}
>  {{[Thread-475] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@6105f8a3\{/mcf-api-service,[file:/tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-9896962439762567079.dir/webapp/,UNAVAILABLE|file:///tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-9896962439762567079.dir/webapp/,UNAVAILABLE]}{/opt/manifoldcf/manifoldcf_single/././web/war/mcf-api-service.war}
>  
>  }}
>  {{[Thread-475] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@12365c88\{/mcf-authority-service,[file:/tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-3954308360064638561.dir/webapp/,UNAVAILABLE|file:///tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-3954308360064638561.dir/webapp/,UNAVAILABLE]}
>  \{/opt/manifoldcf/manifoldcf_single/././web/war/mcf-authority-service.war}
>  
>  }}
>  
>   
>  
>  Follow-up: When these issues occur, the jvm runs out of space:
>  
>  13:39:39.856 [Worker thread '49'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:39.970 [Worker thread '43'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:40.415 [Worker thread '34'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:40.469 [Worker thread '1'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:43.739 [Worker thread '32'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:44.697 [Worker thread '43'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:45.756 [Worker thread '33'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:45.775 [Worker thread '36'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:46.751 [Worker thread '35'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:46.753 [Worker thread '40'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:47.536 [Worker thread '45'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:48.734 [Worker thread '44'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:50.922 [Worker thread '30'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:39:54.930 [Worker thread '28'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  13:40:33.660 [Worker thread '29'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  agents process ran out of memory - shutting down
>  java.lang.OutOfMemoryError: Java heap space
>  at java.base/java.lang.StringLatin1.newString(StringLatin1.java:549)
>  at java.base/java.lang.StringBuilder.toString(StringBuilder.java:415)
>  at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.flushBlock(BoilerpipeHTMLContentHandler.java:341)
>  at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.characters(BoilerpipeHTMLContentHandler.java:198)
>  at org.apache.tika.parser.html.BoilerpipeContentHandler.characters(BoilerpipeContentHandler.java:155)
>  at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
>  at org.apache.tika.sax.SecureContentHandler.characters(SecureContentHandler.java:270)
>  at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
>  at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
>  at org.apache.tika.sax.ContentHandlerDecorator.characters(ContentHandlerDecorator.java:146)
>  at org.apache.tika.sax.SafeContentHandler.access$001(SafeContentHandler.java:46)
>  at org.apache.tika.sax.SafeContentHandler$1.write(SafeContentHandler.java:82)
>  at org.apache.tika.sax.SafeContentHandler.filter(SafeContentHandler.java:140)
>  at org.apache.tika.sax.SafeContentHandler.characters(SafeContentHandler.java:287)
>  at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:279)
>  at org.apache.tika.sax.XHTMLContentHandler.characters(XHTMLContentHandler.java:306)
>  at org.apache.tika.parser.microsoft.TextCell.render(TextCell.java:34)
>  at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processSheet(ExcelExtractor.java:609)
>  at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.internalProcessRecord(ExcelExtractor.java:392)
>  at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processRecord(ExcelExtractor.java:343)
>  at org.apache.poi.hssf.eventusermodel.FormatTrackingHSSFListener.processRecord(FormatTrackingHSSFListener.java:92)
>  at org.apache.poi.hssf.eventusermodel.HSSFRequest.processRecord(HSSFRequest.java:109)
>  at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.genericProcessEvents(HSSFEventFactory.java:179)
>  at org.apache.poi.hssf.eventusermodel.HSSFEventFactory.processEvents(HSSFEventFactory.java:136)
>  at org.apache.tika.parser.microsoft.ExcelExtractor$TikaHSSFListener.processFile(ExcelExtractor.java:319)
>  at org.apache.tika.parser.microsoft.ExcelExtractor.parse(ExcelExtractor.java:170)
>  at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:184)
>  at org.apache.tika.parser.microsoft.OfficeParser.parse(OfficeParser.java:132)
>  at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>  at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>  at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
>  at org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:74)
>  agents process ran out of memory - shutting down
>  java.lang.OutOfMemoryError: Java heap space
>  at java.base/java.util.Arrays.copyOf(Arrays.java:3744)
>  at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:146)
>  at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:531)
>  at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:550)
>  at java.base/java.lang.StringBuilder.append(StringBuilder.java:171)
>  at java.base/java.util.regex.Matcher.appendReplacement(Matcher.java:1002)
>  at java.base/java.util.regex.Matcher.replaceAll(Matcher.java:1181)
>  at de.l3s.boilerpipe.util.UnicodeTokenizer.tokenize(UnicodeTokenizer.java:40)
>  at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.flushBlock(BoilerpipeHTMLContentHandler.java:296)
>  at de.l3s.boilerpipe.sax.CommonTagActions$3.end(CommonTagActions.java:143)
>  at de.l3s.boilerpipe.sax.BoilerpipeHTMLContentHandler.endElement(BoilerpipeHTMLContentHandler.java:183)
>  at org.apache.tika.parser.html.BoilerpipeContentHandler.endElement(BoilerpipeContentHandler.java:175)
>  at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)
>  at org.apache.tika.sax.SecureContentHandler.endElement(SecureContentHandler.java:256)
>  at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)
>  at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)
>  at org.apache.tika.sax.ContentHandlerDecorator.endElement(ContentHandlerDecorator.java:136)
>  at org.apache.tika.sax.SafeContentHandler.endElement(SafeContentHandler.java:273)
>  at org.apache.tika.sax.XHTMLContentHandler.endDocument(XHTMLContentHandler.java:224)
>  at org.apache.tika.parser.txt.TXTParser.parse(TXTParser.java:109)
>  at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>  at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
>  at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
>  at org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(TikaParser.java:74)
>  at org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:235)
>  at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
>  at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
>  at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
>  at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
>  at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
>  at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
>  at org.apache.manifoldcf.crawler.connectors.filesystem.FileConnector.processDocuments(FileConnector.java:448)
>  13:40:33.995 [Worker thread '42'] WARN org.apache.manifoldcf.jobs - Service interruption reported for job 1532551209410 connection 'file': IO exception: null
>  [Thread-475] INFO org.eclipse.jetty.server.ServerConnector - Stopped ServerConnector@5d235104\{HTTP/1.1}{0.0.0.0:8345}
>  [Thread-475] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@6105f8a3{/mcf-api-service,[file:/tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-9896962439762567079.dir/webapp/,UNAVAILABLE|file:///tmp/jetty-0.0.0.0-8345-mcf-api-service.war-_mcf-api-service-any-9896962439762567079.dir/webapp/,UNAVAILABLE]}\{/opt/manifoldcf/manifoldcf_single/././web/war/mcf-api-service.war}
> [Thread-475] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped o.e.j.w.WebAppContext@12365c88{/mcf-authority-service,[file:/tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-3954308360064638561.dir/webapp/,UNAVAILABLE|file:///tmp/jetty-0.0.0.0-8345-mcf-authority-service.war-_mcf-authority-service-any-3954308360064638561.dir/webapp/,UNAVAILABLE]}
> {/opt/manifoldcf/manifoldcf_single/././web/war/mcf-authority-service.war}
>  This occurs when ES Connector has this issue:
> |07-26-2018 19:34:25.356|Indexation (ES)|file:/var/manifoldcf/corpus/000640.html|CLIENTPROTOCOLEXCEPTION|46190|9|



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)