You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Bisonti Mario <Ma...@vimar.com> on 2018/11/08 08:32:51 UTC
Error Job stop after repeatidly interruption
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option "-spawnChild"
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
Zookeeper runs as a separate process. You can kill it any way you kill
processes.
Karl
On Tue, Nov 27, 2018 at 12:11 PM Bisonti Mario <Ma...@vimar.com>
wrote:
> I don’t understand how to stop cleanly zookeper.
>
> I start it by crontab at reboot:
> @reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/runzookeeper.sh
>
>
>
> But now I tried to abort the job buti t hangs in state “Aborting”
>
>
>
> When I didn’t use zookeper I used the lock-clean.sh script but in the
> multiprocess-zk-example-proprietary it doesn’t exist.
>
> So I am in an hang situation…
>
> I am not able how to close zookeeper and I am not able to abort job ☹
>
> ☹
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* martedì 27 novembre 2018 16:58
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> You can shut down ManifoldCF at any time and it will pick up where it left
> off.
>
> But you should never shut down the database or recycle zookeeper while
> ManifoldCF is running, or you can expect it to lock up. If you get into
> that state, you have to shut down both Zookeeper *and* ManifoldCF, then
> start Zookeeper, then start ManifoldCF. It will then pick up where it left
> off.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Nov 27, 2018 at 4:19 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
> No, I didn’t shut MCF during running job
>
>
>
> My zookeeper.cfg is:
> tickTime=5000
>
> dataDir=zookeeper
>
> clientPort=8349
>
> maxClientCnxns=1000
>
> autopurge.snapRetainCount=3
>
> autopurge.purgeInterval=6
>
>
>
> I tried to increase maxClientCnxns to 100000
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* martedì 27 novembre 2018 09:20
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> This error:
>
>
>
> >>>>>>
>
> FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed:
> Can't release lock we don't hold
>
> java.lang.IllegalStateException: Can't release lock we don't hold
>
> <<<<<<
>
>
>
> ... means that Zookeeper failed in some way. Did you shut it down while
> MCF was still running?
>
>
>
> You should probably shut everything down and start it up again, including
> Zookeeper. There are also some configuration properties Zookeeper has (see
> zookeeper.cfg) that you may need to modify to handle a very large crawl, so
> I'd read the "how to build and deploy" page to make sure you've got the
> parameters consistent with that.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Nov 27, 2018 at 3:13 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I enlarge the timeout to process the documents on Tika server and the job
> didn’t stop about repeatidly interruption but now the job is running but
> hangs.
>
>
>
> In manifoldcf.log I see:
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T18:13:39,050 (Worker thread '63') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at
> jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed:
> Can't release lock we don't hold
>
> java.lang.IllegalStateException: Can't release lock we don't hold
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.releaseLock(ZooKeeperConnection.java:815)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearLock(ZooKeeperLockObject.java:218)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearGlobalWriteLockNoWait(ZooKeeperLockObject.java:100)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockObject.clearGlobalWriteLock(LockObject.java:160)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockObject.leaveWriteLock(LockObject.java:141)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockGate.leaveWriteLock(LockGate.java:205)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWrite(BaseLockManager.java:1224)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWriteLock(BaseLockManager.java:771)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.ThrottleBin.poll(ThrottleBin.java:391)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroup.poll(Throttler.java:952)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroups.poll(Throttler.java:290)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler.poll(Throttler.java:165)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.ThrottleGroups.poll(ThrottleGroups.java:114)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.interfaces.ThrottleGroupsFactory$ThrottlerPoll.doPoll(ThrottleGroupsFactory.java:65)
> ~[?:?]
>
> at
> org.apache.manifoldcf.core.system.ManifoldCF.pollAll(ManifoldCF.java:1491)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:93)
> [mcf-agents.jar:?]
>
>
>
>
>
> It seems that the agent crashed?
>
>
>
> What could I do?
>
> I start the agent at the startup of the machine with the command,
> scheduled on crontab, by:
>
> @reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh
>
>
>
> Thanks a lot Karl!
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 21:51
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> As you can see, ManifoldCF retries some error conditions until it either
> succeeds or until it must finally give up. At that point it can choose to
> abort the job or skip the document, depending on the error condition.
>
>
>
> The reason I said to look for worker thread 87 in the first log was
> because I wanted to see if retried after that, or not. The change I
> committed should make it retry for a period of time (10 minutes) before
> giving up. The snippet of log you initially included showed a number of
> threads having difficulty connecting to the Tika service; that should be
> retried. And your log messages show a ServiceInterruption was indeed
> processed, which will cause a retry to occur until the time interval has
> elapsed:
>
>
>
> >>>>>>
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> <<<<<<
>
>
>
> When a document causes Tika to crash, always, eventually ManifoldCF will
> give up and shut the job down, even though it's retried a number of times.
> I warned you about this earlier; that may be what is happening here. It
> would be good if you could verify that.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> After the error, the job stop.
>
>
>
> After, I start the job and I obtain many times:
>
>
>
> WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
>
>
> And teh job currently is running
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 17:41
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> For any one of these warnings, e.g.:
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998
>
>
>
> ... look in the log for the next log message for "Worker thread '87'".
> See what it did next and when.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I built and I am using your trunk.
>
> Thanks a lot.
>
>
>
> But I have always the error on the job.
>
>
>
> In the end of the manifoldcf.log I read:
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
>
>
>
>
> So, I don’t understand if the worker tried to reconnect after 10 seconds
> or not
>
>
>
> How could I check it?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 13:00
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> The easiest way to do it is just to check out current trunk:
>
>
>
> svn co https://svn.apache.org/repos/asf/manifoldcf/trunk
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=viJyEgDhFnQssTakG%2BeIn1LnPZhjlBwdmYUJTzyq5Po%3D&reserved=0>
>
>
>
> No need to apply a patch then. Just build:
>
>
>
> ant make-core-deps
>
> ant make-deps
>
> ant build
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Thanks a lot Karl.
>
>
>
> To overwrite the connector with your patch, have I to download the trunk
> and recompile, isn’t it?
>
>
>
> Excuse me for my questions but I am not expert on programming, compiling,
> etc.
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:48
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> (1) I increased the retries to go at least 10 minutes.
>
> (2) I handled the 503 response explicitly, with the same logic.
>
> See: https://issues.apache.org/jira/browse/CONNECTORS-1556
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xEnK14TELE6UAyTHM39b1vriO6RU%2BPX3ule%2BFcWBXUU%3D&reserved=0>
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=TcHMlNFv%2B4xbwcay8KPNMfXyMJZSUR%2BQdABXWZjji9o%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=6Z3xLrfu077R24HERFN6XxuQCFGzWrJbEZdruliwoC4%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=VGujdJkXebuTfELhZLiOz31%2B9tAxS3c9QCBHQR5Rx2k%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
I don’t understand how to stop cleanly zookeper.
I start it by crontab at reboot:
@reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/runzookeeper.sh
But now I tried to abort the job buti t hangs in state “Aborting”
When I didn’t use zookeper I used the lock-clean.sh script but in the multiprocess-zk-example-proprietary it doesn’t exist.
So I am in an hang situation…
I am not able how to close zookeeper and I am not able to abort job ☹
☹
Da: Karl Wright <da...@gmail.com>
Inviato: martedì 27 novembre 2018 16:58
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
You can shut down ManifoldCF at any time and it will pick up where it left off.
But you should never shut down the database or recycle zookeeper while ManifoldCF is running, or you can expect it to lock up. If you get into that state, you have to shut down both Zookeeper *and* ManifoldCF, then start Zookeeper, then start ManifoldCF. It will then pick up where it left off.
Karl
On Tue, Nov 27, 2018 at 4:19 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
No, I didn’t shut MCF during running job
My zookeeper.cfg is:
tickTime=5000
dataDir=zookeeper
clientPort=8349
maxClientCnxns=1000
autopurge.snapRetainCount=3
autopurge.purgeInterval=6
I tried to increase maxClientCnxns to 100000
Da: Karl Wright <da...@gmail.com>>
Inviato: martedì 27 novembre 2018 09:20
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
This error:
>>>>>>
FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed: Can't release lock we don't hold
java.lang.IllegalStateException: Can't release lock we don't hold
<<<<<<
... means that Zookeeper failed in some way. Did you shut it down while MCF was still running?
You should probably shut everything down and start it up again, including Zookeeper. There are also some configuration properties Zookeeper has (see zookeeper.cfg) that you may need to modify to handle a very large crawl, so I'd read the "how to build and deploy" page to make sure you've got the parameters consistent with that.
Karl
On Tue, Nov 27, 2018 at 3:13 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I enlarge the timeout to process the documents on Tika server and the job didn’t stop about repeatidly interruption but now the job is running but hangs.
In manifoldcf.log I see:
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-26T18:13:39,050 (Worker thread '63') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed: Can't release lock we don't hold
java.lang.IllegalStateException: Can't release lock we don't hold
at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.releaseLock(ZooKeeperConnection.java:815) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearLock(ZooKeeperLockObject.java:218) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearGlobalWriteLockNoWait(ZooKeeperLockObject.java:100) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockObject.clearGlobalWriteLock(LockObject.java:160) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockObject.leaveWriteLock(LockObject.java:141) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockGate.leaveWriteLock(LockGate.java:205) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWrite(BaseLockManager.java:1224) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWriteLock(BaseLockManager.java:771) ~[mcf-core.jar:?]
at org.apache.manifoldcf.connectorcommon.throttler.ThrottleBin.poll(ThrottleBin.java:391) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroup.poll(Throttler.java:952) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroups.poll(Throttler.java:290) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler.poll(Throttler.java:165) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.ThrottleGroups.poll(ThrottleGroups.java:114) ~[?:?]
at org.apache.manifoldcf.connectorcommon.interfaces.ThrottleGroupsFactory$ThrottlerPoll.doPoll(ThrottleGroupsFactory.java:65) ~[?:?]
at org.apache.manifoldcf.core.system.ManifoldCF.pollAll(ManifoldCF.java:1491) ~[mcf-core.jar:?]
at org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:93) [mcf-agents.jar:?]
It seems that the agent crashed?
What could I do?
I start the agent at the startup of the machine with the command, scheduled on crontab, by:
@reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh
Thanks a lot Karl!
Da: Karl Wright <da...@gmail.com>>
Inviato: lunedì 26 novembre 2018 21:51
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
As you can see, ManifoldCF retries some error conditions until it either succeeds or until it must finally give up. At that point it can choose to abort the job or skip the document, depending on the error condition.
The reason I said to look for worker thread 87 in the first log was because I wanted to see if retried after that, or not. The change I committed should make it retry for a period of time (10 minutes) before giving up. The snippet of log you initially included showed a number of threads having difficulty connecting to the Tika service; that should be retried. And your log messages show a ServiceInterruption was indeed processed, which will cause a retry to occur until the time interval has elapsed:
>>>>>>
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0E1iv%2FUpA5ARjT8yyrPJrQgFWZcEbePTD1yHDCOG4bU%3D&reserved=0>] failed: Connection refused (Connection refused)
<<<<<<
When a document causes Tika to crash, always, eventually ManifoldCF will give up and shut the job down, even though it's retried a number of times. I warned you about this earlier; that may be what is happening here. It would be good if you could verify that.
Karl
On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>> wrote:
After the error, the job stop.
After, I start the job and I obtain many times:
WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
And teh job currently is running
Da: Karl Wright <da...@gmail.com>>
Inviato: lunedì 26 novembre 2018 17:41
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
For any one of these warnings, e.g.:
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998
... look in the log for the next log message for "Worker thread '87'". See what it did next and when.
Karl
On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I built and I am using your trunk.
Thanks a lot.
But I have always the error on the job.
In the end of the manifoldcf.log I read:
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=B7iOdxvrSA5KfIIIbyHpZgGQizMhwBFrLJarD5pD4Gw%3D&reserved=0>] failed: Connection refused (Connection refused)
So, I don’t understand if the worker tried to reconnect after 10 seconds or not
How could I check it?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 13:00
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
The easiest way to do it is just to check out current trunk:
svn co https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=viJyEgDhFnQssTakG%2BeIn1LnPZhjlBwdmYUJTzyq5Po%3D&reserved=0>
No need to apply a patch then. Just build:
ant make-core-deps
ant make-deps
ant build
Karl
On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Thanks a lot Karl.
To overwrite the connector with your patch, have I to download the trunk and recompile, isn’t it?
Excuse me for my questions but I am not expert on programming, compiling, etc.
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:48
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xEnK14TELE6UAyTHM39b1vriO6RU%2BPX3ule%2BFcWBXUU%3D&reserved=0>
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=TcHMlNFv%2B4xbwcay8KPNMfXyMJZSUR%2BQdABXWZjji9o%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=6Z3xLrfu077R24HERFN6XxuQCFGzWrJbEZdruliwoC4%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=0gy%2FsM9DftXg%2BCKb66HCqY9lawxCr7K%2BPyaAA1vHSBk%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ca3ad481a5390410a16a508d65481239d%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=VGujdJkXebuTfELhZLiOz31%2B9tAxS3c9QCBHQR5Rx2k%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
Hi Mario,
You can shut down ManifoldCF at any time and it will pick up where it left
off.
But you should never shut down the database or recycle zookeeper while
ManifoldCF is running, or you can expect it to lock up. If you get into
that state, you have to shut down both Zookeeper *and* ManifoldCF, then
start Zookeeper, then start ManifoldCF. It will then pick up where it left
off.
Karl
On Tue, Nov 27, 2018 at 4:19 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo.
>
> No, I didn’t shut MCF during running job
>
>
>
> My zookeeper.cfg is:
> tickTime=5000
>
> dataDir=zookeeper
>
> clientPort=8349
>
> maxClientCnxns=1000
>
> autopurge.snapRetainCount=3
>
> autopurge.purgeInterval=6
>
>
>
> I tried to increase maxClientCnxns to 100000
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* martedì 27 novembre 2018 09:20
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> This error:
>
>
>
> >>>>>>
>
> FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed:
> Can't release lock we don't hold
>
> java.lang.IllegalStateException: Can't release lock we don't hold
>
> <<<<<<
>
>
>
> ... means that Zookeeper failed in some way. Did you shut it down while
> MCF was still running?
>
>
>
> You should probably shut everything down and start it up again, including
> Zookeeper. There are also some configuration properties Zookeeper has (see
> zookeeper.cfg) that you may need to modify to handle a very large crawl, so
> I'd read the "how to build and deploy" page to make sure you've got the
> parameters consistent with that.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Nov 27, 2018 at 3:13 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I enlarge the timeout to process the documents on Tika server and the job
> didn’t stop about repeatidly interruption but now the job is running but
> hangs.
>
>
>
> In manifoldcf.log I see:
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T18:13:39,050 (Worker thread '63') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at
> jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed:
> Can't release lock we don't hold
>
> java.lang.IllegalStateException: Can't release lock we don't hold
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.releaseLock(ZooKeeperConnection.java:815)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearLock(ZooKeeperLockObject.java:218)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearGlobalWriteLockNoWait(ZooKeeperLockObject.java:100)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockObject.clearGlobalWriteLock(LockObject.java:160)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockObject.leaveWriteLock(LockObject.java:141)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockGate.leaveWriteLock(LockGate.java:205)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWrite(BaseLockManager.java:1224)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWriteLock(BaseLockManager.java:771)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.ThrottleBin.poll(ThrottleBin.java:391)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroup.poll(Throttler.java:952)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroups.poll(Throttler.java:290)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler.poll(Throttler.java:165)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.ThrottleGroups.poll(ThrottleGroups.java:114)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.interfaces.ThrottleGroupsFactory$ThrottlerPoll.doPoll(ThrottleGroupsFactory.java:65)
> ~[?:?]
>
> at
> org.apache.manifoldcf.core.system.ManifoldCF.pollAll(ManifoldCF.java:1491)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:93)
> [mcf-agents.jar:?]
>
>
>
>
>
> It seems that the agent crashed?
>
>
>
> What could I do?
>
> I start the agent at the startup of the machine with the command,
> scheduled on crontab, by:
>
> @reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh
>
>
>
> Thanks a lot Karl!
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 21:51
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> As you can see, ManifoldCF retries some error conditions until it either
> succeeds or until it must finally give up. At that point it can choose to
> abort the job or skip the document, depending on the error condition.
>
>
>
> The reason I said to look for worker thread 87 in the first log was
> because I wanted to see if retried after that, or not. The change I
> committed should make it retry for a period of time (10 minutes) before
> giving up. The snippet of log you initially included showed a number of
> threads having difficulty connecting to the Tika service; that should be
> retried. And your log messages show a ServiceInterruption was indeed
> processed, which will cause a retry to occur until the time interval has
> elapsed:
>
>
>
> >>>>>>
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> <<<<<<
>
>
>
> When a document causes Tika to crash, always, eventually ManifoldCF will
> give up and shut the job down, even though it's retried a number of times.
> I warned you about this earlier; that may be what is happening here. It
> would be good if you could verify that.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> After the error, the job stop.
>
>
>
> After, I start the job and I obtain many times:
>
>
>
> WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
>
>
> And teh job currently is running
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 17:41
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> For any one of these warnings, e.g.:
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998
>
>
>
> ... look in the log for the next log message for "Worker thread '87'".
> See what it did next and when.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I built and I am using your trunk.
>
> Thanks a lot.
>
>
>
> But I have always the error on the job.
>
>
>
> In the end of the manifoldcf.log I read:
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
>
>
>
>
> So, I don’t understand if the worker tried to reconnect after 10 seconds
> or not
>
>
>
> How could I check it?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 13:00
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> The easiest way to do it is just to check out current trunk:
>
>
>
> svn co https://svn.apache.org/repos/asf/manifoldcf/trunk
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=99bw%2BU3bYRO6XTfHwkNrfOCWv%2B4P67mrSVfdAU9riS0%3D&reserved=0>
>
>
>
> No need to apply a patch then. Just build:
>
>
>
> ant make-core-deps
>
> ant make-deps
>
> ant build
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Thanks a lot Karl.
>
>
>
> To overwrite the connector with your patch, have I to download the trunk
> and recompile, isn’t it?
>
>
>
> Excuse me for my questions but I am not expert on programming, compiling,
> etc.
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:48
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> (1) I increased the retries to go at least 10 minutes.
>
> (2) I handled the 503 response explicitly, with the same logic.
>
> See: https://issues.apache.org/jira/browse/CONNECTORS-1556
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HUg%2FnEy3kNc0n3JVZWQBw9wLRykK8nZl2vjt3iLrFT0%3D&reserved=0>
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=b4I3WD7QrC%2ByQnIsVET1%2BbXSEYkp62HaF9oWS5%2FgDLk%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=RXHW4f4fPSBnwij3%2BQ4naLpktxtstTo4pupKd%2FnKEIw%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=3Yw9eYl7l3yCTFpO2EXxurWqYd%2BwFvSi5q4MQtGb%2B3E%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
Hallo.
No, I didn’t shut MCF during running job
My zookeeper.cfg is:
tickTime=5000
dataDir=zookeeper
clientPort=8349
maxClientCnxns=1000
autopurge.snapRetainCount=3
autopurge.purgeInterval=6
I tried to increase maxClientCnxns to 100000
Da: Karl Wright <da...@gmail.com>
Inviato: martedì 27 novembre 2018 09:20
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
This error:
>>>>>>
FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed: Can't release lock we don't hold
java.lang.IllegalStateException: Can't release lock we don't hold
<<<<<<
... means that Zookeeper failed in some way. Did you shut it down while MCF was still running?
You should probably shut everything down and start it up again, including Zookeeper. There are also some configuration properties Zookeeper has (see zookeeper.cfg) that you may need to modify to handle a very large crawl, so I'd read the "how to build and deploy" page to make sure you've got the parameters consistent with that.
Karl
On Tue, Nov 27, 2018 at 3:13 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I enlarge the timeout to process the documents on Tika server and the job didn’t stop about repeatidly interruption but now the job is running but hangs.
In manifoldcf.log I see:
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-26T18:13:39,050 (Worker thread '63') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed: Can't release lock we don't hold
java.lang.IllegalStateException: Can't release lock we don't hold
at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.releaseLock(ZooKeeperConnection.java:815) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearLock(ZooKeeperLockObject.java:218) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearGlobalWriteLockNoWait(ZooKeeperLockObject.java:100) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockObject.clearGlobalWriteLock(LockObject.java:160) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockObject.leaveWriteLock(LockObject.java:141) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockGate.leaveWriteLock(LockGate.java:205) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWrite(BaseLockManager.java:1224) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWriteLock(BaseLockManager.java:771) ~[mcf-core.jar:?]
at org.apache.manifoldcf.connectorcommon.throttler.ThrottleBin.poll(ThrottleBin.java:391) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroup.poll(Throttler.java:952) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroups.poll(Throttler.java:290) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler.poll(Throttler.java:165) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.ThrottleGroups.poll(ThrottleGroups.java:114) ~[?:?]
at org.apache.manifoldcf.connectorcommon.interfaces.ThrottleGroupsFactory$ThrottlerPoll.doPoll(ThrottleGroupsFactory.java:65) ~[?:?]
at org.apache.manifoldcf.core.system.ManifoldCF.pollAll(ManifoldCF.java:1491) ~[mcf-core.jar:?]
at org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:93) [mcf-agents.jar:?]
It seems that the agent crashed?
What could I do?
I start the agent at the startup of the machine with the command, scheduled on crontab, by:
@reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh
Thanks a lot Karl!
Da: Karl Wright <da...@gmail.com>>
Inviato: lunedì 26 novembre 2018 21:51
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
As you can see, ManifoldCF retries some error conditions until it either succeeds or until it must finally give up. At that point it can choose to abort the job or skip the document, depending on the error condition.
The reason I said to look for worker thread 87 in the first log was because I wanted to see if retried after that, or not. The change I committed should make it retry for a period of time (10 minutes) before giving up. The snippet of log you initially included showed a number of threads having difficulty connecting to the Tika service; that should be retried. And your log messages show a ServiceInterruption was indeed processed, which will cause a retry to occur until the time interval has elapsed:
>>>>>>
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=vgYmiDvv2r6ki%2B9JwdfcdVK6G1p3NS2WTYippiPmqoU%3D&reserved=0>] failed: Connection refused (Connection refused)
<<<<<<
When a document causes Tika to crash, always, eventually ManifoldCF will give up and shut the job down, even though it's retried a number of times. I warned you about this earlier; that may be what is happening here. It would be good if you could verify that.
Karl
On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>> wrote:
After the error, the job stop.
After, I start the job and I obtain many times:
WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
And teh job currently is running
Da: Karl Wright <da...@gmail.com>>
Inviato: lunedì 26 novembre 2018 17:41
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
For any one of these warnings, e.g.:
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998
... look in the log for the next log message for "Worker thread '87'". See what it did next and when.
Karl
On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I built and I am using your trunk.
Thanks a lot.
But I have always the error on the job.
In the end of the manifoldcf.log I read:
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=M7LlMsJuFNK9xtTaxNsa7zNQMhTyWZp4ffiTaZDUS0Q%3D&reserved=0>] failed: Connection refused (Connection refused)
So, I don’t understand if the worker tried to reconnect after 10 seconds or not
How could I check it?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 13:00
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
The easiest way to do it is just to check out current trunk:
svn co https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=99bw%2BU3bYRO6XTfHwkNrfOCWv%2B4P67mrSVfdAU9riS0%3D&reserved=0>
No need to apply a patch then. Just build:
ant make-core-deps
ant make-deps
ant build
Karl
On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Thanks a lot Karl.
To overwrite the connector with your patch, have I to download the trunk and recompile, isn’t it?
Excuse me for my questions but I am not expert on programming, compiling, etc.
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:48
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HUg%2FnEy3kNc0n3JVZWQBw9wLRykK8nZl2vjt3iLrFT0%3D&reserved=0>
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=b4I3WD7QrC%2ByQnIsVET1%2BbXSEYkp62HaF9oWS5%2FgDLk%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=RXHW4f4fPSBnwij3%2BQ4naLpktxtstTo4pupKd%2FnKEIw%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=cnIPd5eYUIJl64%2FOuf0bJpfhwG4b6ndjqSudw5dCRXY%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C370e129d503d46023a8408d6544133fb%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=3Yw9eYl7l3yCTFpO2EXxurWqYd%2BwFvSi5q4MQtGb%2B3E%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
Hi Mario,
This error:
>>>>>>
FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed:
Can't release lock we don't hold
java.lang.IllegalStateException: Can't release lock we don't hold
<<<<<<
... means that Zookeeper failed in some way. Did you shut it down while
MCF was still running?
You should probably shut everything down and start it up again, including
Zookeeper. There are also some configuration properties Zookeeper has (see
zookeeper.cfg) that you may need to modify to handle a very large crawl, so
I'd read the "how to build and deploy" page to make sure you've got the
parameters consistent with that.
Karl
On Tue, Nov 27, 2018 at 3:13 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo Karl.
>
> I enlarge the timeout to process the documents on Tika server and the job
> didn’t stop about repeatidly interruption but now the job is running but
> hangs.
>
>
>
> In manifoldcf.log I see:
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T18:13:39,050 (Worker thread '63') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at
> jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed:
> Can't release lock we don't hold
>
> java.lang.IllegalStateException: Can't release lock we don't hold
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.releaseLock(ZooKeeperConnection.java:815)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearLock(ZooKeeperLockObject.java:218)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearGlobalWriteLockNoWait(ZooKeeperLockObject.java:100)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockObject.clearGlobalWriteLock(LockObject.java:160)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockObject.leaveWriteLock(LockObject.java:141)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.LockGate.leaveWriteLock(LockGate.java:205)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWrite(BaseLockManager.java:1224)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWriteLock(BaseLockManager.java:771)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.ThrottleBin.poll(ThrottleBin.java:391)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroup.poll(Throttler.java:952)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroups.poll(Throttler.java:290)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.Throttler.poll(Throttler.java:165)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.throttler.ThrottleGroups.poll(ThrottleGroups.java:114)
> ~[?:?]
>
> at
> org.apache.manifoldcf.connectorcommon.interfaces.ThrottleGroupsFactory$ThrottlerPoll.doPoll(ThrottleGroupsFactory.java:65)
> ~[?:?]
>
> at
> org.apache.manifoldcf.core.system.ManifoldCF.pollAll(ManifoldCF.java:1491)
> ~[mcf-core.jar:?]
>
> at
> org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:93)
> [mcf-agents.jar:?]
>
>
>
>
>
> It seems that the agent crashed?
>
>
>
> What could I do?
>
> I start the agent at the startup of the machine with the command,
> scheduled on crontab, by:
>
> @reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh
>
>
>
> Thanks a lot Karl!
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 21:51
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> As you can see, ManifoldCF retries some error conditions until it either
> succeeds or until it must finally give up. At that point it can choose to
> abort the job or skip the document, depending on the error condition.
>
>
>
> The reason I said to look for worker thread 87 in the first log was
> because I wanted to see if retried after that, or not. The change I
> committed should make it retry for a period of time (10 minutes) before
> giving up. The snippet of log you initially included showed a number of
> threads having difficulty connecting to the Tika service; that should be
> retried. And your log messages show a ServiceInterruption was indeed
> processed, which will cause a retry to occur until the time interval has
> elapsed:
>
>
>
> >>>>>>
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> <<<<<<
>
>
>
> When a document causes Tika to crash, always, eventually ManifoldCF will
> give up and shut the job down, even though it's retried a number of times.
> I warned you about this earlier; that may be what is happening here. It
> would be good if you could verify that.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> After the error, the job stop.
>
>
>
> After, I start the job and I obtain many times:
>
>
>
> WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
>
>
> And teh job currently is running
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 17:41
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> For any one of these warnings, e.g.:
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998
>
>
>
> ... look in the log for the next log message for "Worker thread '87'".
> See what it did next and when.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I built and I am using your trunk.
>
> Thanks a lot.
>
>
>
> But I have always the error on the job.
>
>
>
> In the end of the manifoldcf.log I read:
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
>
>
>
>
> So, I don’t understand if the worker tried to reconnect after 10 seconds
> or not
>
>
>
> How could I check it?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 13:00
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> The easiest way to do it is just to check out current trunk:
>
>
>
> svn co https://svn.apache.org/repos/asf/manifoldcf/trunk
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=L1nnYx4w0WEr98mwWcavZYMX91kInD4YvU234V53ezA%3D&reserved=0>
>
>
>
> No need to apply a patch then. Just build:
>
>
>
> ant make-core-deps
>
> ant make-deps
>
> ant build
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Thanks a lot Karl.
>
>
>
> To overwrite the connector with your patch, have I to download the trunk
> and recompile, isn’t it?
>
>
>
> Excuse me for my questions but I am not expert on programming, compiling,
> etc.
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:48
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> (1) I increased the retries to go at least 10 minutes.
>
> (2) I handled the 503 response explicitly, with the same logic.
>
> See: https://issues.apache.org/jira/browse/CONNECTORS-1556
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=JwsO1Fo%2B59DRoYUSmKEBZWoUCcD3AyBsl%2Fe%2F%2FgEw21M%3D&reserved=0>
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=S1jcTBEdqvsyUr6CCYp03jtv%2B1ECucEpi3w848qKdhs%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=mizTXaEbCpdbbTRrcZbOWMyZlmpyIqSpfozbl7h38C4%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Ki%2FfU5wUBpjRVpssXjyXFOWuUWlciUMngogWhA7xYSg%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
Hallo Karl.
I enlarge the timeout to process the documents on Tika server and the job didn’t stop about repeatidly interruption but now the job is running but hangs.
In manifoldcf.log I see:
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-26T18:13:39,050 (Worker thread '63') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-11-26T18:13:50,868 (Agents idle cleanup thread) - Error tossed: Can't release lock we don't hold
java.lang.IllegalStateException: Can't release lock we don't hold
at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.releaseLock(ZooKeeperConnection.java:815) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearLock(ZooKeeperLockObject.java:218) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockObject.clearGlobalWriteLockNoWait(ZooKeeperLockObject.java:100) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockObject.clearGlobalWriteLock(LockObject.java:160) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockObject.leaveWriteLock(LockObject.java:141) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.LockGate.leaveWriteLock(LockGate.java:205) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWrite(BaseLockManager.java:1224) ~[mcf-core.jar:?]
at org.apache.manifoldcf.core.lockmanager.BaseLockManager.leaveWriteLock(BaseLockManager.java:771) ~[mcf-core.jar:?]
at org.apache.manifoldcf.connectorcommon.throttler.ThrottleBin.poll(ThrottleBin.java:391) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroup.poll(Throttler.java:952) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler$ThrottlingGroups.poll(Throttler.java:290) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.Throttler.poll(Throttler.java:165) ~[?:?]
at org.apache.manifoldcf.connectorcommon.throttler.ThrottleGroups.poll(ThrottleGroups.java:114) ~[?:?]
at org.apache.manifoldcf.connectorcommon.interfaces.ThrottleGroupsFactory$ThrottlerPoll.doPoll(ThrottleGroupsFactory.java:65) ~[?:?]
at org.apache.manifoldcf.core.system.ManifoldCF.pollAll(ManifoldCF.java:1491) ~[mcf-core.jar:?]
at org.apache.manifoldcf.agents.system.IdleCleanupThread.run(IdleCleanupThread.java:93) [mcf-agents.jar:?]
It seems that the agent crashed?
What could I do?
I start the agent at the startup of the machine with the command, scheduled on crontab, by:
@reboot /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh
Thanks a lot Karl!
Da: Karl Wright <da...@gmail.com>
Inviato: lunedì 26 novembre 2018 21:51
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
As you can see, ManifoldCF retries some error conditions until it either succeeds or until it must finally give up. At that point it can choose to abort the job or skip the document, depending on the error condition.
The reason I said to look for worker thread 87 in the first log was because I wanted to see if retried after that, or not. The change I committed should make it retry for a period of time (10 minutes) before giving up. The snippet of log you initially included showed a number of threads having difficulty connecting to the Tika service; that should be retried. And your log messages show a ServiceInterruption was indeed processed, which will cause a retry to occur until the time interval has elapsed:
>>>>>>
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135%2F&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2mQreWq8Ibimh%2BdGXPi4TvlV7EYAYG7k%2Fl9SgERqBkY%3D&reserved=0>] failed: Connection refused (Connection refused)
<<<<<<
When a document causes Tika to crash, always, eventually ManifoldCF will give up and shut the job down, even though it's retried a number of times. I warned you about this earlier; that may be what is happening here. It would be good if you could verify that.
Karl
On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>> wrote:
After the error, the job stop.
After, I start the job and I obtain many times:
WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
And teh job currently is running
Da: Karl Wright <da...@gmail.com>>
Inviato: lunedì 26 novembre 2018 17:41
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
For any one of these warnings, e.g.:
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998
... look in the log for the next log message for "Worker thread '87'". See what it did next and when.
Karl
On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I built and I am using your trunk.
Thanks a lot.
But I have always the error on the job.
In the end of the manifoldcf.log I read:
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FQqL%2F747wqxwRuOIpHViPDyJpAOQdjKXmnk8J8DUTac%3D&reserved=0>] failed: Connection refused (Connection refused)
So, I don’t understand if the worker tried to reconnect after 10 seconds or not
How could I check it?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 13:00
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
The easiest way to do it is just to check out current trunk:
svn co https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=L1nnYx4w0WEr98mwWcavZYMX91kInD4YvU234V53ezA%3D&reserved=0>
No need to apply a patch then. Just build:
ant make-core-deps
ant make-deps
ant build
Karl
On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Thanks a lot Karl.
To overwrite the connector with your patch, have I to download the trunk and recompile, isn’t it?
Excuse me for my questions but I am not expert on programming, compiling, etc.
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:48
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=JwsO1Fo%2B59DRoYUSmKEBZWoUCcD3AyBsl%2Fe%2F%2FgEw21M%3D&reserved=0>
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=S1jcTBEdqvsyUr6CCYp03jtv%2B1ECucEpi3w848qKdhs%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=mizTXaEbCpdbbTRrcZbOWMyZlmpyIqSpfozbl7h38C4%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=BqHNJBNYNF4To%2BJn87EeUv4l8ryoZM5ctZhfyYnzAg4%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7ac764f1807341ef9ced08d653e0eb06%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Ki%2FfU5wUBpjRVpssXjyXFOWuUWlciUMngogWhA7xYSg%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
Hi Mario,
As you can see, ManifoldCF retries some error conditions until it either
succeeds or until it must finally give up. At that point it can choose to
abort the job or skip the document, depending on the error condition.
The reason I said to look for worker thread 87 in the first log was because
I wanted to see if retried after that, or not. The change I committed
should make it retry for a period of time (10 minutes) before giving up.
The snippet of log you initially included showed a number of threads having
difficulty connecting to the Tika service; that should be retried. And
your log messages show a ServiceInterruption was indeed processed, which
will cause a retry to occur until the time interval has elapsed:
>>>>>>
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
172.16.1.135] failed: Connection refused (Connection refused)
<<<<<<
When a document causes Tika to crash, always, eventually ManifoldCF will
give up and shut the job down, even though it's retried a number of times.
I warned you about this earlier; that may be what is happening here. It
would be good if you could verify that.
Karl
On Mon, Nov 26, 2018 at 12:19 PM Bisonti Mario <Ma...@vimar.com>
wrote:
> After the error, the job stop.
>
>
>
> After, I start the job and I obtain many times:
>
>
>
> WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
> WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> .
>
> .
>
>
>
> And teh job currently is running
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 26 novembre 2018 17:41
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> For any one of these warnings, e.g.:
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998
>
>
>
> ... look in the log for the next log message for "Worker thread '87'".
> See what it did next and when.
>
>
>
> Karl
>
>
>
>
>
> On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I built and I am using your trunk.
>
> Thanks a lot.
>
>
>
> But I have always the error on the job.
>
>
>
> In the end of the manifoldcf.log I read:
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>]
> failed: Connection refused (Connection refused)
>
>
>
>
>
> So, I don’t understand if the worker tried to reconnect after 10 seconds
> or not
>
>
>
> How could I check it?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 13:00
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> The easiest way to do it is just to check out current trunk:
>
>
>
> svn co https://svn.apache.org/repos/asf/manifoldcf/trunk
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=NU3K0vtfDCnq7o5d1Hp4a0wG3jKcFfX3nuydbGZIjbA%3D&reserved=0>
>
>
>
> No need to apply a patch then. Just build:
>
>
>
> ant make-core-deps
>
> ant make-deps
>
> ant build
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Thanks a lot Karl.
>
>
>
> To overwrite the connector with your patch, have I to download the trunk
> and recompile, isn’t it?
>
>
>
> Excuse me for my questions but I am not expert on programming, compiling,
> etc.
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:48
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> (1) I increased the retries to go at least 10 minutes.
>
> (2) I handled the 503 response explicitly, with the same logic.
>
> See: https://issues.apache.org/jira/browse/CONNECTORS-1556
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=JOahohm10Ha06%2B8Ee16dXn%2FfBEcfwrOdcZ4YF9aHrfc%3D&reserved=0>
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=mi57BJ%2FZGvVz%2F7R9sFXoGEHQiS0REIgZ0pLOWK7EE04%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=g2cej6k%2BMbkCknrOu%2FQ6w78q5szuNAKzDWKk26TLHIA%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xlyMcV%2Bd28onzEvo11jUC1%2BNs2lhmIPqFdZrCrkx4as%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
After the error, the job stop.
After, I start the job and I obtain many times:
WARN 2018-11-26T16:32:55,003 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
WARN 2018-11-26T17:18:34,709 (Worker thread '87') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
.
.
And teh job currently is running
Da: Karl Wright <da...@gmail.com>
Inviato: lunedì 26 novembre 2018 17:41
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
For any one of these warnings, e.g.:
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998
... look in the log for the next log message for "Worker thread '87'". See what it did next and when.
Karl
On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I built and I am using your trunk.
Thanks a lot.
But I have always the error on the job.
In the end of the manifoldcf.log I read:
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F172.16.1.135&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xq8ZKgw8pzlmJLN6aZzaPteyEdT%2B8nL2Yo9ZDADz0RI%3D&reserved=0>] failed: Connection refused (Connection refused)
So, I don’t understand if the worker tried to reconnect after 10 seconds or not
How could I check it?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 13:00
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
The easiest way to do it is just to check out current trunk:
svn co https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=NU3K0vtfDCnq7o5d1Hp4a0wG3jKcFfX3nuydbGZIjbA%3D&reserved=0>
No need to apply a patch then. Just build:
ant make-core-deps
ant make-deps
ant build
Karl
On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Thanks a lot Karl.
To overwrite the connector with your patch, have I to download the trunk and recompile, isn’t it?
Excuse me for my questions but I am not expert on programming, compiling, etc.
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:48
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=JOahohm10Ha06%2B8Ee16dXn%2FfBEcfwrOdcZ4YF9aHrfc%3D&reserved=0>
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=mi57BJ%2FZGvVz%2F7R9sFXoGEHQiS0REIgZ0pLOWK7EE04%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=g2cej6k%2BMbkCknrOu%2FQ6w78q5szuNAKzDWKk26TLHIA%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=v0515gDSO4iLI2RwxPGXWb08lay%2Blq6IUdeZdUjEDAw%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C530da6a96c5b4a13aa9408d653be0c79%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xlyMcV%2Bd28onzEvo11jUC1%2BNs2lhmIPqFdZrCrkx4as%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
For any one of these warnings, e.g.:
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
Connect to sengvivv01.local.domain:9998
... look in the log for the next log message for "Worker thread '87'". See
what it did next and when.
Karl
On Mon, Nov 26, 2018 at 10:41 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo Karl.
>
> I built and I am using your trunk.
>
> Thanks a lot.
>
>
>
> But I have always the error on the job.
>
>
>
> In the end of the manifoldcf.log I read:
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
> WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/
> 172.16.1.135] failed: Connection refused (Connection refused)
>
>
>
>
>
> So, I don’t understand if the worker tried to reconnect after 10 seconds
> or not
>
>
>
> How could I check it?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 13:00
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> The easiest way to do it is just to check out current trunk:
>
>
>
> svn co https://svn.apache.org/repos/asf/manifoldcf/trunk
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=H%2BeYb5xRnrIp%2FIlcR%2F%2FR2FaX8KjYR3Ec6Dvyeii8OnU%3D&reserved=0>
>
>
>
> No need to apply a patch then. Just build:
>
>
>
> ant make-core-deps
>
> ant make-deps
>
> ant build
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Thanks a lot Karl.
>
>
>
> To overwrite the connector with your patch, have I to download the trunk
> and recompile, isn’t it?
>
>
>
> Excuse me for my questions but I am not expert on programming, compiling,
> etc.
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:48
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> (1) I increased the retries to go at least 10 minutes.
>
> (2) I handled the 503 response explicitly, with the same logic.
>
> See: https://issues.apache.org/jira/browse/CONNECTORS-1556
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=LWrGl7ELqT4IZRvO0%2Fq43m1W6D40jY0RvU5JA%2F2fjLk%3D&reserved=0>
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=wXG6Jw3eq4rDtrCMqQ2zF%2FoN1oOoExaUCdgeOjbWrMc%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=u7jF3EMriH2diLdxdiDn5l0pwEZKB0b9FIeC0LqgOvY%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=ByTQBrjzn4nuwM7X73uUWBOyUNzpZ6f6NcQkGa3fv2o%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
Hallo Karl.
I built and I am using your trunk.
Thanks a lot.
But I have always the error on the job.
In the end of the manifoldcf.log I read:
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-26T13:18:25,913 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,039 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,179 (Worker thread '8') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,232 (Worker thread '86') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,329 (Worker thread '87') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,668 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,722 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '75') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
WARN 2018-11-26T13:18:26,862 (Worker thread '12') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to sengvivv01.local.domain:9998 [sengvivv01.local.domain/172.16.1.135] failed: Connection refused (Connection refused)
So, I don’t understand if the worker tried to reconnect after 10 seconds or not
How could I check it?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>
Inviato: giovedì 15 novembre 2018 13:00
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
The easiest way to do it is just to check out current trunk:
svn co https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=H%2BeYb5xRnrIp%2FIlcR%2F%2FR2FaX8KjYR3Ec6Dvyeii8OnU%3D&reserved=0>
No need to apply a patch then. Just build:
ant make-core-deps
ant make-deps
ant build
Karl
On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Thanks a lot Karl.
To overwrite the connector with your patch, have I to download the trunk and recompile, isn’t it?
Excuse me for my questions but I am not expert on programming, compiling, etc.
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:48
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=LWrGl7ELqT4IZRvO0%2Fq43m1W6D40jY0RvU5JA%2F2fjLk%3D&reserved=0>
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=wXG6Jw3eq4rDtrCMqQ2zF%2FoN1oOoExaUCdgeOjbWrMc%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=u7jF3EMriH2diLdxdiDn5l0pwEZKB0b9FIeC0LqgOvY%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=HBvaloOWfu7U4P9vlx4E0HLbvWezFkot%2FFXzvAZ6bUE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ce35d2a3ecf744ed254da08d64af1ecf0%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=ByTQBrjzn4nuwM7X73uUWBOyUNzpZ6f6NcQkGa3fv2o%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
The easiest way to do it is just to check out current trunk:
svn co https://svn.apache.org/repos/asf/manifoldcf/trunk
No need to apply a patch then. Just build:
ant make-core-deps
ant make-deps
ant build
Karl
On Thu, Nov 15, 2018 at 4:30 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Thanks a lot Karl.
>
>
>
> To overwrite the connector with your patch, have I to download the trunk
> and recompile, isn’t it?
>
>
>
> Excuse me for my questions but I am not expert on programming, compiling,
> etc.
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:48
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> (1) I increased the retries to go at least 10 minutes.
>
> (2) I handled the 503 response explicitly, with the same logic.
>
> See: https://issues.apache.org/jira/browse/CONNECTORS-1556
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2taQ0zeXdM9VOD1LAA%2F%2B4gCW4ToX99CufaxplUbxgBU%3D&reserved=0>
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=H40PxbAsUH56Dn6cWkSxzQtKgfdDle6rllc%2Fvqzvl8E%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=wF7C%2B8VedS8HOE5Pox5bNyYeQGGUdod0Lmt4m547PtQ%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=MkMfFOb6ls58c2UNQqSDsja1ND1%2BZVsiB1BsnLZDuLk%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
Thanks a lot Karl.
To overwrite the connector with your patch, have I to download the trunk and recompile, isn’t it?
Excuse me for my questions but I am not expert on programming, compiling, etc.
Da: Karl Wright <da...@gmail.com>
Inviato: giovedì 15 novembre 2018 09:48
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1556&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=2taQ0zeXdM9VOD1LAA%2F%2B4gCW4ToX99CufaxplUbxgBU%3D&reserved=0>
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=H40PxbAsUH56Dn6cWkSxzQtKgfdDle6rllc%2Fvqzvl8E%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=wF7C%2B8VedS8HOE5Pox5bNyYeQGGUdod0Lmt4m547PtQ%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=4FxNdl74pig8iz4y1Y%2F4UIQVupUq52Z38rNwZdLbDsE%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C7d6cdb8059a54ddba5ee08d64ad715e5%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=MkMfFOb6ls58c2UNQqSDsja1ND1%2BZVsiB1BsnLZDuLk%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
(1) I increased the retries to go at least 10 minutes.
(2) I handled the 503 response explicitly, with the same logic.
See: https://issues.apache.org/jira/browse/CONNECTORS-1556
Karl
On Thu, Nov 15, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Yes, Karl.
>
>
>
> Is it possible to apply the same your concept , wait 10 sec and retry
> three times , to the 503 error , too?
>
>
>
> So, I would like to try, if, with the modification, I obtain that job end
> correctly instead of failure.
>
>
>
>
>
> Thanks a lot
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 15 novembre 2018 09:17
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> Here's the code:
>
>
>
> >>>>>>
>
> try {
>
> //System.out.println("About to do a content PUT");
>
> response = this.httpClient.execute(tikaHost, httpPut);
>
> //System.out.println("... content PUT succeeded");
>
> } catch (IOException e) {
>
> // Retry 3 times, 10000 ms between retries, and abort if
> doesn't work
>
> final long currentTime = System.currentTimeMillis();
>
> throw new ServiceInterruption("Tika down, retrying:
> "+e.getMessage(),e,currentTime + 10000L,
>
> -1L,3,true);
>
> }
>
>
>
> responseCode = response.getStatusLine().getStatusCode();
>
> if (response.getStatusLine().getStatusCode() == 200 ||
> response.getStatusLine().getStatusCode() == 204) {
>
> tikaServerIs = response.getEntity().getContent();
>
> try {
>
> responseDs = new FileDestinationStorage();
>
> final OutputStream os2 = responseDs.getOutputStream();
>
> try {
>
> IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
>
> } finally {
>
> os2.close();
>
> }
>
> length = new Long(responseDs.getBinaryLength());
>
> } finally {
>
> tikaServerIs.close();
>
> }
>
> } else {
>
> activities.noDocument();
>
> if (responseCode == 422) {
>
> resultCode = "TIKASERVERREJECTS";
>
> description = "Tika Server rejected document with the
> following reason: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerRejects(description);
>
> } else {
>
> resultCode = "TIKASERVERERROR";
>
> description = "Tika Server failed to parse document with
> the following error: "
>
> + response.getStatusLine().getReasonPhrase();
>
> return handleTikaServerError(description);
>
> }
>
> }
>
>
>
> } catch (IOException | ParseException e) {
>
> resultCode = "TIKASERVERRESPONSEISSUE";
>
> description = e.getMessage();
>
> int rval;
>
> if (e instanceof IOException) {
>
> rval = handleTikaServerException((IOException) e);
>
> } else {
>
> rval = handleTikaServerException((ParseException) e);
>
> }
>
> if (rval == DOCUMENTSTATUS_REJECTED) {
>
> activities.noDocument();
>
> }
>
> return rval;
>
> }
>
> <<<<<<
>
> and
>
> >>>>>>
>
> protected static int handleTikaServerError(String description)
>
> throws IOException, ManifoldCFException, ServiceInterruption {
>
> // MHL - what does Tika throw if it gets an IOException reading the
> stream??
>
> Logging.ingest.warn("Tika Server: Tika Server error: " + description);
>
> return DOCUMENTSTATUS_REJECTED;
>
> }
>
> <<<<<<
>
>
>
> The summary:
>
> (1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
> at least 10 seconds and then retry -- up to three times.
>
> (2) When Manifold sees a 503 error it immediately just rejects the
> document.
>
> So you are requesting different handling for 503 errors?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FcT8zGM2tDjYAPgal%2F2ZE7ql969t%2FCrjw1mH7iW%2BcGI%3D&reserved=0>
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=b3g9uENO8UmC1c9f3gK19mxGXzhjqaj9z01ZcEUe6F4%3D&reserved=0>
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=6Fag3tDYKFv%2B%2BOI1kATaBhnWTfta%2BT6Q76497%2FnyN1o%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
Yes, Karl.
Is it possible to apply the same your concept , wait 10 sec and retry three times , to the 503 error , too?
So, I would like to try, if, with the modification, I obtain that job end correctly instead of failure.
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>
Inviato: giovedì 15 novembre 2018 09:17
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying: "+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 || response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=FcT8zGM2tDjYAPgal%2F2ZE7ql969t%2FCrjw1mH7iW%2BcGI%3D&reserved=0>
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FTIKA-2776%3FfocusedCommentId%3D16686620%26page%3Dcom.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel%23comment-16686620&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=b3g9uENO8UmC1c9f3gK19mxGXzhjqaj9z01ZcEUe6F4%3D&reserved=0>
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=7N%2BdrgAOmEtmxPNRb7gnopRoYlDzDWXALxfqEO2EEDI%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C08b7435f3a454c15a50408d64ad2c224%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=6Fag3tDYKFv%2B%2BOI1kATaBhnWTfta%2BT6Q76497%2FnyN1o%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
Hi Mario,
Here's the code:
>>>>>>
try {
//System.out.println("About to do a content PUT");
response = this.httpClient.execute(tikaHost, httpPut);
//System.out.println("... content PUT succeeded");
} catch (IOException e) {
// Retry 3 times, 10000 ms between retries, and abort if
doesn't work
final long currentTime = System.currentTimeMillis();
throw new ServiceInterruption("Tika down, retrying:
"+e.getMessage(),e,currentTime + 10000L,
-1L,3,true);
}
responseCode = response.getStatusLine().getStatusCode();
if (response.getStatusLine().getStatusCode() == 200 ||
response.getStatusLine().getStatusCode() == 204) {
tikaServerIs = response.getEntity().getContent();
try {
responseDs = new FileDestinationStorage();
final OutputStream os2 = responseDs.getOutputStream();
try {
IOUtils.copyLarge(tikaServerIs, os2, 0L, sp.writeLimit);
} finally {
os2.close();
}
length = new Long(responseDs.getBinaryLength());
} finally {
tikaServerIs.close();
}
} else {
activities.noDocument();
if (responseCode == 422) {
resultCode = "TIKASERVERREJECTS";
description = "Tika Server rejected document with the
following reason: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerRejects(description);
} else {
resultCode = "TIKASERVERERROR";
description = "Tika Server failed to parse document with
the following error: "
+ response.getStatusLine().getReasonPhrase();
return handleTikaServerError(description);
}
}
} catch (IOException | ParseException e) {
resultCode = "TIKASERVERRESPONSEISSUE";
description = e.getMessage();
int rval;
if (e instanceof IOException) {
rval = handleTikaServerException((IOException) e);
} else {
rval = handleTikaServerException((ParseException) e);
}
if (rval == DOCUMENTSTATUS_REJECTED) {
activities.noDocument();
}
return rval;
}
<<<<<<
and
>>>>>>
protected static int handleTikaServerError(String description)
throws IOException, ManifoldCFException, ServiceInterruption {
// MHL - what does Tika throw if it gets an IOException reading the
stream??
Logging.ingest.warn("Tika Server: Tika Server error: " + description);
return DOCUMENTSTATUS_REJECTED;
}
<<<<<<
The summary:
(1) If ManifoldCF cannot connect at all, or gets an IO error, it will wait
at least 10 seconds and then retry -- up to three times.
(2) When Manifold sees a 503 error it immediately just rejects the document.
So you are requesting different handling for 503 errors?
Karl
On Thu, Nov 15, 2018 at 2:42 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo Karl.
>
> I opened an issue on Tika here:
>
> https://issues.apache.org/jira/browse/TIKA-2776
>
>
>
> The person that develops tika, suggests me to put a waiting on the client
> (in my case manifoldcf)
>
>
> https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
>
>
>
> I am not able to do this…
>
> Is it possible to implement on the MCF source?
>
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* giovedì 8 novembre 2018 21:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Error Job stop after repeatidly interruption
>
>
>
> Hi Mario,
>
>
>
> The Tika external connector retries for a while before it gives up and
> aborts the job. If you can get the Tika server back up within a reasonable
> period of time all should be well. But if one specific document *always*
> brings down the Tika server, it will be hard to recover from that.
>
>
>
> Karl
>
>
>
>
>
> On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1
> <https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=nFzraVQFabwO7U5wRYC3GwFxIHMH5Y9wd5o8%2FebPkNA%3D&reserved=0>)
>
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
R: Error Job stop after repeatidly interruption
Posted by Bisonti Mario <Ma...@vimar.com>.
Hallo Karl.
I opened an issue on Tika here:
https://issues.apache.org/jira/browse/TIKA-2776
The person that develops tika, suggests me to put a waiting on the client (in my case manifoldcf)
https://issues.apache.org/jira/browse/TIKA-2776?focusedCommentId=16686620&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16686620
I am not able to do this…
Is it possible to implement on the MCF source?
Thanks a lot
Mario
Da: Karl Wright <da...@gmail.com>
Inviato: giovedì 8 novembre 2018 21:03
A: user@manifoldcf.apache.org
Oggetto: Re: Error Job stop after repeatidly interruption
Hi Mario,
The Tika external connector retries for a while before it gives up and aborts the job. If you can get the Tika server back up within a reasonable period of time all should be well. But if one specific document *always* brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
I am trying to index more than 5000000 documents in a Windows Share.
It happens that job is interrupted due to repeatidly interruption.
This is the manifold.log:
.
.
WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: The target server failed to respond
ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed: Repeated service interruptions - failure processing document: The target server failed to respond
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated service interruptions - failure processing document: The target server failed to respond
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489) [mcf-pull-agent.jar:?]
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[httpcore-4.4.10.jar:4.4.10]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) ~[httpclient-4.5.6.jar:4.5.6]
at org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) ~[mcf-pull-agent.jar:?]
WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption reported for job 1533797717712 connection 'WinShare': Tika down, retrying: Connect to localhost:9998 [localhost/127.0.0.1<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F127.0.0.1&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=Yeum49pwR3TPvu1lIS5UTnVX6A%2BimwqT61tkdFb1Mkc%3D&reserved=0>, localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
My Tika server is standalone and started with the option “-spawnChild”
Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
This option, permits to Tika to restart a process f it crash for some cause as reported here:
As of Tika 1.19, users can make tika-server more robust by running it with the -spawnChild option. This starts tika-server in a child process, and if there's an OOM, a timeout or other catastrophic problem with the child process, the parent process will kill and/or restart the child process.
(https://wiki.apache.org/tika/TikaJAXRS<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.apache.org%2Ftika%2FTikaJAXRS&data=01%7C01%7CMario.Bisonti%40vimar.com%7C266767b739214f1fd13608d645ba4cfe%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=nFzraVQFabwO7U5wRYC3GwFxIHMH5Y9wd5o8%2FebPkNA%3D&reserved=0>)
I suppose, that manifold, when many times Tika server restart a child, it collects an interruption, and after many interruption the job stop.
After that, I need to start again the job.
Is there any way to workaround this?
Thanks a lot
Mario
Re: Error Job stop after repeatidly interruption
Posted by Karl Wright <da...@gmail.com>.
Hi Mario,
The Tika external connector retries for a while before it gives up and
aborts the job. If you can get the Tika server back up within a reasonable
period of time all should be well. But if one specific document *always*
brings down the Tika server, it will be hard to recover from that.
Karl
On Thu, Nov 8, 2018 at 2:56 PM Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo.
>
>
>
> I am trying to index more than 5000000 documents in a Windows Share.
>
>
>
> It happens that job is interrupted due to repeatidly interruption.
>
> This is the manifold.log:
>
> .
> .
> WARN 2018-11-07T21:53:25,296 (Worker thread '59') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:25,476 (Worker thread '89') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:53:33,814 (Worker thread '15') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:53:57,861 (Worker thread '12') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,765 (Worker thread '24') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,770 (Worker thread '4') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,775 (Worker thread '95') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> WARN 2018-11-07T21:56:04,793 (Worker thread '54') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> The target server failed to respond
>
> ERROR 2018-11-07T21:56:04,802 (Worker thread '24') - Exception tossed:
> Repeated service interruptions - failure processing document: The target
> server failed to respond
>
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: Repeated
> service interruptions - failure processing document: The target server
> failed to respond
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:489)
> [mcf-pull-agent.jar:?]
>
> Caused by: org.apache.http.NoHttpResponseException: The target server
> failed to respond
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
> ~[httpcore-4.4.10.jar:4.4.10]
>
> at
> org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:118)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
> ~[httpclient-4.5.6.jar:4.5.6]
>
> at
> org.apache.manifoldcf.agents.transformation.tikaservice.TikaExtractor.addOrReplaceDocumentWithException(TikaExtractor.java:608)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$MonitoredAddActivityWrapper.sendDocument(IncrementalIngester.java:3471)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.transformation.documentfilter.DocumentFilter.addOrReplaceDocumentWithException(DocumentFilter.java:208)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3226)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.java:2708)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.documentIngest(IncrementalIngester.java:756)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1583)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ingestDocumentWithException(WorkerThread.java:1548)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:939)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> ~[mcf-pull-agent.jar:?]
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '33') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,866 (Worker thread '22') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,872 (Worker thread '92') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '37') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,883 (Worker thread '82') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,952 (Worker thread '13') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,953 (Worker thread '0') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '47') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '97') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:04,976 (Worker thread '84') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,004 (Worker thread '19') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,053 (Worker thread '16') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,079 (Worker thread '20') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,080 (Worker thread '76') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,114 (Worker thread '63') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,117 (Worker thread '93') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,143 (Worker thread '43') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '9') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,191 (Worker thread '44') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,262 (Worker thread '99') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,276 (Worker thread '11') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,327 (Worker thread '48') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,353 (Worker thread '5') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,389 (Worker thread '31') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,414 (Worker thread '96') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,511 (Worker thread '65') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,514 (Worker thread '52') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,545 (Worker thread '39') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,561 (Worker thread '88') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,569 (Worker thread '72') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,640 (Worker thread '27') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,661 (Worker thread '51') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,765 (Worker thread '81') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,769 (Worker thread '2') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,787 (Worker thread '1') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,859 (Worker thread '83') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,886 (Worker thread '55') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,892 (Worker thread '3') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '77') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '40') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
> WARN 2018-11-07T21:56:05,910 (Worker thread '32') - Service interruption
> reported for job 1533797717712 connection 'WinShare': Tika down, retrying:
> Connect to localhost:9998 [localhost/127.0.0.1,
> localhost/0:0:0:0:0:0:0:1] failed: Connection refused (Connection refused)
>
>
>
>
>
>
>
>
>
> My Tika server is standalone and started with the option “-spawnChild”
>
> Java -jar /opt/tika/tika-server-1.19.jar -spawnChild
>
> This option, permits to Tika to restart a process f it crash for some
> cause as reported here:
>
> As of Tika 1.19, users can make tika-server more robust by running it with
> the -spawnChild option. This starts tika-server in a child process, and if
> there's an OOM, a timeout or other catastrophic problem with the child
> process, the parent process will kill and/or restart the child process.
>
>
>
> (https://wiki.apache.org/tika/TikaJAXRS)
>
>
>
> I suppose, that manifold, when many times Tika server restart a child, it
> collects an interruption, and after many interruption the job stop.
>
>
>
> After that, I need to start again the job.
>
>
>
> Is there any way to workaround this?
>
>
>
> Thanks a lot
>
>
>
> Mario
>