You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Bisonti Mario <Ma...@vimar.com> on 2018/06/18 16:34:40 UTC
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed:
null
Hallo
I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
I configured multiprocess-file-example
When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,) manifoldcf crash with the message:
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
at org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811) ~[?:?]
at org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874) ~[?:?]
at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
If I use a smaller windows share it works.
Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.
What could I do?
Thanks a lot
Mario
R: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed:
null
Posted by Bisonti Mario <Ma...@vimar.com>.
Hallo Karl!
Now I found how to build (my first building … 😊 ) and now I am using:
/multiprocess-file-example-proprietary folder and I deployed into Tomcat .
I recreated the configuration that I used on binary version and I created the same job
It works !!!
I see on manifold cf.lg the error:
WARN 2018-06-19T12:11:52,476 (Worker thread '5') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-06-19T12:11:53,435 (Worker thread '3') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs.jar:?]
but it doesn’t stuck no more , so for me it isn’t a problem if they are only warning!
Very very good!
Thanks for your help!
Mario
Da: Karl Wright <da...@gmail.com>
Inviato: martedì 19 giugno 2018 09:52
A: user@manifoldcf.apache.org
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
Hi Mario,
You cannot patch the binary. You must build from source to apply the patch.
The easiest way forward is to check out trunk directly (with svn) and build it. The trunk svn URL is https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ccea091a50fe347b3c2cc08d5d5b9888e%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=XzTfotr%2B8KS4wjHws3sKU79U9xgrWwfSAS4qK7Y%2F7NI%3D&reserved=0> .
Karl
On Tue, Jun 19, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo.
Note that I specified the mime types on my solr output connection
Furthermore, I used the binary distribution, how cold I path it with tour fix?
I read on my job, that stuck up on 3 docs with:
WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs-1.3.19.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null
Thanks a lot for your great help
Da: Karl Wright <da...@gmail.com>>
Inviato: lunedì 18 giugno 2018 20:39
A: user@manifoldcf.apache.org<ma...@manifoldcf.apache.org>
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
Created CONNECTORS-1510 and committed a fix.
Karl
On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <da...@gmail.com>> wrote:
It certainly is a particular file -- the mime type is null, and that's causing this line to blow up:
final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
That code was added a couple of revs back to address a different problem; it's a trivial fix:
final String lowerMimeType = (mimeType != null)?mimeType.toLowerCase(Locale.ROOT):null;
(This is HttpPoster line 811)
Karl
On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <st...@remcam.net>> wrote:
Looks like a particular file may be causing this. Try to find the filanem it crashes on and copy that to asmall crawl directory. Repeat crawl.
On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo
I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
I configured multiprocess-file-example
When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,) manifoldcf crash with the message:
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
at org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811) ~[?:?]
at org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874) ~[?:?]
at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
If I use a smaller windows share it works.
Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.
What could I do?
Thanks a lot
Mario
Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error
tossed: null
Posted by Karl Wright <da...@gmail.com>.
Hi Mario,
You cannot patch the binary. You must build from source to apply the patch.
The easiest way forward is to check out trunk directly (with svn) and build
it. The trunk svn URL is https://svn.apache.org/repos/asf/manifoldcf/trunk
.
Karl
On Tue, Jun 19, 2018 at 3:35 AM Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo.
>
> Note that I specified the mime types on my solr output connection
>
>
>
> Furthermore, I used the binary distribution, how cold I path it with tour
> fix?
>
>
>
> I read on my job, that stuck up on 3 docs with:
>
>
>
> WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
> at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbTransport.send(SmbTransport.java:663)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
>
> at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.19.jar:?]
>
> at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.19.jar:?]
>
> at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.19.jar:?]
>
> at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.19.jar:?]
>
> at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946)
> ~[jcifs-1.3.19.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
>
> java.lang.NullPointerException
>
> FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null
>
>
>
>
>
> Thanks a lot for your great help
>
>
>
> *Da:* Karl Wright <da...@gmail.com>
> *Inviato:* lunedì 18 giugno 2018 20:39
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error
> tossed: null
>
>
>
> Created CONNECTORS-1510 and committed a fix.
>
> Karl
>
>
>
> On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <da...@gmail.com> wrote:
>
> It certainly is a particular file -- the mime type is null, and that's
> causing this line to blow up:
>
> final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
>
>
>
>
>
> That code was added a couple of revs back to address a different problem;
> it's a trivial fix:
>
>
>
> final String lowerMimeType = (mimeType !=
> null)?mimeType.toLowerCase(Locale.ROOT):null;
>
>
> (This is HttpPoster line 811)
>
>
>
> Karl
>
>
>
>
>
> On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <st...@remcam.net>
> wrote:
>
>
>
> Looks like a particular file may be causing this. Try to find the filanem
> it crashes on and copy that to asmall crawl directory. Repeat crawl.
>
>
>
>
>
> On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Ma...@vimar.com>
> wrote:
>
> Hallo
>
>
>
> I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
>
>
>
> I configured multiprocess-file-example
>
>
>
>
>
> When I create a Job to scan a big Windows share (22000 docs word, pdf,
> etc,) manifoldcf crash with the message:
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
>
> java.lang.NullPointerException
>
> at
> org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
> ~[?:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
> ~[mcf-agents.jar:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
> ~[mcf-pull-agent.jar:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
> ~[?:?]
>
> at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
>
> at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
>
> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
>
> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
> ~[?:?]
>
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
>
>
>
>
> If I use a smaller windows share it works.
>
>
>
> Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it
> worked.
>
>
>
> What could I do?
>
>
>
> Thanks a lot
>
> Mario
>
>
>
>
R: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed:
null
Posted by Bisonti Mario <Ma...@vimar.com>.
Hallo.
Note that I specified the mime types on my solr output connection
Furthermore, I used the binary distribution, how cold I path it with tour fix?
I read on my job, that stuck up on 3 docs with:
WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.19.jar:?]
at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.19.jar:?]
at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs-1.3.19.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null
Thanks a lot for your great help
Da: Karl Wright <da...@gmail.com>
Inviato: lunedì 18 giugno 2018 20:39
A: user@manifoldcf.apache.org
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
Created CONNECTORS-1510 and committed a fix.
Karl
On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <da...@gmail.com>> wrote:
It certainly is a particular file -- the mime type is null, and that's causing this line to blow up:
final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
That code was added a couple of revs back to address a different problem; it's a trivial fix:
final String lowerMimeType = (mimeType != null)?mimeType.toLowerCase(Locale.ROOT):null;
(This is HttpPoster line 811)
Karl
On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <st...@remcam.net>> wrote:
Looks like a particular file may be causing this. Try to find the filanem it crashes on and copy that to asmall crawl directory. Repeat crawl.
On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Ma...@vimar.com>> wrote:
Hallo
I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
I configured multiprocess-file-example
When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,) manifoldcf crash with the message:
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
at org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811) ~[?:?]
at org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534) ~[?:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273) ~[mcf-agents.jar:?]
at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029) ~[mcf-pull-agent.jar:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874) ~[?:?]
at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318) ~[?:?]
at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798) ~[?:?]
at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
If I use a smaller windows share it works.
Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.
What could I do?
Thanks a lot
Mario
Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error
tossed: null
Posted by Karl Wright <da...@gmail.com>.
Created CONNECTORS-1510 and committed a fix.
Karl
On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <da...@gmail.com> wrote:
> It certainly is a particular file -- the mime type is null, and that's
> causing this line to blow up:
>
> final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
>
>
> That code was added a couple of revs back to address a different problem;
> it's a trivial fix:
>
> final String lowerMimeType = (mimeType !=
> null)?mimeType.toLowerCase(Locale.ROOT):null;
>
> (This is HttpPoster line 811)
>
> Karl
>
>
> On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <st...@remcam.net>
> wrote:
>
>>
>> Looks like a particular file may be causing this. Try to find the filanem
>> it crashes on and copy that to asmall crawl directory. Repeat crawl.
>>
>>
>> On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Ma...@vimar.com>
>> wrote:
>>
>>> Hallo
>>>
>>>
>>>
>>> I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
>>>
>>>
>>>
>>> I configured multiprocess-file-example
>>>
>>>
>>>
>>>
>>>
>>> When I create a Job to scan a big Windows share (22000 docs word, pdf,
>>> etc,) manifoldcf crash with the message:
>>> at
>>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>>> [mcf-pull-agent.jar:?]
>>>
>>> FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
>>>
>>> java.lang.NullPointerException
>>>
>>> at
>>> org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
>>> ~[?:?]
>>>
>>> at
>>> org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
>>> ~[?:?]
>>>
>>> at
>>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
>>> ~[mcf-agents.jar:?]
>>>
>>> at
>>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
>>> ~[mcf-agents.jar:?]
>>>
>>> at
>>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
>>> ~[mcf-agents.jar:?]
>>>
>>> at
>>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
>>> ~[mcf-agents.jar:?]
>>>
>>> at
>>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
>>> ~[mcf-pull-agent.jar:?]
>>>
>>> at
>>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
>>> ~[?:?]
>>>
>>> at
>>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
>>> ~[?:?]
>>>
>>> at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
>>>
>>> at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
>>>
>>> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
>>>
>>> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
>>>
>>> at
>>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
>>> ~[?:?]
>>>
>>> at
>>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
>>> ~[?:?]
>>>
>>> at
>>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>>> [mcf-pull-agent.jar:?]
>>>
>>>
>>>
>>>
>>>
>>> If I use a smaller windows share it works.
>>>
>>>
>>>
>>> Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it
>>> worked.
>>>
>>>
>>>
>>> What could I do?
>>>
>>>
>>>
>>> Thanks a lot
>>>
>>> Mario
>>>
>>
>>
Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error
tossed: null
Posted by Karl Wright <da...@gmail.com>.
It certainly is a particular file -- the mime type is null, and that's
causing this line to blow up:
final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);
That code was added a couple of revs back to address a different problem;
it's a trivial fix:
final String lowerMimeType = (mimeType !=
null)?mimeType.toLowerCase(Locale.ROOT):null;
(This is HttpPoster line 811)
Karl
On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <st...@remcam.net>
wrote:
>
> Looks like a particular file may be causing this. Try to find the filanem
> it crashes on and copy that to asmall crawl directory. Repeat crawl.
>
>
> On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Ma...@vimar.com>
> wrote:
>
>> Hallo
>>
>>
>>
>> I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
>>
>>
>>
>> I configured multiprocess-file-example
>>
>>
>>
>>
>>
>> When I create a Job to scan a big Windows share (22000 docs word, pdf,
>> etc,) manifoldcf crash with the message:
>> at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
>>
>> java.lang.NullPointerException
>>
>> at
>> org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
>> ~[?:?]
>>
>> at
>> org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
>> ~[?:?]
>>
>> at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
>> ~[mcf-agents.jar:?]
>>
>> at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
>> ~[mcf-agents.jar:?]
>>
>> at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
>> ~[mcf-agents.jar:?]
>>
>> at
>> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
>> ~[mcf-agents.jar:?]
>>
>> at
>> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
>> ~[mcf-pull-agent.jar:?]
>>
>> at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
>> ~[?:?]
>>
>> at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
>> ~[?:?]
>>
>> at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
>>
>> at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
>>
>> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
>>
>> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
>>
>> at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
>> ~[?:?]
>>
>> at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
>> ~[?:?]
>>
>> at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>>
>>
>>
>>
>> If I use a smaller windows share it works.
>>
>>
>>
>> Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it
>> worked.
>>
>>
>>
>> What could I do?
>>
>>
>>
>> Thanks a lot
>>
>> Mario
>>
>
>
Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error
tossed: null
Posted by Steph van Schalkwyk <st...@remcam.net>.
Looks like a particular file may be causing this. Try to find the filanem
it crashes on and copy that to asmall crawl directory. Repeat crawl.
On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Ma...@vimar.com>
wrote:
> Hallo
>
>
>
> I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3
>
>
>
> I configured multiprocess-file-example
>
>
>
>
>
> When I create a Job to scan a big Windows share (22000 docs word, pdf,
> etc,) manifoldcf crash with the message:
> at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
>
> java.lang.NullPointerException
>
> at org.apache.manifoldcf.agents.output.solr.HttpPoster.
> checkMimeTypeIndexable(HttpPoster.java:811) ~[?:?]
>
> at org.apache.manifoldcf.agents.output.solr.SolrConnector.
> checkMimeTypeIndexable(SolrConnector.java:534) ~[?:?]
>
> at org.apache.manifoldcf.agents.incrementalingest.
> IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
> ~[mcf-agents.jar:?]
>
> at org.apache.manifoldcf.agents.incrementalingest.
> IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
> ~[mcf-agents.jar:?]
>
> at org.apache.manifoldcf.agents.incrementalingest.
> IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
> ~[mcf-agents.jar:?]
>
> at org.apache.manifoldcf.agents.incrementalingest.
> IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
> ~[mcf-agents.jar:?]
>
> at org.apache.manifoldcf.crawler.system.WorkerThread$
> ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
> ~[mcf-pull-agent.jar:?]
>
> at org.apache.manifoldcf.crawler.connectors.sharedrive.
> SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
> ~[?:?]
>
> at org.apache.manifoldcf.crawler.connectors.sharedrive.
> SharedDriveConnector$ProcessDocumentsFilter.accept(
> SharedDriveConnector.java:4874) ~[?:?]
>
> at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
>
> at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
>
> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
>
> at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
>
> at org.apache.manifoldcf.crawler.connectors.sharedrive.
> SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318) ~[?:?]
>
> at org.apache.manifoldcf.crawler.connectors.sharedrive.
> SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
> ~[?:?]
>
> at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
>
>
>
>
> If I use a smaller windows share it works.
>
>
>
> Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it
> worked.
>
>
>
> What could I do?
>
>
>
> Thanks a lot
>
> Mario
>