You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@manifoldcf.apache.org by Priya Arora <pr...@smartshore.nl> on 2019/08/14 05:46:17 UTC
Unexpected HTTP result code: -1: null
Hi Karl,
I have an web Repository connector(Seeds:- an intranet Site)., and job i
son Production server.
When i ran job on PROD, the job stops itself 2 times with and error:Error:
Unexpected HTTP result code: -1: null.
Can you please provide me an idea, in which it happens so?
Thanks and regards
Priya Arora
Re: Unexpected HTTP result code: -1: null
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Priya,
Set Xms and Xmx values as same. Try with a fairly bigger value, i.e. 4096m
or more and share the result. Then we can adjust that value.
What is your main purpose to use ManifoldCF, just web crawl? If so, you can
check Apache Nutch project too.
Kind Regards,
Furkan KAMACI
16 Ağu 2019 Cum, saat 13:55 tarihinde Priya Arora <pr...@smartshore.nl>
şunu yazdı:
> Hi Furkan,
>
> These are the configuration defined I have -Xms1024m ,-Xmx1024m memory
> allocated *in start-options.env.unix, start-options.env.win* file.
> Also Configuration:-
> 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660
> v3 @ 2.60GHz and -ManifolCF
>
> 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660
> v3 @ 2.60GHz and i am using postgres as database.
>
> When this parameter -Xms1024m was set as -Xms512m., even then i was facing
> this error.
>
> Thanks and Regards
> Priya
>
> On Fri, Aug 16, 2019 at 4:21 PM Furkan KAMACI <fu...@gmail.com>
> wrote:
>
> > Hi Priya,
> >
> > What is your GC params and does it throw error at a particular Zip file?
> > Check its size and if much, consider limiting allowed file size to
> ingest.
> >
> > Kind Regards,
> > Furkan KAMACI
> >
> > 16 Ağu 2019 Cum, saat 13:40 tarihinde Karl Wright <da...@gmail.com>
> > şunu
> > yazdı:
> >
> > > I see nothing indicating any single Tika extraction content type. It's
> > > basically just unhappy with heap fragmentation and is GC'ing too
> > > frequently. I would suggest just increasing the amount of memory you
> > give
> > > the process for an experiment. This might allow it to succeed.
> > >
> > > MCF uses the principle of "bounded memory", which means that every
> > > connector cannot put whole documents into memory, but must be limited.
> > But
> > > there is no stipulation as to any specific limit that each connector
> must
> > > work within. Some connectors, therefore, use a lot more memory than
> > > others, and Tika is one of the ones that can use a lot. But it is
> still
> > > bounded unless there's a bug, so just try increasing for a start.
> > >
> > > Karl
> > >
> > >
> > > On Fri, Aug 16, 2019 at 6:25 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> > >
> > > > Please Find below error stack trace:-
> > > >
> > > > ERROR: agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > at java.util.HashMap.newNode(HashMap.java:1750)
> > > > at java.util.HashMap.putVal(HashMap.java:631)
> > > > at java.util.HashMap.put(HashMap.java:612)
> > > > at
> > > > org.apache.manifoldcf.connectorcommon.fuzzyml.HTMLParseState.noteTag(
> > > >
> > > > HTMLParseState.java:51)
> > > > at
> > > > org.apache.manifoldcf.connectorcommon.fuzzyml.TagParseState.dealWithC
> > > >
> > > > haracter(TagParseState.java:638)
> > > > at
> > > > org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
> > > >
> > > > .dealWithCharacters(SingleCharacterReceiver.java:51)
> > > > at
> > > >
> org.apache.manifoldcf.connectorcommon.fuzzyml.DecodingByteReceiver.de
> > > >
> > > > alWithBytes(DecodingByteReceiver.java:48)
> > > > at
> > > > org.apache.manifoldcf.connectorcommon.fuzzyml.Parser.parseWithoutChar
> > > >
> > > > setDetection(Parser.java:99)
> > > > at
> > > > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> > > >
> > > > or.handleHTML(WebcrawlerConnector.java:4918)
> > > > at
> > > > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> > > >
> > > > or.extractLinks(WebcrawlerConnector.java:3852)
> > > > at
> > > > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> > > >
> > > > or.processDocuments(WebcrawlerConnector.java:747)
> > > > at
> > > > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
> > > >
> > > > va:399)
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > at java.nio.ByteBuffer.wrap(ByteBuffer.java:373)
> > > > at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
> > > > at
> > > > org.apache.commons.compress.archivers.zip.ZipFile.resolveLocalFileHea
> > > >
> > > > derData(ZipFile.java:1059)
> > > > at
> > > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > > >
> > > > :296)
> > > > at
> > > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > > >
> > > > :218)
> > > > at
> > > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > > >
> > > > :201)
> > > > at
> > > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > > >
> > > > :162)
> > > > at
> > > > org.apache.tika.parser.pkg.ZipContainerDetector.detectOPCBased(ZipCon
> > > >
> > > > tainerDetector.java:241)
> > > > at
> > > > org.apache.tika.parser.pkg.ZipContainerDetector.detectZipFormat(ZipCo
> > > >
> > > > ntainerDetector.java:173)
> > > > at
> > > > org.apache.tika.parser.pkg.ZipContainerDetector.detect(ZipContainerDe
> > > >
> > > > tector.java:110)
> > > > at
> > > > org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.jav
> > > >
> > > > a:84)
> > > > at
> > > > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> > > >
> > > > 16)
> > > > at
> > > > org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
> > > >
> > > > 2)
> > > > at
> > > > org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
> > > >
> > > > ded(ParsingEmbeddedDocumentExtractor.java:102)
> > > > at
> > > > org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.jav
> > > >
> > > > a:350)
> > > > at
> > > > org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:287
> > > >
> > > > )
> > > > at
> > > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > > >
> > > > )
> > > > at
> > > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > > >
> > > > )
> > > > at
> > > > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> > > >
> > > > 43)
> > > > at
> > > > org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
> > > >
> > > > 2)
> > > > at
> > > > org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
> > > >
> > > > ded(ParsingEmbeddedDocumentExtractor.java:102)
> > > > at
> > > > org.apache.tika.parser.pkg.CompressorParser.parse(CompressorParser.ja
> > > >
> > > > va:280)
> > > > at
> > > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > > >
> > > > )
> > > > at
> > > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > > >
> > > > )
> > > > at
> > > > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> > > >
> > > > 43)
> > > > at
> > > > org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(Tik
> > > >
> > > > aParser.java:74)
> > > > at
> > > > org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrR
> > > >
> > > > eplaceDocumentWithException(TikaExtractor.java:235)
> > > > at
> > > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> > > >
> > > >
> > > >
> > > >
> > >
> >
> pelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3
> > > >
> > > > 226)
> > > > at
> > > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> > > >
> > > > pelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> > > > at
> > > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> > > >
> > > >
> > > >
> > > >
> > >
> >
> pelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.j
> > > >
> > > > ava:2708)
> > > > at
> > > >
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
> > > >
> > > > cumentIngest(IncrementalIngester.java:756)
> > > > at
> > > > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
> > > >
> > > > estDocumentWithException(WorkerThread.java:1583)
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > [Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
> > > > ServerConne
> > > > ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> > > Stopped
> > > > o.e.
> > > >
> > > > j.w.WebAppContext@6a57ae10
> > > > {/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
> > > >
> > > >
> > > >
> > > >
> > >
> >
> -service.war-_mcf-api-service-any-3323783172971878700.dir/webapp/,UNAVAILABLE}{/
> > > >
> > > >
> > usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > > [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> > > Stopped
> > > > o.e.
> > > >
> > > > j.w.WebAppContext@51c693d
> > > > {/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
> > > >
> > > >
> > > >
> > > >
> > >
> >
> f-authority-service.war-_mcf-authority-service-any-3706951886687463454.dir/webap
> > > >
> > > >
> > > >
> > > >
> > >
> >
> p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
> > > >
> > > > .war}
> > > >
> > > > On Fri, Aug 16, 2019 at 3:22 PM Karl Wright <da...@gmail.com>
> > wrote:
> > > >
> > > > > Without an out-of-memory stack trace, I cannot definitively point
> to
> > > Tika
> > > > > or say that it's a specific kind of file. Please send one.
> > > > >
> > > > > Karl
> > > > >
> > > > >
> > > > > On Fri, Aug 16, 2019 at 2:09 AM Priya Arora <pr...@smartshore.nl>
> > > wrote:
> > > > >
> > > > > > *Existing Threads/connections configuration is :-*
> > > > > >
> > > > > > How many worker threads do you have? - 15 worker threads has been
> > > > > > allocated(in properties.xml file).
> > > > > > And the Tika Extractor connections -10 connections are defined.
> > > > > >
> > > > > > Is this suggested to reduce the number more.
> > > > > > If not, what else can be a solution
> > > > > >
> > > > > > Thanks
> > > > > > Priya
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > How many worker threads do you have?
> > > > > > > Even if each worker thread is constrained in memory, and they
> > > should
> > > > > be,
> > > > > > > you can easily cause things to run out of memory by giving too
> > many
> > > > > > worker
> > > > > > > threads. Another way to keep Tika's usage constrained would be
> > to
> > > > > reduce
> > > > > > > the number of Tika Extractor connections, because that
> > effectively
> > > > > limits
> > > > > > > the number of extractions that can be going on at the same
> time.
> > > > > > >
> > > > > > > Karl
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <
> priya@smartshore.nl
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Yes , I am using Tika Extractor. And the version used for
> > > manifold
> > > > is
> > > > > > > 2.13.
> > > > > > > > Also I am using postgres as database.
> > > > > > > >
> > > > > > > > I have 4 types of jobs
> > > > > > > > One is accessing/re crawling data from a public site. Other
> > three
> > > > are
> > > > > > > > accessing intranet site.
> > > > > > > > Out of which two are giving me correct output-without any
> error
> > > and
> > > > > > third
> > > > > > > > one which is having data more than the other two , and
> giving
> > me
> > > > > this
> > > > > > > > error.
> > > > > > > >
> > > > > > > > Is there any possibility with site accessibility issue. Can
> you
> > > > > please
> > > > > > > > suggest some solution
> > > > > > > > Thanks and regards
> > > > > > > > Priya
> > > > > > > >
> > > > > > > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <
> > daddywri@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > I will need to know more. Do you have the tika extractor
> in
> > > your
> > > > > > > > > pipeline? If so, what version of ManifoldCF are you using?
> > > Tika
> > > > > has
> > > > > > > had
> > > > > > > > > bugs related to memory consumption in the past; the out of
> > > memory
> > > > > > > > exception
> > > > > > > > > may be coming from it and therefore a stack trace is
> critical
> > > to
> > > > > > have.
> > > > > > > > >
> > > > > > > > > Alternatively, you can upgrade to the latest version of MCF
> > > > (2.13)
> > > > > > and
> > > > > > > > that
> > > > > > > > > has a newer version of Tika without those problem. But you
> > may
> > > > > need
> > > > > > to
> > > > > > > > get
> > > > > > > > > the agents process more memory.
> > > > > > > > >
> > > > > > > > > Another possible cause is that you're using hsqldb in
> > > production.
> > > > > > > HSQLDB
> > > > > > > > > keeps all of its tables in memory. If you have a large
> > crawl,
> > > > you
> > > > > do
> > > > > > > not
> > > > > > > > > want to use HSQLDB.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Karl
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <
> > > priya@smartshore.nl
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Karl,
> > > > > > > > > >
> > > > > > > > > > Manifold CF logs hints out me an error like :
> > > > > > > > > > agents process ran out of memory - shutting down
> > > > > > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > > > > > >
> > > > > > > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > > > > > > start-options.env.unix, start-options.env.win file.
> > > > > > > > > > Also Configuration:-
> > > > > > > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R)
> > Xeon(R)
> > > > CPU
> > > > > > > > E5-2660
> > > > > > > > > > v3 @ 2.60GHz and
> > > > > > > > > >
> > > > > > > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R)
> > > Xeon(R)
> > > > > CPU
> > > > > > > > > E5-2660
> > > > > > > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > > > > > > >
> > > > > > > > > > Can you please help me out, what to do in this case.
> > > > > > > > > >
> > > > > > > > > > Thanks
> > > > > > > > > > Priya
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <
> > > > daddywri@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > The error occurs, I believe, as the result of basic
> > > > connection
> > > > > > > > > problems,
> > > > > > > > > > > e.g. the connection is getting rejected. You can find
> > more
> > > > > > > > information
> > > > > > > > > > in
> > > > > > > > > > > the simple history, and in the manifoldcf log.
> > > > > > > > > > >
> > > > > > > > > > > I would like to know the underlying cause, since the
> > > > connector
> > > > > > > should
> > > > > > > > > be
> > > > > > > > > > > resilient against errors of this kind.
> > > > > > > > > > >
> > > > > > > > > > > Karl
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <
> > > > priya@smartshore.nl
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Karl,
> > > > > > > > > > > >
> > > > > > > > > > > > I have an web Repository connector(Seeds:- an
> intranet
> > > > > Site).,
> > > > > > > and
> > > > > > > > > job
> > > > > > > > > > i
> > > > > > > > > > > > son Production server.
> > > > > > > > > > > >
> > > > > > > > > > > > When i ran job on PROD, the job stops itself 2 times
> > with
> > > > and
> > > > > > > > > > > error:Error:
> > > > > > > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Can you please provide me an idea, in which it
> happens
> > > so?
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks and regards
> > > > > > > > > > > > Priya Arora
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
Hi Furkan,
These are the configuration defined I have -Xms1024m ,-Xmx1024m memory
allocated *in start-options.env.unix, start-options.env.win* file.
Also Configuration:-
1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660
v3 @ 2.60GHz and -ManifolCF
2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660
v3 @ 2.60GHz and i am using postgres as database.
When this parameter -Xms1024m was set as -Xms512m., even then i was facing
this error.
Thanks and Regards
Priya
On Fri, Aug 16, 2019 at 4:21 PM Furkan KAMACI <fu...@gmail.com>
wrote:
> Hi Priya,
>
> What is your GC params and does it throw error at a particular Zip file?
> Check its size and if much, consider limiting allowed file size to ingest.
>
> Kind Regards,
> Furkan KAMACI
>
> 16 Ağu 2019 Cum, saat 13:40 tarihinde Karl Wright <da...@gmail.com>
> şunu
> yazdı:
>
> > I see nothing indicating any single Tika extraction content type. It's
> > basically just unhappy with heap fragmentation and is GC'ing too
> > frequently. I would suggest just increasing the amount of memory you
> give
> > the process for an experiment. This might allow it to succeed.
> >
> > MCF uses the principle of "bounded memory", which means that every
> > connector cannot put whole documents into memory, but must be limited.
> But
> > there is no stipulation as to any specific limit that each connector must
> > work within. Some connectors, therefore, use a lot more memory than
> > others, and Tika is one of the ones that can use a lot. But it is still
> > bounded unless there's a bug, so just try increasing for a start.
> >
> > Karl
> >
> >
> > On Fri, Aug 16, 2019 at 6:25 AM Priya Arora <pr...@smartshore.nl> wrote:
> >
> > > Please Find below error stack trace:-
> > >
> > > ERROR: agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > at java.util.HashMap.newNode(HashMap.java:1750)
> > > at java.util.HashMap.putVal(HashMap.java:631)
> > > at java.util.HashMap.put(HashMap.java:612)
> > > at
> > > org.apache.manifoldcf.connectorcommon.fuzzyml.HTMLParseState.noteTag(
> > >
> > > HTMLParseState.java:51)
> > > at
> > > org.apache.manifoldcf.connectorcommon.fuzzyml.TagParseState.dealWithC
> > >
> > > haracter(TagParseState.java:638)
> > > at
> > > org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
> > >
> > > .dealWithCharacters(SingleCharacterReceiver.java:51)
> > > at
> > > org.apache.manifoldcf.connectorcommon.fuzzyml.DecodingByteReceiver.de
> > >
> > > alWithBytes(DecodingByteReceiver.java:48)
> > > at
> > > org.apache.manifoldcf.connectorcommon.fuzzyml.Parser.parseWithoutChar
> > >
> > > setDetection(Parser.java:99)
> > > at
> > > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> > >
> > > or.handleHTML(WebcrawlerConnector.java:4918)
> > > at
> > > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> > >
> > > or.extractLinks(WebcrawlerConnector.java:3852)
> > > at
> > > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> > >
> > > or.processDocuments(WebcrawlerConnector.java:747)
> > > at
> > > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
> > >
> > > va:399)
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > at java.nio.ByteBuffer.wrap(ByteBuffer.java:373)
> > > at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
> > > at
> > > org.apache.commons.compress.archivers.zip.ZipFile.resolveLocalFileHea
> > >
> > > derData(ZipFile.java:1059)
> > > at
> > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > >
> > > :296)
> > > at
> > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > >
> > > :218)
> > > at
> > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > >
> > > :201)
> > > at
> > > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> > >
> > > :162)
> > > at
> > > org.apache.tika.parser.pkg.ZipContainerDetector.detectOPCBased(ZipCon
> > >
> > > tainerDetector.java:241)
> > > at
> > > org.apache.tika.parser.pkg.ZipContainerDetector.detectZipFormat(ZipCo
> > >
> > > ntainerDetector.java:173)
> > > at
> > > org.apache.tika.parser.pkg.ZipContainerDetector.detect(ZipContainerDe
> > >
> > > tector.java:110)
> > > at
> > > org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.jav
> > >
> > > a:84)
> > > at
> > > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> > >
> > > 16)
> > > at
> > > org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
> > >
> > > 2)
> > > at
> > > org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
> > >
> > > ded(ParsingEmbeddedDocumentExtractor.java:102)
> > > at
> > > org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.jav
> > >
> > > a:350)
> > > at
> > > org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:287
> > >
> > > )
> > > at
> > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > >
> > > )
> > > at
> > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > >
> > > )
> > > at
> > > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> > >
> > > 43)
> > > at
> > > org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
> > >
> > > 2)
> > > at
> > > org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
> > >
> > > ded(ParsingEmbeddedDocumentExtractor.java:102)
> > > at
> > > org.apache.tika.parser.pkg.CompressorParser.parse(CompressorParser.ja
> > >
> > > va:280)
> > > at
> > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > >
> > > )
> > > at
> > > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> > >
> > > )
> > > at
> > > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> > >
> > > 43)
> > > at
> > > org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(Tik
> > >
> > > aParser.java:74)
> > > at
> > > org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrR
> > >
> > > eplaceDocumentWithException(TikaExtractor.java:235)
> > > at
> > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> > >
> > >
> > >
> > >
> >
> pelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3
> > >
> > > 226)
> > > at
> > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> > >
> > > pelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> > > at
> > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> > >
> > >
> > >
> > >
> >
> pelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.j
> > >
> > > ava:2708)
> > > at
> > > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
> > >
> > > cumentIngest(IncrementalIngester.java:756)
> > > at
> > > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
> > >
> > > estDocumentWithException(WorkerThread.java:1583)
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > [Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
> > > ServerConne
> > > ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> > Stopped
> > > o.e.
> > >
> > > j.w.WebAppContext@6a57ae10
> > > {/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
> > >
> > >
> > >
> > >
> >
> -service.war-_mcf-api-service-any-3323783172971878700.dir/webapp/,UNAVAILABLE}{/
> > >
> > >
> usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > > [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> > Stopped
> > > o.e.
> > >
> > > j.w.WebAppContext@51c693d
> > > {/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
> > >
> > >
> > >
> > >
> >
> f-authority-service.war-_mcf-authority-service-any-3706951886687463454.dir/webap
> > >
> > >
> > >
> > >
> >
> p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
> > >
> > > .war}
> > >
> > > On Fri, Aug 16, 2019 at 3:22 PM Karl Wright <da...@gmail.com>
> wrote:
> > >
> > > > Without an out-of-memory stack trace, I cannot definitively point to
> > Tika
> > > > or say that it's a specific kind of file. Please send one.
> > > >
> > > > Karl
> > > >
> > > >
> > > > On Fri, Aug 16, 2019 at 2:09 AM Priya Arora <pr...@smartshore.nl>
> > wrote:
> > > >
> > > > > *Existing Threads/connections configuration is :-*
> > > > >
> > > > > How many worker threads do you have? - 15 worker threads has been
> > > > > allocated(in properties.xml file).
> > > > > And the Tika Extractor connections -10 connections are defined.
> > > > >
> > > > > Is this suggested to reduce the number more.
> > > > > If not, what else can be a solution
> > > > >
> > > > > Thanks
> > > > > Priya
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com>
> > > wrote:
> > > > >
> > > > > > How many worker threads do you have?
> > > > > > Even if each worker thread is constrained in memory, and they
> > should
> > > > be,
> > > > > > you can easily cause things to run out of memory by giving too
> many
> > > > > worker
> > > > > > threads. Another way to keep Tika's usage constrained would be
> to
> > > > reduce
> > > > > > the number of Tika Extractor connections, because that
> effectively
> > > > limits
> > > > > > the number of extractions that can be going on at the same time.
> > > > > >
> > > > > > Karl
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <priya@smartshore.nl
> >
> > > > wrote:
> > > > > >
> > > > > > > Yes , I am using Tika Extractor. And the version used for
> > manifold
> > > is
> > > > > > 2.13.
> > > > > > > Also I am using postgres as database.
> > > > > > >
> > > > > > > I have 4 types of jobs
> > > > > > > One is accessing/re crawling data from a public site. Other
> three
> > > are
> > > > > > > accessing intranet site.
> > > > > > > Out of which two are giving me correct output-without any error
> > and
> > > > > third
> > > > > > > one which is having data more than the other two , and giving
> me
> > > > this
> > > > > > > error.
> > > > > > >
> > > > > > > Is there any possibility with site accessibility issue. Can you
> > > > please
> > > > > > > suggest some solution
> > > > > > > Thanks and regards
> > > > > > > Priya
> > > > > > >
> > > > > > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <
> daddywri@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > I will need to know more. Do you have the tika extractor in
> > your
> > > > > > > > pipeline? If so, what version of ManifoldCF are you using?
> > Tika
> > > > has
> > > > > > had
> > > > > > > > bugs related to memory consumption in the past; the out of
> > memory
> > > > > > > exception
> > > > > > > > may be coming from it and therefore a stack trace is critical
> > to
> > > > > have.
> > > > > > > >
> > > > > > > > Alternatively, you can upgrade to the latest version of MCF
> > > (2.13)
> > > > > and
> > > > > > > that
> > > > > > > > has a newer version of Tika without those problem. But you
> may
> > > > need
> > > > > to
> > > > > > > get
> > > > > > > > the agents process more memory.
> > > > > > > >
> > > > > > > > Another possible cause is that you're using hsqldb in
> > production.
> > > > > > HSQLDB
> > > > > > > > keeps all of its tables in memory. If you have a large
> crawl,
> > > you
> > > > do
> > > > > > not
> > > > > > > > want to use HSQLDB.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Karl
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <
> > priya@smartshore.nl
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Karl,
> > > > > > > > >
> > > > > > > > > Manifold CF logs hints out me an error like :
> > > > > > > > > agents process ran out of memory - shutting down
> > > > > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > > > > >
> > > > > > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > > > > > start-options.env.unix, start-options.env.win file.
> > > > > > > > > Also Configuration:-
> > > > > > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R)
> Xeon(R)
> > > CPU
> > > > > > > E5-2660
> > > > > > > > > v3 @ 2.60GHz and
> > > > > > > > >
> > > > > > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R)
> > Xeon(R)
> > > > CPU
> > > > > > > > E5-2660
> > > > > > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > > > > > >
> > > > > > > > > Can you please help me out, what to do in this case.
> > > > > > > > >
> > > > > > > > > Thanks
> > > > > > > > > Priya
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <
> > > daddywri@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > The error occurs, I believe, as the result of basic
> > > connection
> > > > > > > > problems,
> > > > > > > > > > e.g. the connection is getting rejected. You can find
> more
> > > > > > > information
> > > > > > > > > in
> > > > > > > > > > the simple history, and in the manifoldcf log.
> > > > > > > > > >
> > > > > > > > > > I would like to know the underlying cause, since the
> > > connector
> > > > > > should
> > > > > > > > be
> > > > > > > > > > resilient against errors of this kind.
> > > > > > > > > >
> > > > > > > > > > Karl
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <
> > > priya@smartshore.nl
> > > > >
> > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Karl,
> > > > > > > > > > >
> > > > > > > > > > > I have an web Repository connector(Seeds:- an intranet
> > > > Site).,
> > > > > > and
> > > > > > > > job
> > > > > > > > > i
> > > > > > > > > > > son Production server.
> > > > > > > > > > >
> > > > > > > > > > > When i ran job on PROD, the job stops itself 2 times
> with
> > > and
> > > > > > > > > > error:Error:
> > > > > > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Can you please provide me an idea, in which it happens
> > so?
> > > > > > > > > > >
> > > > > > > > > > > Thanks and regards
> > > > > > > > > > > Priya Arora
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Priya,
What is your GC params and does it throw error at a particular Zip file?
Check its size and if much, consider limiting allowed file size to ingest.
Kind Regards,
Furkan KAMACI
16 Ağu 2019 Cum, saat 13:40 tarihinde Karl Wright <da...@gmail.com> şunu
yazdı:
> I see nothing indicating any single Tika extraction content type. It's
> basically just unhappy with heap fragmentation and is GC'ing too
> frequently. I would suggest just increasing the amount of memory you give
> the process for an experiment. This might allow it to succeed.
>
> MCF uses the principle of "bounded memory", which means that every
> connector cannot put whole documents into memory, but must be limited. But
> there is no stipulation as to any specific limit that each connector must
> work within. Some connectors, therefore, use a lot more memory than
> others, and Tika is one of the ones that can use a lot. But it is still
> bounded unless there's a bug, so just try increasing for a start.
>
> Karl
>
>
> On Fri, Aug 16, 2019 at 6:25 AM Priya Arora <pr...@smartshore.nl> wrote:
>
> > Please Find below error stack trace:-
> >
> > ERROR: agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > at java.util.HashMap.newNode(HashMap.java:1750)
> > at java.util.HashMap.putVal(HashMap.java:631)
> > at java.util.HashMap.put(HashMap.java:612)
> > at
> > org.apache.manifoldcf.connectorcommon.fuzzyml.HTMLParseState.noteTag(
> >
> > HTMLParseState.java:51)
> > at
> > org.apache.manifoldcf.connectorcommon.fuzzyml.TagParseState.dealWithC
> >
> > haracter(TagParseState.java:638)
> > at
> > org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
> >
> > .dealWithCharacters(SingleCharacterReceiver.java:51)
> > at
> > org.apache.manifoldcf.connectorcommon.fuzzyml.DecodingByteReceiver.de
> >
> > alWithBytes(DecodingByteReceiver.java:48)
> > at
> > org.apache.manifoldcf.connectorcommon.fuzzyml.Parser.parseWithoutChar
> >
> > setDetection(Parser.java:99)
> > at
> > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> >
> > or.handleHTML(WebcrawlerConnector.java:4918)
> > at
> > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> >
> > or.extractLinks(WebcrawlerConnector.java:3852)
> > at
> > org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
> >
> > or.processDocuments(WebcrawlerConnector.java:747)
> > at
> > org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
> >
> > va:399)
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > at java.nio.ByteBuffer.wrap(ByteBuffer.java:373)
> > at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
> > at
> > org.apache.commons.compress.archivers.zip.ZipFile.resolveLocalFileHea
> >
> > derData(ZipFile.java:1059)
> > at
> > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> >
> > :296)
> > at
> > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> >
> > :218)
> > at
> > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> >
> > :201)
> > at
> > org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
> >
> > :162)
> > at
> > org.apache.tika.parser.pkg.ZipContainerDetector.detectOPCBased(ZipCon
> >
> > tainerDetector.java:241)
> > at
> > org.apache.tika.parser.pkg.ZipContainerDetector.detectZipFormat(ZipCo
> >
> > ntainerDetector.java:173)
> > at
> > org.apache.tika.parser.pkg.ZipContainerDetector.detect(ZipContainerDe
> >
> > tector.java:110)
> > at
> > org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.jav
> >
> > a:84)
> > at
> > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> >
> > 16)
> > at
> > org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
> >
> > 2)
> > at
> > org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
> >
> > ded(ParsingEmbeddedDocumentExtractor.java:102)
> > at
> > org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.jav
> >
> > a:350)
> > at
> > org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:287
> >
> > )
> > at
> > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> >
> > )
> > at
> > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> >
> > )
> > at
> > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> >
> > 43)
> > at
> > org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
> >
> > 2)
> > at
> > org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
> >
> > ded(ParsingEmbeddedDocumentExtractor.java:102)
> > at
> > org.apache.tika.parser.pkg.CompressorParser.parse(CompressorParser.ja
> >
> > va:280)
> > at
> > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> >
> > )
> > at
> > org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
> >
> > )
> > at
> > org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
> >
> > 43)
> > at
> > org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(Tik
> >
> > aParser.java:74)
> > at
> > org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrR
> >
> > eplaceDocumentWithException(TikaExtractor.java:235)
> > at
> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> >
> >
> >
> >
> pelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3
> >
> > 226)
> > at
> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> >
> > pelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> > at
> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
> >
> >
> >
> >
> pelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.j
> >
> > ava:2708)
> > at
> > org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
> >
> > cumentIngest(IncrementalIngester.java:756)
> > at
> > org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
> >
> > estDocumentWithException(WorkerThread.java:1583)
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > [Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
> > ServerConne
> > ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> Stopped
> > o.e.
> >
> > j.w.WebAppContext@6a57ae10
> > {/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
> >
> >
> >
> >
> -service.war-_mcf-api-service-any-3323783172971878700.dir/webapp/,UNAVAILABLE}{/
> >
> > usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: GC overhead limit exceeded
> > [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> Stopped
> > o.e.
> >
> > j.w.WebAppContext@51c693d
> > {/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
> >
> >
> >
> >
> f-authority-service.war-_mcf-authority-service-any-3706951886687463454.dir/webap
> >
> >
> >
> >
> p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
> >
> > .war}
> >
> > On Fri, Aug 16, 2019 at 3:22 PM Karl Wright <da...@gmail.com> wrote:
> >
> > > Without an out-of-memory stack trace, I cannot definitively point to
> Tika
> > > or say that it's a specific kind of file. Please send one.
> > >
> > > Karl
> > >
> > >
> > > On Fri, Aug 16, 2019 at 2:09 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> > >
> > > > *Existing Threads/connections configuration is :-*
> > > >
> > > > How many worker threads do you have? - 15 worker threads has been
> > > > allocated(in properties.xml file).
> > > > And the Tika Extractor connections -10 connections are defined.
> > > >
> > > > Is this suggested to reduce the number more.
> > > > If not, what else can be a solution
> > > >
> > > > Thanks
> > > > Priya
> > > >
> > > >
> > > >
> > > > On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com>
> > wrote:
> > > >
> > > > > How many worker threads do you have?
> > > > > Even if each worker thread is constrained in memory, and they
> should
> > > be,
> > > > > you can easily cause things to run out of memory by giving too many
> > > > worker
> > > > > threads. Another way to keep Tika's usage constrained would be to
> > > reduce
> > > > > the number of Tika Extractor connections, because that effectively
> > > limits
> > > > > the number of extractions that can be going on at the same time.
> > > > >
> > > > > Karl
> > > > >
> > > > >
> > > > > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl>
> > > wrote:
> > > > >
> > > > > > Yes , I am using Tika Extractor. And the version used for
> manifold
> > is
> > > > > 2.13.
> > > > > > Also I am using postgres as database.
> > > > > >
> > > > > > I have 4 types of jobs
> > > > > > One is accessing/re crawling data from a public site. Other three
> > are
> > > > > > accessing intranet site.
> > > > > > Out of which two are giving me correct output-without any error
> and
> > > > third
> > > > > > one which is having data more than the other two , and giving me
> > > this
> > > > > > error.
> > > > > >
> > > > > > Is there any possibility with site accessibility issue. Can you
> > > please
> > > > > > suggest some solution
> > > > > > Thanks and regards
> > > > > > Priya
> > > > > >
> > > > > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > I will need to know more. Do you have the tika extractor in
> your
> > > > > > > pipeline? If so, what version of ManifoldCF are you using?
> Tika
> > > has
> > > > > had
> > > > > > > bugs related to memory consumption in the past; the out of
> memory
> > > > > > exception
> > > > > > > may be coming from it and therefore a stack trace is critical
> to
> > > > have.
> > > > > > >
> > > > > > > Alternatively, you can upgrade to the latest version of MCF
> > (2.13)
> > > > and
> > > > > > that
> > > > > > > has a newer version of Tika without those problem. But you may
> > > need
> > > > to
> > > > > > get
> > > > > > > the agents process more memory.
> > > > > > >
> > > > > > > Another possible cause is that you're using hsqldb in
> production.
> > > > > HSQLDB
> > > > > > > keeps all of its tables in memory. If you have a large crawl,
> > you
> > > do
> > > > > not
> > > > > > > want to use HSQLDB.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Karl
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <
> priya@smartshore.nl
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi Karl,
> > > > > > > >
> > > > > > > > Manifold CF logs hints out me an error like :
> > > > > > > > agents process ran out of memory - shutting down
> > > > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > > > >
> > > > > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > > > > start-options.env.unix, start-options.env.win file.
> > > > > > > > Also Configuration:-
> > > > > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R)
> > CPU
> > > > > > E5-2660
> > > > > > > > v3 @ 2.60GHz and
> > > > > > > >
> > > > > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R)
> Xeon(R)
> > > CPU
> > > > > > > E5-2660
> > > > > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > > > > >
> > > > > > > > Can you please help me out, what to do in this case.
> > > > > > > >
> > > > > > > > Thanks
> > > > > > > > Priya
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <
> > daddywri@gmail.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > The error occurs, I believe, as the result of basic
> > connection
> > > > > > > problems,
> > > > > > > > > e.g. the connection is getting rejected. You can find more
> > > > > > information
> > > > > > > > in
> > > > > > > > > the simple history, and in the manifoldcf log.
> > > > > > > > >
> > > > > > > > > I would like to know the underlying cause, since the
> > connector
> > > > > should
> > > > > > > be
> > > > > > > > > resilient against errors of this kind.
> > > > > > > > >
> > > > > > > > > Karl
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <
> > priya@smartshore.nl
> > > >
> > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Karl,
> > > > > > > > > >
> > > > > > > > > > I have an web Repository connector(Seeds:- an intranet
> > > Site).,
> > > > > and
> > > > > > > job
> > > > > > > > i
> > > > > > > > > > son Production server.
> > > > > > > > > >
> > > > > > > > > > When i ran job on PROD, the job stops itself 2 times with
> > and
> > > > > > > > > error:Error:
> > > > > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Can you please provide me an idea, in which it happens
> so?
> > > > > > > > > >
> > > > > > > > > > Thanks and regards
> > > > > > > > > > Priya Arora
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Karl Wright <da...@gmail.com>.
I see nothing indicating any single Tika extraction content type. It's
basically just unhappy with heap fragmentation and is GC'ing too
frequently. I would suggest just increasing the amount of memory you give
the process for an experiment. This might allow it to succeed.
MCF uses the principle of "bounded memory", which means that every
connector cannot put whole documents into memory, but must be limited. But
there is no stipulation as to any specific limit that each connector must
work within. Some connectors, therefore, use a lot more memory than
others, and Tika is one of the ones that can use a lot. But it is still
bounded unless there's a bug, so just try increasing for a start.
Karl
On Fri, Aug 16, 2019 at 6:25 AM Priya Arora <pr...@smartshore.nl> wrote:
> Please Find below error stack trace:-
>
> ERROR: agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.util.HashMap.newNode(HashMap.java:1750)
> at java.util.HashMap.putVal(HashMap.java:631)
> at java.util.HashMap.put(HashMap.java:612)
> at
> org.apache.manifoldcf.connectorcommon.fuzzyml.HTMLParseState.noteTag(
>
> HTMLParseState.java:51)
> at
> org.apache.manifoldcf.connectorcommon.fuzzyml.TagParseState.dealWithC
>
> haracter(TagParseState.java:638)
> at
> org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
>
> .dealWithCharacters(SingleCharacterReceiver.java:51)
> at
> org.apache.manifoldcf.connectorcommon.fuzzyml.DecodingByteReceiver.de
>
> alWithBytes(DecodingByteReceiver.java:48)
> at
> org.apache.manifoldcf.connectorcommon.fuzzyml.Parser.parseWithoutChar
>
> setDetection(Parser.java:99)
> at
> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
>
> or.handleHTML(WebcrawlerConnector.java:4918)
> at
> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
>
> or.extractLinks(WebcrawlerConnector.java:3852)
> at
> org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
>
> or.processDocuments(WebcrawlerConnector.java:747)
> at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
>
> va:399)
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.nio.ByteBuffer.wrap(ByteBuffer.java:373)
> at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
> at
> org.apache.commons.compress.archivers.zip.ZipFile.resolveLocalFileHea
>
> derData(ZipFile.java:1059)
> at
> org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
>
> :296)
> at
> org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
>
> :218)
> at
> org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
>
> :201)
> at
> org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
>
> :162)
> at
> org.apache.tika.parser.pkg.ZipContainerDetector.detectOPCBased(ZipCon
>
> tainerDetector.java:241)
> at
> org.apache.tika.parser.pkg.ZipContainerDetector.detectZipFormat(ZipCo
>
> ntainerDetector.java:173)
> at
> org.apache.tika.parser.pkg.ZipContainerDetector.detect(ZipContainerDe
>
> tector.java:110)
> at
> org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.jav
>
> a:84)
> at
> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
>
> 16)
> at
> org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
>
> 2)
> at
> org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
>
> ded(ParsingEmbeddedDocumentExtractor.java:102)
> at
> org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.jav
>
> a:350)
> at
> org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:287
>
> )
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
>
> )
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
>
> )
> at
> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
>
> 43)
> at
> org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
>
> 2)
> at
> org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
>
> ded(ParsingEmbeddedDocumentExtractor.java:102)
> at
> org.apache.tika.parser.pkg.CompressorParser.parse(CompressorParser.ja
>
> va:280)
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
>
> )
> at
> org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
>
> )
> at
> org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
>
> 43)
> at
> org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(Tik
>
> aParser.java:74)
> at
> org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrR
>
> eplaceDocumentWithException(TikaExtractor.java:235)
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
>
>
>
> pelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3
>
> 226)
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
>
> pelineAddFanout.sendDocument(IncrementalIngester.java:3077)
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
>
>
>
> pelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.j
>
> ava:2708)
> at
> org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
>
> cumentIngest(IncrementalIngester.java:756)
> at
> org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
>
> estDocumentWithException(WorkerThread.java:1583)
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
> ServerConne
> ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped
> o.e.
>
> j.w.WebAppContext@6a57ae10
> {/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
>
>
>
> -service.war-_mcf-api-service-any-3323783172971878700.dir/webapp/,UNAVAILABLE}{/
>
> usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped
> o.e.
>
> j.w.WebAppContext@51c693d
> {/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
>
>
>
> f-authority-service.war-_mcf-authority-service-any-3706951886687463454.dir/webap
>
>
>
> p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
>
> .war}
>
> On Fri, Aug 16, 2019 at 3:22 PM Karl Wright <da...@gmail.com> wrote:
>
> > Without an out-of-memory stack trace, I cannot definitively point to Tika
> > or say that it's a specific kind of file. Please send one.
> >
> > Karl
> >
> >
> > On Fri, Aug 16, 2019 at 2:09 AM Priya Arora <pr...@smartshore.nl> wrote:
> >
> > > *Existing Threads/connections configuration is :-*
> > >
> > > How many worker threads do you have? - 15 worker threads has been
> > > allocated(in properties.xml file).
> > > And the Tika Extractor connections -10 connections are defined.
> > >
> > > Is this suggested to reduce the number more.
> > > If not, what else can be a solution
> > >
> > > Thanks
> > > Priya
> > >
> > >
> > >
> > > On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com>
> wrote:
> > >
> > > > How many worker threads do you have?
> > > > Even if each worker thread is constrained in memory, and they should
> > be,
> > > > you can easily cause things to run out of memory by giving too many
> > > worker
> > > > threads. Another way to keep Tika's usage constrained would be to
> > reduce
> > > > the number of Tika Extractor connections, because that effectively
> > limits
> > > > the number of extractions that can be going on at the same time.
> > > >
> > > > Karl
> > > >
> > > >
> > > > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl>
> > wrote:
> > > >
> > > > > Yes , I am using Tika Extractor. And the version used for manifold
> is
> > > > 2.13.
> > > > > Also I am using postgres as database.
> > > > >
> > > > > I have 4 types of jobs
> > > > > One is accessing/re crawling data from a public site. Other three
> are
> > > > > accessing intranet site.
> > > > > Out of which two are giving me correct output-without any error and
> > > third
> > > > > one which is having data more than the other two , and giving me
> > this
> > > > > error.
> > > > >
> > > > > Is there any possibility with site accessibility issue. Can you
> > please
> > > > > suggest some solution
> > > > > Thanks and regards
> > > > > Priya
> > > > >
> > > > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
> > > wrote:
> > > > >
> > > > > > I will need to know more. Do you have the tika extractor in your
> > > > > > pipeline? If so, what version of ManifoldCF are you using? Tika
> > has
> > > > had
> > > > > > bugs related to memory consumption in the past; the out of memory
> > > > > exception
> > > > > > may be coming from it and therefore a stack trace is critical to
> > > have.
> > > > > >
> > > > > > Alternatively, you can upgrade to the latest version of MCF
> (2.13)
> > > and
> > > > > that
> > > > > > has a newer version of Tika without those problem. But you may
> > need
> > > to
> > > > > get
> > > > > > the agents process more memory.
> > > > > >
> > > > > > Another possible cause is that you're using hsqldb in production.
> > > > HSQLDB
> > > > > > keeps all of its tables in memory. If you have a large crawl,
> you
> > do
> > > > not
> > > > > > want to use HSQLDB.
> > > > > >
> > > > > > Thanks,
> > > > > > Karl
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <priya@smartshore.nl
> >
> > > > wrote:
> > > > > >
> > > > > > > Hi Karl,
> > > > > > >
> > > > > > > Manifold CF logs hints out me an error like :
> > > > > > > agents process ran out of memory - shutting down
> > > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > > >
> > > > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > > > start-options.env.unix, start-options.env.win file.
> > > > > > > Also Configuration:-
> > > > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R)
> CPU
> > > > > E5-2660
> > > > > > > v3 @ 2.60GHz and
> > > > > > >
> > > > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R)
> > CPU
> > > > > > E5-2660
> > > > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > > > >
> > > > > > > Can you please help me out, what to do in this case.
> > > > > > >
> > > > > > > Thanks
> > > > > > > Priya
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <
> daddywri@gmail.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > The error occurs, I believe, as the result of basic
> connection
> > > > > > problems,
> > > > > > > > e.g. the connection is getting rejected. You can find more
> > > > > information
> > > > > > > in
> > > > > > > > the simple history, and in the manifoldcf log.
> > > > > > > >
> > > > > > > > I would like to know the underlying cause, since the
> connector
> > > > should
> > > > > > be
> > > > > > > > resilient against errors of this kind.
> > > > > > > >
> > > > > > > > Karl
> > > > > > > >
> > > > > > > >
> > > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <
> priya@smartshore.nl
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Karl,
> > > > > > > > >
> > > > > > > > > I have an web Repository connector(Seeds:- an intranet
> > Site).,
> > > > and
> > > > > > job
> > > > > > > i
> > > > > > > > > son Production server.
> > > > > > > > >
> > > > > > > > > When i ran job on PROD, the job stops itself 2 times with
> and
> > > > > > > > error:Error:
> > > > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Can you please provide me an idea, in which it happens so?
> > > > > > > > >
> > > > > > > > > Thanks and regards
> > > > > > > > > Priya Arora
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
Please Find below error stack trace:-
ERROR: agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.newNode(HashMap.java:1750)
at java.util.HashMap.putVal(HashMap.java:631)
at java.util.HashMap.put(HashMap.java:612)
at
org.apache.manifoldcf.connectorcommon.fuzzyml.HTMLParseState.noteTag(
HTMLParseState.java:51)
at
org.apache.manifoldcf.connectorcommon.fuzzyml.TagParseState.dealWithC
haracter(TagParseState.java:638)
at
org.apache.manifoldcf.connectorcommon.fuzzyml.SingleCharacterReceiver
.dealWithCharacters(SingleCharacterReceiver.java:51)
at
org.apache.manifoldcf.connectorcommon.fuzzyml.DecodingByteReceiver.de
alWithBytes(DecodingByteReceiver.java:48)
at
org.apache.manifoldcf.connectorcommon.fuzzyml.Parser.parseWithoutChar
setDetection(Parser.java:99)
at
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
or.handleHTML(WebcrawlerConnector.java:4918)
at
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
or.extractLinks(WebcrawlerConnector.java:3852)
at
org.apache.manifoldcf.crawler.connectors.webcrawler.WebcrawlerConnect
or.processDocuments(WebcrawlerConnector.java:747)
at
org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.ja
va:399)
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.nio.ByteBuffer.wrap(ByteBuffer.java:373)
at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
at
org.apache.commons.compress.archivers.zip.ZipFile.resolveLocalFileHea
derData(ZipFile.java:1059)
at
org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
:296)
at
org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
:218)
at
org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
:201)
at
org.apache.commons.compress.archivers.zip.ZipFile.<init>(ZipFile.java
:162)
at
org.apache.tika.parser.pkg.ZipContainerDetector.detectOPCBased(ZipCon
tainerDetector.java:241)
at
org.apache.tika.parser.pkg.ZipContainerDetector.detectZipFormat(ZipCo
ntainerDetector.java:173)
at
org.apache.tika.parser.pkg.ZipContainerDetector.detect(ZipContainerDe
tector.java:110)
at
org.apache.tika.detect.CompositeDetector.detect(CompositeDetector.jav
a:84)
at
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
16)
at
org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
2)
at
org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
ded(ParsingEmbeddedDocumentExtractor.java:102)
at
org.apache.tika.parser.pkg.PackageParser.parseEntry(PackageParser.jav
a:350)
at
org.apache.tika.parser.pkg.PackageParser.parse(PackageParser.java:287
)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
)
at
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
43)
at
org.apache.tika.parser.DelegatingParser.parse(DelegatingParser.java:7
2)
at
org.apache.tika.extractor.ParsingEmbeddedDocumentExtractor.parseEmbed
ded(ParsingEmbeddedDocumentExtractor.java:102)
at
org.apache.tika.parser.pkg.CompressorParser.parse(CompressorParser.ja
va:280)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
)
at
org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280
)
at
org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:1
43)
at
org.apache.manifoldcf.agents.transformation.tika.TikaParser.parse(Tik
aParser.java:74)
at
org.apache.manifoldcf.agents.transformation.tika.TikaExtractor.addOrR
eplaceDocumentWithException(TikaExtractor.java:235)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
pelineAddEntryPoint.addOrReplaceDocumentWithException(IncrementalIngester.java:3
226)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
pelineAddFanout.sendDocument(IncrementalIngester.java:3077)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$Pi
pelineObjectWithVersions.addOrReplaceDocumentWithException(IncrementalIngester.j
ava:2708)
at
org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.do
cumentIngest(IncrementalIngester.java:756)
at
org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.ing
estDocumentWithException(WorkerThread.java:1583)
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
[Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
ServerConne
ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
[Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped
o.e.
j.w.WebAppContext@6a57ae10{/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
-service.war-_mcf-api-service-any-3323783172971878700.dir/webapp/,UNAVAILABLE}{/
usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
[Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped
o.e.
j.w.WebAppContext@51c693d{/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
f-authority-service.war-_mcf-authority-service-any-3706951886687463454.dir/webap
p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
.war}
On Fri, Aug 16, 2019 at 3:22 PM Karl Wright <da...@gmail.com> wrote:
> Without an out-of-memory stack trace, I cannot definitively point to Tika
> or say that it's a specific kind of file. Please send one.
>
> Karl
>
>
> On Fri, Aug 16, 2019 at 2:09 AM Priya Arora <pr...@smartshore.nl> wrote:
>
> > *Existing Threads/connections configuration is :-*
> >
> > How many worker threads do you have? - 15 worker threads has been
> > allocated(in properties.xml file).
> > And the Tika Extractor connections -10 connections are defined.
> >
> > Is this suggested to reduce the number more.
> > If not, what else can be a solution
> >
> > Thanks
> > Priya
> >
> >
> >
> > On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com> wrote:
> >
> > > How many worker threads do you have?
> > > Even if each worker thread is constrained in memory, and they should
> be,
> > > you can easily cause things to run out of memory by giving too many
> > worker
> > > threads. Another way to keep Tika's usage constrained would be to
> reduce
> > > the number of Tika Extractor connections, because that effectively
> limits
> > > the number of extractions that can be going on at the same time.
> > >
> > > Karl
> > >
> > >
> > > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> > >
> > > > Yes , I am using Tika Extractor. And the version used for manifold is
> > > 2.13.
> > > > Also I am using postgres as database.
> > > >
> > > > I have 4 types of jobs
> > > > One is accessing/re crawling data from a public site. Other three are
> > > > accessing intranet site.
> > > > Out of which two are giving me correct output-without any error and
> > third
> > > > one which is having data more than the other two , and giving me
> this
> > > > error.
> > > >
> > > > Is there any possibility with site accessibility issue. Can you
> please
> > > > suggest some solution
> > > > Thanks and regards
> > > > Priya
> > > >
> > > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
> > wrote:
> > > >
> > > > > I will need to know more. Do you have the tika extractor in your
> > > > > pipeline? If so, what version of ManifoldCF are you using? Tika
> has
> > > had
> > > > > bugs related to memory consumption in the past; the out of memory
> > > > exception
> > > > > may be coming from it and therefore a stack trace is critical to
> > have.
> > > > >
> > > > > Alternatively, you can upgrade to the latest version of MCF (2.13)
> > and
> > > > that
> > > > > has a newer version of Tika without those problem. But you may
> need
> > to
> > > > get
> > > > > the agents process more memory.
> > > > >
> > > > > Another possible cause is that you're using hsqldb in production.
> > > HSQLDB
> > > > > keeps all of its tables in memory. If you have a large crawl, you
> do
> > > not
> > > > > want to use HSQLDB.
> > > > >
> > > > > Thanks,
> > > > > Karl
> > > > >
> > > > >
> > > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
> > > wrote:
> > > > >
> > > > > > Hi Karl,
> > > > > >
> > > > > > Manifold CF logs hints out me an error like :
> > > > > > agents process ran out of memory - shutting down
> > > > > > java.lang.OutOfMemoryError: Java heap space
> > > > > >
> > > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > > start-options.env.unix, start-options.env.win file.
> > > > > > Also Configuration:-
> > > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> > > > E5-2660
> > > > > > v3 @ 2.60GHz and
> > > > > >
> > > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R)
> CPU
> > > > > E5-2660
> > > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > > >
> > > > > > Can you please help me out, what to do in this case.
> > > > > >
> > > > > > Thanks
> > > > > > Priya
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <daddywri@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > > > The error occurs, I believe, as the result of basic connection
> > > > > problems,
> > > > > > > e.g. the connection is getting rejected. You can find more
> > > > information
> > > > > > in
> > > > > > > the simple history, and in the manifoldcf log.
> > > > > > >
> > > > > > > I would like to know the underlying cause, since the connector
> > > should
> > > > > be
> > > > > > > resilient against errors of this kind.
> > > > > > >
> > > > > > > Karl
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <priya@smartshore.nl
> >
> > > > wrote:
> > > > > > >
> > > > > > > > Hi Karl,
> > > > > > > >
> > > > > > > > I have an web Repository connector(Seeds:- an intranet
> Site).,
> > > and
> > > > > job
> > > > > > i
> > > > > > > > son Production server.
> > > > > > > >
> > > > > > > > When i ran job on PROD, the job stops itself 2 times with and
> > > > > > > error:Error:
> > > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > > >
> > > > > > > >
> > > > > > > > Can you please provide me an idea, in which it happens so?
> > > > > > > >
> > > > > > > > Thanks and regards
> > > > > > > > Priya Arora
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Karl Wright <da...@gmail.com>.
Without an out-of-memory stack trace, I cannot definitively point to Tika
or say that it's a specific kind of file. Please send one.
Karl
On Fri, Aug 16, 2019 at 2:09 AM Priya Arora <pr...@smartshore.nl> wrote:
> *Existing Threads/connections configuration is :-*
>
> How many worker threads do you have? - 15 worker threads has been
> allocated(in properties.xml file).
> And the Tika Extractor connections -10 connections are defined.
>
> Is this suggested to reduce the number more.
> If not, what else can be a solution
>
> Thanks
> Priya
>
>
>
> On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com> wrote:
>
> > How many worker threads do you have?
> > Even if each worker thread is constrained in memory, and they should be,
> > you can easily cause things to run out of memory by giving too many
> worker
> > threads. Another way to keep Tika's usage constrained would be to reduce
> > the number of Tika Extractor connections, because that effectively limits
> > the number of extractions that can be going on at the same time.
> >
> > Karl
> >
> >
> > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl> wrote:
> >
> > > Yes , I am using Tika Extractor. And the version used for manifold is
> > 2.13.
> > > Also I am using postgres as database.
> > >
> > > I have 4 types of jobs
> > > One is accessing/re crawling data from a public site. Other three are
> > > accessing intranet site.
> > > Out of which two are giving me correct output-without any error and
> third
> > > one which is having data more than the other two , and giving me this
> > > error.
> > >
> > > Is there any possibility with site accessibility issue. Can you please
> > > suggest some solution
> > > Thanks and regards
> > > Priya
> > >
> > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
> wrote:
> > >
> > > > I will need to know more. Do you have the tika extractor in your
> > > > pipeline? If so, what version of ManifoldCF are you using? Tika has
> > had
> > > > bugs related to memory consumption in the past; the out of memory
> > > exception
> > > > may be coming from it and therefore a stack trace is critical to
> have.
> > > >
> > > > Alternatively, you can upgrade to the latest version of MCF (2.13)
> and
> > > that
> > > > has a newer version of Tika without those problem. But you may need
> to
> > > get
> > > > the agents process more memory.
> > > >
> > > > Another possible cause is that you're using hsqldb in production.
> > HSQLDB
> > > > keeps all of its tables in memory. If you have a large crawl, you do
> > not
> > > > want to use HSQLDB.
> > > >
> > > > Thanks,
> > > > Karl
> > > >
> > > >
> > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
> > wrote:
> > > >
> > > > > Hi Karl,
> > > > >
> > > > > Manifold CF logs hints out me an error like :
> > > > > agents process ran out of memory - shutting down
> > > > > java.lang.OutOfMemoryError: Java heap space
> > > > >
> > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > start-options.env.unix, start-options.env.win file.
> > > > > Also Configuration:-
> > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> > > E5-2660
> > > > > v3 @ 2.60GHz and
> > > > >
> > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
> > > > E5-2660
> > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > >
> > > > > Can you please help me out, what to do in this case.
> > > > >
> > > > > Thanks
> > > > > Priya
> > > > >
> > > > >
> > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
> > > wrote:
> > > > >
> > > > > > The error occurs, I believe, as the result of basic connection
> > > > problems,
> > > > > > e.g. the connection is getting rejected. You can find more
> > > information
> > > > > in
> > > > > > the simple history, and in the manifoldcf log.
> > > > > >
> > > > > > I would like to know the underlying cause, since the connector
> > should
> > > > be
> > > > > > resilient against errors of this kind.
> > > > > >
> > > > > > Karl
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
> > > wrote:
> > > > > >
> > > > > > > Hi Karl,
> > > > > > >
> > > > > > > I have an web Repository connector(Seeds:- an intranet Site).,
> > and
> > > > job
> > > > > i
> > > > > > > son Production server.
> > > > > > >
> > > > > > > When i ran job on PROD, the job stops itself 2 times with and
> > > > > > error:Error:
> > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > >
> > > > > > >
> > > > > > > Can you please provide me an idea, in which it happens so?
> > > > > > >
> > > > > > > Thanks and regards
> > > > > > > Priya Arora
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Karl Wright <da...@gmail.com>.
"I have -Xms1024m ,-Xmx1024m memory allocated in
*start-options.env.unix, start-options.env.win file.*
Also Configuration:-
1) *For Crawler server *- 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
E5-2660v3 @ 2.60GHz and"
What is "crawler server"?
You said "single process example", right? That means there is one MCF
process. That process is running out of memory. You are giving it one
gigabyte. It is not enough. Add more memory to the -Xmx setting in
start-options.
Karl
On Wed, Sep 11, 2019 at 1:43 AM Priya Arora <pr...@smartshore.nl> wrote:
> Hi all,
>
> Manifold CF crashes.
> Raising this issue again, after enhancing memory and otherwise, still
> facing the same issue. TRied every possible solution If anybody can help ,
> please suggest me some solution.
> *Problem:-* Manifoldcf crashes while crawling data from an intranet URL
> hinting out below error in logs:-
>
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at java.util.HashMap.resize(HashMap.java:704)
> at java.util.HashMap.putVal(HashMap.java:629)
> at java.util.HashMap.put(HashMap.java:612)
> at
> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndEx
>
> ecute(CacheManager.java:135)
> at
> org.apache.manifoldcf.agents.transformationconnection.TransformationC
>
>
> onnectionManager.loadMultipleInternal(TransformationConnectionManager.java:265)
> at
> org.apache.manifoldcf.agents.transformationconnection.TransformationC
>
> onnectionManager.loadMultiple(TransformationConnectionManager.java:246)
> agents process could not start - shutting down
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> at
> org.postgresql.jdbc.PgConnection.createStatement(PgConnection.java:11
>
> 93)
> at
> org.postgresql.jdbc.PgConnection.createStatement(PgConnection.java:15
>
> 77)
> at
> org.postgresql.jdbc.PgConnection.createStatement(PgConnection.java:36
>
> 7)
> at
> org.apache.manifoldcf.core.database.Database.execute(Database.java:87
>
> 3)
> at
> org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(D
>
> atabase.java:696)
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
> ServerConne
> ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> Stopped o.e.
> j.w.WebAppContext@6a57ae10{/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
>
>
> -service.war-_mcf-api-service-any-2010273384908625074.dir/webapp/,UNAVAILABLE}{/
>
> usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
> [Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler -
> Stopped o.e.
> j.w.WebAppContext@51c693d{/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
>
>
> f-authority-service.war-_mcf-authority-service-any-1431262612693374489.dir/webap
>
>
> p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
>
> .war}
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: Java heap space
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: Java heap space
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: Java heap space
> Exception in thread "PostgreSQL-JDBC-SharedTimer-1"
> java.lang.OutOfMemoryError:
>
> Java heap space
>
> All configuration are:-
> I have -Xms1024m ,-Xmx1024m memory allocated in
> *start-options.env.unix, start-options.env.win file.*
> Also Configuration:-
> 1) *For Crawler server *- 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> E5-2660v3 @ 2.60GHz and
>
> 2) *For Elasticsearch server* - 48GB and 16-Core Intel(R) Xeon(R) CPU
> E5-2660 v3 @ 2.60GHz and i am using postgres as database.
>
> 3) Database as PostgreSQL
> 4) No of worker threads:-20.
> 5) manifold Version :-2.13
>
> Can anybody please suggest some solution to get rid of this error and
> crash, as whenever this istutaion comes, manifoldCF UI results me this:_
> [image: image.png]
>
> Any suggestion would be highly appreciated.
>
> Thanks
> Priya
>
> On Fri, Aug 16, 2019 at 6:41 PM Priya Arora <pr...@smartshore.nl> wrote:
>
>> Sure I will try to do that.. and will get back to you
>>
>> Sent from my iPhone
>>
>> > On 16-Aug-2019, at 6:23 PM, Michael Cizmar <mi...@michaelcizmar.com>
>> wrote:
>> >
>> > Priya - Was this right?
>> >
>> > 48GB and 1-Core Intel(R) Xeon(R) CPU
>> >
>> > While not directly related to an out of memory issue, you should have
>> more
>> > cores allocated to ES.
>> >
>> >> On Fri, Aug 16, 2019 at 1:09 AM Priya Arora <pr...@smartshore.nl>
>> wrote:
>> >>
>> >> *Existing Threads/connections configuration is :-*
>> >>
>> >> How many worker threads do you have? - 15 worker threads has been
>> >> allocated(in properties.xml file).
>> >> And the Tika Extractor connections -10 connections are defined.
>> >>
>> >> Is this suggested to reduce the number more.
>> >> If not, what else can be a solution
>> >>
>> >> Thanks
>> >> Priya
>> >>
>> >>
>> >>
>> >>> On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com>
>> wrote:
>> >>>
>> >>> How many worker threads do you have?
>> >>> Even if each worker thread is constrained in memory, and they should
>> be,
>> >>> you can easily cause things to run out of memory by giving too many
>> >> worker
>> >>> threads. Another way to keep Tika's usage constrained would be to
>> reduce
>> >>> the number of Tika Extractor connections, because that effectively
>> limits
>> >>> the number of extractions that can be going on at the same time.
>> >>>
>> >>> Karl
>> >>>
>> >>>
>> >>>> On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl>
>> wrote:
>> >>>>
>> >>>> Yes , I am using Tika Extractor. And the version used for manifold is
>> >>> 2.13.
>> >>>> Also I am using postgres as database.
>> >>>>
>> >>>> I have 4 types of jobs
>> >>>> One is accessing/re crawling data from a public site. Other three are
>> >>>> accessing intranet site.
>> >>>> Out of which two are giving me correct output-without any error and
>> >> third
>> >>>> one which is having data more than the other two , and giving me
>> this
>> >>>> error.
>> >>>>
>> >>>> Is there any possibility with site accessibility issue. Can you
>> please
>> >>>> suggest some solution
>> >>>> Thanks and regards
>> >>>> Priya
>> >>>>
>> >>>> On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
>> >> wrote:
>> >>>>
>> >>>>> I will need to know more. Do you have the tika extractor in your
>> >>>>> pipeline? If so, what version of ManifoldCF are you using? Tika
>> has
>> >>> had
>> >>>>> bugs related to memory consumption in the past; the out of memory
>> >>>> exception
>> >>>>> may be coming from it and therefore a stack trace is critical to
>> >> have.
>> >>>>>
>> >>>>> Alternatively, you can upgrade to the latest version of MCF (2.13)
>> >> and
>> >>>> that
>> >>>>> has a newer version of Tika without those problem. But you may need
>> >> to
>> >>>> get
>> >>>>> the agents process more memory.
>> >>>>>
>> >>>>> Another possible cause is that you're using hsqldb in production.
>> >>> HSQLDB
>> >>>>> keeps all of its tables in memory. If you have a large crawl, you
>> do
>> >>> not
>> >>>>> want to use HSQLDB.
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Karl
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
>> >>> wrote:
>> >>>>>
>> >>>>>> Hi Karl,
>> >>>>>>
>> >>>>>> Manifold CF logs hints out me an error like :
>> >>>>>> agents process ran out of memory - shutting down
>> >>>>>> java.lang.OutOfMemoryError: Java heap space
>> >>>>>>
>> >>>>>> Also I have -Xms1024m ,-Xmx1024m memory allocated in
>> >>>>>> start-options.env.unix, start-options.env.win file.
>> >>>>>> Also Configuration:-
>> >>>>>> 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
>> >>>> E5-2660
>> >>>>>> v3 @ 2.60GHz and
>> >>>>>>
>> >>>>>> 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
>> >>>>> E5-2660
>> >>>>>> v3 @ 2.60GHz and i am using postgres as database.
>> >>>>>>
>> >>>>>> Can you please help me out, what to do in this case.
>> >>>>>>
>> >>>>>> Thanks
>> >>>>>> Priya
>> >>>>>>
>> >>>>>>
>> >>>>>> On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> The error occurs, I believe, as the result of basic connection
>> >>>>> problems,
>> >>>>>>> e.g. the connection is getting rejected. You can find more
>> >>>> information
>> >>>>>> in
>> >>>>>>> the simple history, and in the manifoldcf log.
>> >>>>>>>
>> >>>>>>> I would like to know the underlying cause, since the connector
>> >>> should
>> >>>>> be
>> >>>>>>> resilient against errors of this kind.
>> >>>>>>>
>> >>>>>>> Karl
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
>> >>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi Karl,
>> >>>>>>>>
>> >>>>>>>> I have an web Repository connector(Seeds:- an intranet Site).,
>> >>> and
>> >>>>> job
>> >>>>>> i
>> >>>>>>>> son Production server.
>> >>>>>>>>
>> >>>>>>>> When i ran job on PROD, the job stops itself 2 times with and
>> >>>>>>> error:Error:
>> >>>>>>>> Unexpected HTTP result code: -1: null.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Can you please provide me an idea, in which it happens so?
>> >>>>>>>>
>> >>>>>>>> Thanks and regards
>> >>>>>>>> Priya Arora
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>>
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
Hi all,
Manifold CF crashes.
Raising this issue again, after enhancing memory and otherwise, still
facing the same issue. TRied every possible solution If anybody can help ,
please suggest me some solution.
*Problem:-* Manifoldcf crashes while crawling data from an intranet URL
hinting out below error in logs:-
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.resize(HashMap.java:704)
at java.util.HashMap.putVal(HashMap.java:629)
at java.util.HashMap.put(HashMap.java:612)
at
org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndEx
ecute(CacheManager.java:135)
at
org.apache.manifoldcf.agents.transformationconnection.TransformationC
onnectionManager.loadMultipleInternal(TransformationConnectionManager.java:265)
at
org.apache.manifoldcf.agents.transformationconnection.TransformationC
onnectionManager.loadMultiple(TransformationConnectionManager.java:246)
agents process could not start - shutting down
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
at
org.postgresql.jdbc.PgConnection.createStatement(PgConnection.java:11
93)
at
org.postgresql.jdbc.PgConnection.createStatement(PgConnection.java:15
77)
at
org.postgresql.jdbc.PgConnection.createStatement(PgConnection.java:36
7)
at
org.apache.manifoldcf.core.database.Database.execute(Database.java:87
3)
at
org.apache.manifoldcf.core.database.Database$ExecuteQueryThread.run(D
atabase.java:696)
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
[Thread-491] INFO org.eclipse.jetty.server.ServerConnector - Stopped
ServerConne
ctor@3a4621bd{HTTP/1.1}{0.0.0.0:8345}
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: GC overhead limit exceeded
[Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped
o.e.
j.w.WebAppContext@6a57ae10{/mcf-api-service,file:/tmp/jetty-0.0.0.0-8345-mcf-api
-service.war-_mcf-api-service-any-2010273384908625074.dir/webapp/,UNAVAILABLE}{/
usr/share/manifoldcf/example/./../web/war/mcf-api-service.war}
[Thread-491] INFO org.eclipse.jetty.server.handler.ContextHandler - Stopped
o.e.
j.w.WebAppContext@51c693d{/mcf-authority-service,file:/tmp/jetty-0.0.0.0-8345-mc
f-authority-service.war-_mcf-authority-service-any-1431262612693374489.dir/webap
p/,UNAVAILABLE}{/usr/share/manifoldcf/example/./../web/war/mcf-authority-service
.war}
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
Exception in thread "PostgreSQL-JDBC-SharedTimer-1"
java.lang.OutOfMemoryError:
Java heap space
All configuration are:-
I have -Xms1024m ,-Xmx1024m memory allocated in
*start-options.env.unix, start-options.env.win file.*
Also Configuration:-
1) *For Crawler server *- 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
E5-2660v3 @ 2.60GHz and
2) *For Elasticsearch server* - 48GB and 16-Core Intel(R) Xeon(R) CPU
E5-2660 v3 @ 2.60GHz and i am using postgres as database.
3) Database as PostgreSQL
4) No of worker threads:-20.
5) manifold Version :-2.13
Can anybody please suggest some solution to get rid of this error and
crash, as whenever this istutaion comes, manifoldCF UI results me this:_
[image: image.png]
Any suggestion would be highly appreciated.
Thanks
Priya
On Fri, Aug 16, 2019 at 6:41 PM Priya Arora <pr...@smartshore.nl> wrote:
> Sure I will try to do that.. and will get back to you
>
> Sent from my iPhone
>
> > On 16-Aug-2019, at 6:23 PM, Michael Cizmar <mi...@michaelcizmar.com>
> wrote:
> >
> > Priya - Was this right?
> >
> > 48GB and 1-Core Intel(R) Xeon(R) CPU
> >
> > While not directly related to an out of memory issue, you should have
> more
> > cores allocated to ES.
> >
> >> On Fri, Aug 16, 2019 at 1:09 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> >>
> >> *Existing Threads/connections configuration is :-*
> >>
> >> How many worker threads do you have? - 15 worker threads has been
> >> allocated(in properties.xml file).
> >> And the Tika Extractor connections -10 connections are defined.
> >>
> >> Is this suggested to reduce the number more.
> >> If not, what else can be a solution
> >>
> >> Thanks
> >> Priya
> >>
> >>
> >>
> >>> On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com>
> wrote:
> >>>
> >>> How many worker threads do you have?
> >>> Even if each worker thread is constrained in memory, and they should
> be,
> >>> you can easily cause things to run out of memory by giving too many
> >> worker
> >>> threads. Another way to keep Tika's usage constrained would be to
> reduce
> >>> the number of Tika Extractor connections, because that effectively
> limits
> >>> the number of extractions that can be going on at the same time.
> >>>
> >>> Karl
> >>>
> >>>
> >>>> On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> >>>>
> >>>> Yes , I am using Tika Extractor. And the version used for manifold is
> >>> 2.13.
> >>>> Also I am using postgres as database.
> >>>>
> >>>> I have 4 types of jobs
> >>>> One is accessing/re crawling data from a public site. Other three are
> >>>> accessing intranet site.
> >>>> Out of which two are giving me correct output-without any error and
> >> third
> >>>> one which is having data more than the other two , and giving me this
> >>>> error.
> >>>>
> >>>> Is there any possibility with site accessibility issue. Can you please
> >>>> suggest some solution
> >>>> Thanks and regards
> >>>> Priya
> >>>>
> >>>> On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
> >> wrote:
> >>>>
> >>>>> I will need to know more. Do you have the tika extractor in your
> >>>>> pipeline? If so, what version of ManifoldCF are you using? Tika has
> >>> had
> >>>>> bugs related to memory consumption in the past; the out of memory
> >>>> exception
> >>>>> may be coming from it and therefore a stack trace is critical to
> >> have.
> >>>>>
> >>>>> Alternatively, you can upgrade to the latest version of MCF (2.13)
> >> and
> >>>> that
> >>>>> has a newer version of Tika without those problem. But you may need
> >> to
> >>>> get
> >>>>> the agents process more memory.
> >>>>>
> >>>>> Another possible cause is that you're using hsqldb in production.
> >>> HSQLDB
> >>>>> keeps all of its tables in memory. If you have a large crawl, you do
> >>> not
> >>>>> want to use HSQLDB.
> >>>>>
> >>>>> Thanks,
> >>>>> Karl
> >>>>>
> >>>>>
> >>>>> On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
> >>> wrote:
> >>>>>
> >>>>>> Hi Karl,
> >>>>>>
> >>>>>> Manifold CF logs hints out me an error like :
> >>>>>> agents process ran out of memory - shutting down
> >>>>>> java.lang.OutOfMemoryError: Java heap space
> >>>>>>
> >>>>>> Also I have -Xms1024m ,-Xmx1024m memory allocated in
> >>>>>> start-options.env.unix, start-options.env.win file.
> >>>>>> Also Configuration:-
> >>>>>> 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> >>>> E5-2660
> >>>>>> v3 @ 2.60GHz and
> >>>>>>
> >>>>>> 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
> >>>>> E5-2660
> >>>>>> v3 @ 2.60GHz and i am using postgres as database.
> >>>>>>
> >>>>>> Can you please help me out, what to do in this case.
> >>>>>>
> >>>>>> Thanks
> >>>>>> Priya
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> The error occurs, I believe, as the result of basic connection
> >>>>> problems,
> >>>>>>> e.g. the connection is getting rejected. You can find more
> >>>> information
> >>>>>> in
> >>>>>>> the simple history, and in the manifoldcf log.
> >>>>>>>
> >>>>>>> I would like to know the underlying cause, since the connector
> >>> should
> >>>>> be
> >>>>>>> resilient against errors of this kind.
> >>>>>>>
> >>>>>>> Karl
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Karl,
> >>>>>>>>
> >>>>>>>> I have an web Repository connector(Seeds:- an intranet Site).,
> >>> and
> >>>>> job
> >>>>>> i
> >>>>>>>> son Production server.
> >>>>>>>>
> >>>>>>>> When i ran job on PROD, the job stops itself 2 times with and
> >>>>>>> error:Error:
> >>>>>>>> Unexpected HTTP result code: -1: null.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Can you please provide me an idea, in which it happens so?
> >>>>>>>>
> >>>>>>>> Thanks and regards
> >>>>>>>> Priya Arora
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
Sure I will try to do that.. and will get back to you
Sent from my iPhone
> On 16-Aug-2019, at 6:23 PM, Michael Cizmar <mi...@michaelcizmar.com> wrote:
>
> Priya - Was this right?
>
> 48GB and 1-Core Intel(R) Xeon(R) CPU
>
> While not directly related to an out of memory issue, you should have more
> cores allocated to ES.
>
>> On Fri, Aug 16, 2019 at 1:09 AM Priya Arora <pr...@smartshore.nl> wrote:
>>
>> *Existing Threads/connections configuration is :-*
>>
>> How many worker threads do you have? - 15 worker threads has been
>> allocated(in properties.xml file).
>> And the Tika Extractor connections -10 connections are defined.
>>
>> Is this suggested to reduce the number more.
>> If not, what else can be a solution
>>
>> Thanks
>> Priya
>>
>>
>>
>>> On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com> wrote:
>>>
>>> How many worker threads do you have?
>>> Even if each worker thread is constrained in memory, and they should be,
>>> you can easily cause things to run out of memory by giving too many
>> worker
>>> threads. Another way to keep Tika's usage constrained would be to reduce
>>> the number of Tika Extractor connections, because that effectively limits
>>> the number of extractions that can be going on at the same time.
>>>
>>> Karl
>>>
>>>
>>>> On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl> wrote:
>>>>
>>>> Yes , I am using Tika Extractor. And the version used for manifold is
>>> 2.13.
>>>> Also I am using postgres as database.
>>>>
>>>> I have 4 types of jobs
>>>> One is accessing/re crawling data from a public site. Other three are
>>>> accessing intranet site.
>>>> Out of which two are giving me correct output-without any error and
>> third
>>>> one which is having data more than the other two , and giving me this
>>>> error.
>>>>
>>>> Is there any possibility with site accessibility issue. Can you please
>>>> suggest some solution
>>>> Thanks and regards
>>>> Priya
>>>>
>>>> On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
>> wrote:
>>>>
>>>>> I will need to know more. Do you have the tika extractor in your
>>>>> pipeline? If so, what version of ManifoldCF are you using? Tika has
>>> had
>>>>> bugs related to memory consumption in the past; the out of memory
>>>> exception
>>>>> may be coming from it and therefore a stack trace is critical to
>> have.
>>>>>
>>>>> Alternatively, you can upgrade to the latest version of MCF (2.13)
>> and
>>>> that
>>>>> has a newer version of Tika without those problem. But you may need
>> to
>>>> get
>>>>> the agents process more memory.
>>>>>
>>>>> Another possible cause is that you're using hsqldb in production.
>>> HSQLDB
>>>>> keeps all of its tables in memory. If you have a large crawl, you do
>>> not
>>>>> want to use HSQLDB.
>>>>>
>>>>> Thanks,
>>>>> Karl
>>>>>
>>>>>
>>>>> On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
>>> wrote:
>>>>>
>>>>>> Hi Karl,
>>>>>>
>>>>>> Manifold CF logs hints out me an error like :
>>>>>> agents process ran out of memory - shutting down
>>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>>
>>>>>> Also I have -Xms1024m ,-Xmx1024m memory allocated in
>>>>>> start-options.env.unix, start-options.env.win file.
>>>>>> Also Configuration:-
>>>>>> 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
>>>> E5-2660
>>>>>> v3 @ 2.60GHz and
>>>>>>
>>>>>> 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
>>>>> E5-2660
>>>>>> v3 @ 2.60GHz and i am using postgres as database.
>>>>>>
>>>>>> Can you please help me out, what to do in this case.
>>>>>>
>>>>>> Thanks
>>>>>> Priya
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> The error occurs, I believe, as the result of basic connection
>>>>> problems,
>>>>>>> e.g. the connection is getting rejected. You can find more
>>>> information
>>>>>> in
>>>>>>> the simple history, and in the manifoldcf log.
>>>>>>>
>>>>>>> I would like to know the underlying cause, since the connector
>>> should
>>>>> be
>>>>>>> resilient against errors of this kind.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
>>>> wrote:
>>>>>>>
>>>>>>>> Hi Karl,
>>>>>>>>
>>>>>>>> I have an web Repository connector(Seeds:- an intranet Site).,
>>> and
>>>>> job
>>>>>> i
>>>>>>>> son Production server.
>>>>>>>>
>>>>>>>> When i ran job on PROD, the job stops itself 2 times with and
>>>>>>> error:Error:
>>>>>>>> Unexpected HTTP result code: -1: null.
>>>>>>>>
>>>>>>>>
>>>>>>>> Can you please provide me an idea, in which it happens so?
>>>>>>>>
>>>>>>>> Thanks and regards
>>>>>>>> Priya Arora
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Unexpected HTTP result code: -1: null
Posted by Michael Cizmar <mi...@michaelcizmar.com>.
Priya - Was this right?
48GB and 1-Core Intel(R) Xeon(R) CPU
While not directly related to an out of memory issue, you should have more
cores allocated to ES.
On Fri, Aug 16, 2019 at 1:09 AM Priya Arora <pr...@smartshore.nl> wrote:
> *Existing Threads/connections configuration is :-*
>
> How many worker threads do you have? - 15 worker threads has been
> allocated(in properties.xml file).
> And the Tika Extractor connections -10 connections are defined.
>
> Is this suggested to reduce the number more.
> If not, what else can be a solution
>
> Thanks
> Priya
>
>
>
> On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com> wrote:
>
> > How many worker threads do you have?
> > Even if each worker thread is constrained in memory, and they should be,
> > you can easily cause things to run out of memory by giving too many
> worker
> > threads. Another way to keep Tika's usage constrained would be to reduce
> > the number of Tika Extractor connections, because that effectively limits
> > the number of extractions that can be going on at the same time.
> >
> > Karl
> >
> >
> > On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl> wrote:
> >
> > > Yes , I am using Tika Extractor. And the version used for manifold is
> > 2.13.
> > > Also I am using postgres as database.
> > >
> > > I have 4 types of jobs
> > > One is accessing/re crawling data from a public site. Other three are
> > > accessing intranet site.
> > > Out of which two are giving me correct output-without any error and
> third
> > > one which is having data more than the other two , and giving me this
> > > error.
> > >
> > > Is there any possibility with site accessibility issue. Can you please
> > > suggest some solution
> > > Thanks and regards
> > > Priya
> > >
> > > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com>
> wrote:
> > >
> > > > I will need to know more. Do you have the tika extractor in your
> > > > pipeline? If so, what version of ManifoldCF are you using? Tika has
> > had
> > > > bugs related to memory consumption in the past; the out of memory
> > > exception
> > > > may be coming from it and therefore a stack trace is critical to
> have.
> > > >
> > > > Alternatively, you can upgrade to the latest version of MCF (2.13)
> and
> > > that
> > > > has a newer version of Tika without those problem. But you may need
> to
> > > get
> > > > the agents process more memory.
> > > >
> > > > Another possible cause is that you're using hsqldb in production.
> > HSQLDB
> > > > keeps all of its tables in memory. If you have a large crawl, you do
> > not
> > > > want to use HSQLDB.
> > > >
> > > > Thanks,
> > > > Karl
> > > >
> > > >
> > > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
> > wrote:
> > > >
> > > > > Hi Karl,
> > > > >
> > > > > Manifold CF logs hints out me an error like :
> > > > > agents process ran out of memory - shutting down
> > > > > java.lang.OutOfMemoryError: Java heap space
> > > > >
> > > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > > start-options.env.unix, start-options.env.win file.
> > > > > Also Configuration:-
> > > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> > > E5-2660
> > > > > v3 @ 2.60GHz and
> > > > >
> > > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
> > > > E5-2660
> > > > > v3 @ 2.60GHz and i am using postgres as database.
> > > > >
> > > > > Can you please help me out, what to do in this case.
> > > > >
> > > > > Thanks
> > > > > Priya
> > > > >
> > > > >
> > > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
> > > wrote:
> > > > >
> > > > > > The error occurs, I believe, as the result of basic connection
> > > > problems,
> > > > > > e.g. the connection is getting rejected. You can find more
> > > information
> > > > > in
> > > > > > the simple history, and in the manifoldcf log.
> > > > > >
> > > > > > I would like to know the underlying cause, since the connector
> > should
> > > > be
> > > > > > resilient against errors of this kind.
> > > > > >
> > > > > > Karl
> > > > > >
> > > > > >
> > > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
> > > wrote:
> > > > > >
> > > > > > > Hi Karl,
> > > > > > >
> > > > > > > I have an web Repository connector(Seeds:- an intranet Site).,
> > and
> > > > job
> > > > > i
> > > > > > > son Production server.
> > > > > > >
> > > > > > > When i ran job on PROD, the job stops itself 2 times with and
> > > > > > error:Error:
> > > > > > > Unexpected HTTP result code: -1: null.
> > > > > > >
> > > > > > >
> > > > > > > Can you please provide me an idea, in which it happens so?
> > > > > > >
> > > > > > > Thanks and regards
> > > > > > > Priya Arora
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
*Existing Threads/connections configuration is :-*
How many worker threads do you have? - 15 worker threads has been
allocated(in properties.xml file).
And the Tika Extractor connections -10 connections are defined.
Is this suggested to reduce the number more.
If not, what else can be a solution
Thanks
Priya
On Wed, Aug 14, 2019 at 5:32 PM Karl Wright <da...@gmail.com> wrote:
> How many worker threads do you have?
> Even if each worker thread is constrained in memory, and they should be,
> you can easily cause things to run out of memory by giving too many worker
> threads. Another way to keep Tika's usage constrained would be to reduce
> the number of Tika Extractor connections, because that effectively limits
> the number of extractions that can be going on at the same time.
>
> Karl
>
>
> On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl> wrote:
>
> > Yes , I am using Tika Extractor. And the version used for manifold is
> 2.13.
> > Also I am using postgres as database.
> >
> > I have 4 types of jobs
> > One is accessing/re crawling data from a public site. Other three are
> > accessing intranet site.
> > Out of which two are giving me correct output-without any error and third
> > one which is having data more than the other two , and giving me this
> > error.
> >
> > Is there any possibility with site accessibility issue. Can you please
> > suggest some solution
> > Thanks and regards
> > Priya
> >
> > On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com> wrote:
> >
> > > I will need to know more. Do you have the tika extractor in your
> > > pipeline? If so, what version of ManifoldCF are you using? Tika has
> had
> > > bugs related to memory consumption in the past; the out of memory
> > exception
> > > may be coming from it and therefore a stack trace is critical to have.
> > >
> > > Alternatively, you can upgrade to the latest version of MCF (2.13) and
> > that
> > > has a newer version of Tika without those problem. But you may need to
> > get
> > > the agents process more memory.
> > >
> > > Another possible cause is that you're using hsqldb in production.
> HSQLDB
> > > keeps all of its tables in memory. If you have a large crawl, you do
> not
> > > want to use HSQLDB.
> > >
> > > Thanks,
> > > Karl
> > >
> > >
> > > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> > >
> > > > Hi Karl,
> > > >
> > > > Manifold CF logs hints out me an error like :
> > > > agents process ran out of memory - shutting down
> > > > java.lang.OutOfMemoryError: Java heap space
> > > >
> > > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > > start-options.env.unix, start-options.env.win file.
> > > > Also Configuration:-
> > > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> > E5-2660
> > > > v3 @ 2.60GHz and
> > > >
> > > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
> > > E5-2660
> > > > v3 @ 2.60GHz and i am using postgres as database.
> > > >
> > > > Can you please help me out, what to do in this case.
> > > >
> > > > Thanks
> > > > Priya
> > > >
> > > >
> > > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
> > wrote:
> > > >
> > > > > The error occurs, I believe, as the result of basic connection
> > > problems,
> > > > > e.g. the connection is getting rejected. You can find more
> > information
> > > > in
> > > > > the simple history, and in the manifoldcf log.
> > > > >
> > > > > I would like to know the underlying cause, since the connector
> should
> > > be
> > > > > resilient against errors of this kind.
> > > > >
> > > > > Karl
> > > > >
> > > > >
> > > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
> > wrote:
> > > > >
> > > > > > Hi Karl,
> > > > > >
> > > > > > I have an web Repository connector(Seeds:- an intranet Site).,
> and
> > > job
> > > > i
> > > > > > son Production server.
> > > > > >
> > > > > > When i ran job on PROD, the job stops itself 2 times with and
> > > > > error:Error:
> > > > > > Unexpected HTTP result code: -1: null.
> > > > > >
> > > > > >
> > > > > > Can you please provide me an idea, in which it happens so?
> > > > > >
> > > > > > Thanks and regards
> > > > > > Priya Arora
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Karl Wright <da...@gmail.com>.
How many worker threads do you have?
Even if each worker thread is constrained in memory, and they should be,
you can easily cause things to run out of memory by giving too many worker
threads. Another way to keep Tika's usage constrained would be to reduce
the number of Tika Extractor connections, because that effectively limits
the number of extractions that can be going on at the same time.
Karl
On Wed, Aug 14, 2019 at 7:23 AM Priya Arora <pr...@smartshore.nl> wrote:
> Yes , I am using Tika Extractor. And the version used for manifold is 2.13.
> Also I am using postgres as database.
>
> I have 4 types of jobs
> One is accessing/re crawling data from a public site. Other three are
> accessing intranet site.
> Out of which two are giving me correct output-without any error and third
> one which is having data more than the other two , and giving me this
> error.
>
> Is there any possibility with site accessibility issue. Can you please
> suggest some solution
> Thanks and regards
> Priya
>
> On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com> wrote:
>
> > I will need to know more. Do you have the tika extractor in your
> > pipeline? If so, what version of ManifoldCF are you using? Tika has had
> > bugs related to memory consumption in the past; the out of memory
> exception
> > may be coming from it and therefore a stack trace is critical to have.
> >
> > Alternatively, you can upgrade to the latest version of MCF (2.13) and
> that
> > has a newer version of Tika without those problem. But you may need to
> get
> > the agents process more memory.
> >
> > Another possible cause is that you're using hsqldb in production. HSQLDB
> > keeps all of its tables in memory. If you have a large crawl, you do not
> > want to use HSQLDB.
> >
> > Thanks,
> > Karl
> >
> >
> > On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl> wrote:
> >
> > > Hi Karl,
> > >
> > > Manifold CF logs hints out me an error like :
> > > agents process ran out of memory - shutting down
> > > java.lang.OutOfMemoryError: Java heap space
> > >
> > > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > > start-options.env.unix, start-options.env.win file.
> > > Also Configuration:-
> > > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU
> E5-2660
> > > v3 @ 2.60GHz and
> > >
> > > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
> > E5-2660
> > > v3 @ 2.60GHz and i am using postgres as database.
> > >
> > > Can you please help me out, what to do in this case.
> > >
> > > Thanks
> > > Priya
> > >
> > >
> > > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com>
> wrote:
> > >
> > > > The error occurs, I believe, as the result of basic connection
> > problems,
> > > > e.g. the connection is getting rejected. You can find more
> information
> > > in
> > > > the simple history, and in the manifoldcf log.
> > > >
> > > > I would like to know the underlying cause, since the connector should
> > be
> > > > resilient against errors of this kind.
> > > >
> > > > Karl
> > > >
> > > >
> > > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl>
> wrote:
> > > >
> > > > > Hi Karl,
> > > > >
> > > > > I have an web Repository connector(Seeds:- an intranet Site)., and
> > job
> > > i
> > > > > son Production server.
> > > > >
> > > > > When i ran job on PROD, the job stops itself 2 times with and
> > > > error:Error:
> > > > > Unexpected HTTP result code: -1: null.
> > > > >
> > > > >
> > > > > Can you please provide me an idea, in which it happens so?
> > > > >
> > > > > Thanks and regards
> > > > > Priya Arora
> > > > >
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
Yes , I am using Tika Extractor. And the version used for manifold is 2.13.
Also I am using postgres as database.
I have 4 types of jobs
One is accessing/re crawling data from a public site. Other three are
accessing intranet site.
Out of which two are giving me correct output-without any error and third
one which is having data more than the other two , and giving me this
error.
Is there any possibility with site accessibility issue. Can you please
suggest some solution
Thanks and regards
Priya
On Wed, Aug 14, 2019 at 3:11 PM Karl Wright <da...@gmail.com> wrote:
> I will need to know more. Do you have the tika extractor in your
> pipeline? If so, what version of ManifoldCF are you using? Tika has had
> bugs related to memory consumption in the past; the out of memory exception
> may be coming from it and therefore a stack trace is critical to have.
>
> Alternatively, you can upgrade to the latest version of MCF (2.13) and that
> has a newer version of Tika without those problem. But you may need to get
> the agents process more memory.
>
> Another possible cause is that you're using hsqldb in production. HSQLDB
> keeps all of its tables in memory. If you have a large crawl, you do not
> want to use HSQLDB.
>
> Thanks,
> Karl
>
>
> On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl> wrote:
>
> > Hi Karl,
> >
> > Manifold CF logs hints out me an error like :
> > agents process ran out of memory - shutting down
> > java.lang.OutOfMemoryError: Java heap space
> >
> > Also I have -Xms1024m ,-Xmx1024m memory allocated in
> > start-options.env.unix, start-options.env.win file.
> > Also Configuration:-
> > 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660
> > v3 @ 2.60GHz and
> >
> > 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU
> E5-2660
> > v3 @ 2.60GHz and i am using postgres as database.
> >
> > Can you please help me out, what to do in this case.
> >
> > Thanks
> > Priya
> >
> >
> > On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com> wrote:
> >
> > > The error occurs, I believe, as the result of basic connection
> problems,
> > > e.g. the connection is getting rejected. You can find more information
> > in
> > > the simple history, and in the manifoldcf log.
> > >
> > > I would like to know the underlying cause, since the connector should
> be
> > > resilient against errors of this kind.
> > >
> > > Karl
> > >
> > >
> > > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl> wrote:
> > >
> > > > Hi Karl,
> > > >
> > > > I have an web Repository connector(Seeds:- an intranet Site)., and
> job
> > i
> > > > son Production server.
> > > >
> > > > When i ran job on PROD, the job stops itself 2 times with and
> > > error:Error:
> > > > Unexpected HTTP result code: -1: null.
> > > >
> > > >
> > > > Can you please provide me an idea, in which it happens so?
> > > >
> > > > Thanks and regards
> > > > Priya Arora
> > > >
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Karl Wright <da...@gmail.com>.
I will need to know more. Do you have the tika extractor in your
pipeline? If so, what version of ManifoldCF are you using? Tika has had
bugs related to memory consumption in the past; the out of memory exception
may be coming from it and therefore a stack trace is critical to have.
Alternatively, you can upgrade to the latest version of MCF (2.13) and that
has a newer version of Tika without those problem. But you may need to get
the agents process more memory.
Another possible cause is that you're using hsqldb in production. HSQLDB
keeps all of its tables in memory. If you have a large crawl, you do not
want to use HSQLDB.
Thanks,
Karl
On Wed, Aug 14, 2019 at 3:41 AM Priya Arora <pr...@smartshore.nl> wrote:
> Hi Karl,
>
> Manifold CF logs hints out me an error like :
> agents process ran out of memory - shutting down
> java.lang.OutOfMemoryError: Java heap space
>
> Also I have -Xms1024m ,-Xmx1024m memory allocated in
> start-options.env.unix, start-options.env.win file.
> Also Configuration:-
> 1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660
> v3 @ 2.60GHz and
>
> 2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660
> v3 @ 2.60GHz and i am using postgres as database.
>
> Can you please help me out, what to do in this case.
>
> Thanks
> Priya
>
>
> On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com> wrote:
>
> > The error occurs, I believe, as the result of basic connection problems,
> > e.g. the connection is getting rejected. You can find more information
> in
> > the simple history, and in the manifoldcf log.
> >
> > I would like to know the underlying cause, since the connector should be
> > resilient against errors of this kind.
> >
> > Karl
> >
> >
> > On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl> wrote:
> >
> > > Hi Karl,
> > >
> > > I have an web Repository connector(Seeds:- an intranet Site)., and job
> i
> > > son Production server.
> > >
> > > When i ran job on PROD, the job stops itself 2 times with and
> > error:Error:
> > > Unexpected HTTP result code: -1: null.
> > >
> > >
> > > Can you please provide me an idea, in which it happens so?
> > >
> > > Thanks and regards
> > > Priya Arora
> > >
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Priya Arora <pr...@smartshore.nl>.
Hi Karl,
Manifold CF logs hints out me an error like :
agents process ran out of memory - shutting down
java.lang.OutOfMemoryError: Java heap space
Also I have -Xms1024m ,-Xmx1024m memory allocated in
start-options.env.unix, start-options.env.win file.
Also Configuration:-
1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660
v3 @ 2.60GHz and
2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660
v3 @ 2.60GHz and i am using postgres as database.
Can you please help me out, what to do in this case.
Thanks
Priya
On Wed, Aug 14, 2019 at 12:33 PM Karl Wright <da...@gmail.com> wrote:
> The error occurs, I believe, as the result of basic connection problems,
> e.g. the connection is getting rejected. You can find more information in
> the simple history, and in the manifoldcf log.
>
> I would like to know the underlying cause, since the connector should be
> resilient against errors of this kind.
>
> Karl
>
>
> On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl> wrote:
>
> > Hi Karl,
> >
> > I have an web Repository connector(Seeds:- an intranet Site)., and job i
> > son Production server.
> >
> > When i ran job on PROD, the job stops itself 2 times with and
> error:Error:
> > Unexpected HTTP result code: -1: null.
> >
> >
> > Can you please provide me an idea, in which it happens so?
> >
> > Thanks and regards
> > Priya Arora
> >
>
Re: Unexpected HTTP result code: -1: null
Posted by Karl Wright <da...@gmail.com>.
The error occurs, I believe, as the result of basic connection problems,
e.g. the connection is getting rejected. You can find more information in
the simple history, and in the manifoldcf log.
I would like to know the underlying cause, since the connector should be
resilient against errors of this kind.
Karl
On Wed, Aug 14, 2019, 1:46 AM Priya Arora <pr...@smartshore.nl> wrote:
> Hi Karl,
>
> I have an web Repository connector(Seeds:- an intranet Site)., and job i
> son Production server.
>
> When i ran job on PROD, the job stops itself 2 times with and error:Error:
> Unexpected HTTP result code: -1: null.
>
>
> Can you please provide me an idea, in which it happens so?
>
> Thanks and regards
> Priya Arora
>