FW: ManifoldCF Documentum connector slowness

[Gomathi Palanisamy <gp...@worldbankgroup.org>]

Documentum connector crawling performance can be improved with changing the MCF default HSQL to PostgreSQL? Any suggestions?

From: Gomathi Palanisamy
Sent: Wednesday, January 16, 2019 2:54 PM
To: 'user-info@manifoldcf.apache.org' <us...@manifoldcf.apache.org>>; 'user-faq@manifoldcf.apache.org' <us...@manifoldcf.apache.org>>
Subject: RE: ManifoldCF Documentum connector slowness


Hi Team,

We are crawling data from DCTM repository using ManiFoldCF documentum connector and writing the crawled data to MongoDB. Crawling triggered with throttling value 500.But crawling speed is very slow per minute connector is fetching only 170 documents. The server where MCF installed is configured with enough memory with 8 logical cores (CPU). Can someone help us here to improve crawling speed?

Thanks,
Gomathi

Re: FW: ManifoldCF Documentum connector slowness

[Karl Wright <da...@gmail.com>]

Hi,

HSQLDB is actually reasonably fast, but it has other problems, namely that
it stores whole DB tables in memory so if your crawl is large enough it
will run out.

The reason for Documentum connector slowness is almost always poor
Documentum performance, and has nothing to do with MCF itself.  You can
prove that by looking at the Simple History and seeing how long it takes to
fetch documents (for example).

Karl




On Thu, Jan 17, 2019 at 9:05 AM Gomathi Palanisamy <
gpalanisamy@worldbankgroup.org> wrote:

>
>
> Documentum connector crawling performance can be improved with changing
> the MCF default HSQL to PostgreSQL? Any suggestions?
>
>
>
> *From:* Gomathi Palanisamy
> *Sent:* Wednesday, January 16, 2019 2:54 PM
> *To:* 'user-info@manifoldcf.apache.org' <us...@manifoldcf.apache.org>;
> 'user-faq@manifoldcf.apache.org' <us...@manifoldcf.apache.org>
> *Subject:* RE: ManifoldCF Documentum connector slowness
>
>
>
>
>
> Hi Team,
>
>
>
> We are crawling data from DCTM repository using ManiFoldCF documentum
> connector and writing the crawled data to MongoDB. Crawling triggered with
> throttling value 500.But crawling speed is very slow per minute connector
> is fetching only 170 documents. The server where MCF installed is
> configured with enough memory with 8 logical cores (CPU). Can someone help
> us here to improve crawling speed?
>
>
>
> Thanks,
>
> Gomathi
>