You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Remko Mantel <re...@smartshore.nl> on 2019/02/21 13:18:35 UTC
Active documents
Good afternoon all,
I have actually 2 questions concerning 'Active documents' in a running job.
We are currently using ManifoldCF 2.10 and crawl and process more than 500k
documents. In Manifold we see a huge amount of documents in status
'active'. The processed amount is only going up slowly.
1) Am I correct that in the Status of Jobs overview, the number of Active
documents are actually documents in queue - waiting to be processed and
that by the end of a job this should end up in 0?
2) How should we set the settings in such a way that the number of active
documents stays limited and the queue does not get too long? Which Output,
Repository or Job settings could/should we change for this? Are there any
recommendations for this?
--
Met vriendelijke groet / Kind regards,
Remko Mantel
*Smartshore*
Email: remko@smartshore.nl
Mobile: +31 (0)6 15371601
Web: http://www.smartshore.nl
Re: Active documents
Posted by Remko Mantel <re...@smartshore.nl>.
Hi Karl,
We have set the Recrawl interval (I guess that is what you were asking for)
at 8640 - that is (according to us) 6 days. Or do you want to know wore
(specific) settings?
With the smaller jobs we actually see that the amount of active documents
is diminishing towards the end of the job, and in the end result in 0
before the job is Done.
Greetings,
Remko
On Thu, Feb 21, 2019 at 3:23 PM Karl Wright <da...@gmail.com> wrote:
> Please tell us how you have configured your job. Is it running in
> continuous mode? Because if so, that is exactly how it's supposed to look
> -- all documents remain active forever, until you stop the job.
>
> Karl
>
>
> On Thu, Feb 21, 2019 at 8:18 AM Remko Mantel <re...@smartshore.nl> wrote:
>
>> Good afternoon all,
>>
>> I have actually 2 questions concerning 'Active documents' in a running
>> job.
>>
>> We are currently using ManifoldCF 2.10 and crawl and process more than
>> 500k documents. In Manifold we see a huge amount of documents in status
>> 'active'. The processed amount is only going up slowly.
>>
>> 1) Am I correct that in the Status of Jobs overview, the number of Active
>> documents are actually documents in queue - waiting to be processed and
>> that by the end of a job this should end up in 0?
>>
>> 2) How should we set the settings in such a way that the number of active
>> documents stays limited and the queue does not get too long? Which Output,
>> Repository or Job settings could/should we change for this? Are there any
>> recommendations for this?
>> --
>> Met vriendelijke groet / Kind regards,
>>
>> Remko Mantel
>>
>> *Smartshore*
>> Email: remko@smartshore.nl
>> Mobile: +31 (0)6 15371601
>> Web: http://www.smartshore.nl
>>
>
--
Met vriendelijke groet / Kind regards,
Remko Mantel
*Smartshore*
Email: remko@smartshore.nl
Mobile: +31 (0)6 15371601
Web: http://www.smartshore.nl
Re: Active documents
Posted by Karl Wright <da...@gmail.com>.
Please tell us how you have configured your job. Is it running in
continuous mode? Because if so, that is exactly how it's supposed to look
-- all documents remain active forever, until you stop the job.
Karl
On Thu, Feb 21, 2019 at 8:18 AM Remko Mantel <re...@smartshore.nl> wrote:
> Good afternoon all,
>
> I have actually 2 questions concerning 'Active documents' in a running
> job.
>
> We are currently using ManifoldCF 2.10 and crawl and process more than
> 500k documents. In Manifold we see a huge amount of documents in status
> 'active'. The processed amount is only going up slowly.
>
> 1) Am I correct that in the Status of Jobs overview, the number of Active
> documents are actually documents in queue - waiting to be processed and
> that by the end of a job this should end up in 0?
>
> 2) How should we set the settings in such a way that the number of active
> documents stays limited and the queue does not get too long? Which Output,
> Repository or Job settings could/should we change for this? Are there any
> recommendations for this?
> --
> Met vriendelijke groet / Kind regards,
>
> Remko Mantel
>
> *Smartshore*
> Email: remko@smartshore.nl
> Mobile: +31 (0)6 15371601
> Web: http://www.smartshore.nl
>