You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Romaric Pighetti <ro...@francelabs.com> on 2018/10/08 08:23:23 UTC
Query to get the number of documents processed from PostgreSQL
Hi Karl,
I am currently facing the need of getting the number of documents
processed by MCF in a specific job.
This number is getting bigger than the limit set for the web interface
and i don't want to increase this limit because of the stress it will
put on the database (openning the tab in the UI will pop queries for all
the jobs, and I know from previous readings that these queries are heavy
to process for postgre).
Thus i would like to know if you can provide me with the query used in
the interface to display the number of processed documents so that i can
fire it to postgreSQL manually and request it only for the job i am
interested in; lowering the impact on postgre.
Thanks for your help.
Romaric
--
Romaric Pighetti
France Labs – Les experts du Search
Les créateurs de Datafari 4, LA solution de recherche pour entreprise
www.francelabs.com
Re: Query to get the number of documents processed from PostgreSQL
Posted by Romaric Pighetti <ro...@francelabs.com>.
Thanks Karl for the quick answer.
I guess to get only the documents completed while the job is running i
will have to fiddle around with the status, for which potential values
are expressed in the JobQueue class.
I noticed that sometimes (mainly when pausing and restarting a job),
selecting only the elements with status 'C' does not seem to give the
exact same value as the "Completed" column of the UI.
If you have any hint I am interested, else I will try to figure that out.
Thanks again,
Romaric
P.S.: copying my answer to the mailing list. Sorry Karl for mailing you
directly too, I wanted to answer to the mailing list.
Le 08/10/2018 à 12:17, Karl Wright a écrit :
> If you want all the documents for a specific job, the query is:
>
> select count(*) from jobqueue where jobid=<the_job_id>
>
> Karl
>
>
> On Mon, Oct 8, 2018 at 4:23 AM Romaric Pighetti
> <romaric.pighetti@francelabs.com
> <ma...@francelabs.com>> wrote:
>
> Hi Karl,
>
> I am currently facing the need of getting the number of documents
> processed by MCF in a specific job.
> This number is getting bigger than the limit set for the web
> interface
> and i don't want to increase this limit because of the stress it will
> put on the database (openning the tab in the UI will pop queries
> for all
> the jobs, and I know from previous readings that these queries are
> heavy
> to process for postgre).
> Thus i would like to know if you can provide me with the query
> used in
> the interface to display the number of processed documents so that
> i can
> fire it to postgreSQL manually and request it only for the job i am
> interested in; lowering the impact on postgre.
>
> Thanks for your help.
> Romaric
>
> --
> Romaric Pighetti
> France Labs – Les experts du Search
>
> Les créateurs de Datafari 4, LA solution de recherche pour entreprise
>
> www.francelabs.com <http://www.francelabs.com>
>
--
Re: Nouvelle signature jusqu'à novembre Romaric Pighetti
France Labs – Les experts du Search
Retrouvez-nous à l’Enterprise Search & Discovery
<http://www.enterprisesearchanddiscovery.com/2018/default.aspx> Summit à
Washington DC
cid:image001.png@01D42F35.80534520
<http://www.enterprisesearchanddiscovery.com/2018/default.aspx>
www.francelabs.com <http://www.francelabs.com/>
Re: Query to get the number of documents processed from PostgreSQL
Posted by Karl Wright <da...@gmail.com>.
If you want all the documents for a specific job, the query is:
select count(*) from jobqueue where jobid=<the_job_id>
Karl
On Mon, Oct 8, 2018 at 4:23 AM Romaric Pighetti <
romaric.pighetti@francelabs.com> wrote:
> Hi Karl,
>
> I am currently facing the need of getting the number of documents
> processed by MCF in a specific job.
> This number is getting bigger than the limit set for the web interface
> and i don't want to increase this limit because of the stress it will
> put on the database (openning the tab in the UI will pop queries for all
> the jobs, and I know from previous readings that these queries are heavy
> to process for postgre).
> Thus i would like to know if you can provide me with the query used in
> the interface to display the number of processed documents so that i can
> fire it to postgreSQL manually and request it only for the job i am
> interested in; lowering the impact on postgre.
>
> Thanks for your help.
> Romaric
>
> --
> Romaric Pighetti
> France Labs – Les experts du Search
>
> Les créateurs de Datafari 4, LA solution de recherche pour entreprise
>
> www.francelabs.com
>
>