You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Paul Libbrecht <pa...@hoplahup.net> on 2012/05/09 12:21:23 UTC
anticipating the indexing completion
Hello SOLR experts,
I have my own indexing web-application which talks in XML to SOLR. It works wonderfully well.
The queue is displayed in the indexer, so that experts can have a track that it went well into the index.
However, i see no way currently to display that solr's searcher includes the changed resource.
The commit triggers after a few, then the warm-up time arrives, ... it can be as long as a few minutes.
Well, I can read and check version numbers.
Wouldn't there be a better way?
thanks in advance
Paul
Re: anticipating the indexing completion
Posted by Jack Krupansky <ja...@basetechnology.com>.
I don't personally know the details, but I heard somebody at the conference
say that you could hit some solr admin stats URL to access some MBeans stat
that tells you whether there are pending documents that are not yet
committed.
I see a reference to "docsPending" mentioned here:
http://lucidworks.lucidimagination.com/display/lweug/Integrating+Monitoring+Services
Here's a reference for viewing that stat on Solr admin:
http://lucene.472066.n3.nabble.com/Easy-way-to-tell-if-there-are-pending-documents-td3507319.html
And:
http://lucene.472066.n3.nabble.com/Get-of-docs-pending-commit-td494627.html
-- Jack Krupansky
-----Original Message-----
From: Paul Libbrecht
Sent: Wednesday, May 09, 2012 5:22 PM
To: solr-user@lucene.apache.org
Subject: Re: anticipating the indexing completion
Hi Otis,
I was not so much trying to find estimates but trying to indicate if it was
done.
I understand the indexing works in batches after which there's a commit
followed by a warm-phase: if my add could be responded with a "commit id"
and that one could check that this commit is now available, I'd have my
trick.
paul
Le 9 mai 2012 à 22:45, Otis Gospodnetic a écrit :
> Are you asking how to figure out the time between "add doc" and "see doc"?
>
> I suppose it could be useful to have Solr expose info about "how much time
> until the next autocommit" and then you could add that to the warmup time
> from previous warming and estimate.
>
> Otis
> ----
> Performance Monitoring for Solr / ElasticSearch / HBase -
> http://sematext.com/spm
>
>
>
>> ________________________________
>> From: Paul Libbrecht <pa...@hoplahup.net>
>> To: solr-user@lucene.apache.org
>> Sent: Wednesday, May 9, 2012 6:21 AM
>> Subject: anticipating the indexing completion
>>
>>
>> Hello SOLR experts,
>>
>> I have my own indexing web-application which talks in XML to SOLR. It
>> works wonderfully well.
>> The queue is displayed in the indexer, so that experts can have a track
>> that it went well into the index.
>>
>> However, i see no way currently to display that solr's searcher includes
>> the changed resource.
>> The commit triggers after a few, then the warm-up time arrives, ... it
>> can be as long as a few minutes.
>>
>> Well, I can read and check version numbers.
>> Wouldn't there be a better way?
>>
>> thanks in advance
>>
>> Paul
>>
Re: anticipating the indexing completion
Posted by Paul Libbrecht <pa...@hoplahup.net>.
Hi Otis,
I was not so much trying to find estimates but trying to indicate if it was done.
I understand the indexing works in batches after which there's a commit followed by a warm-phase: if my add could be responded with a "commit id" and that one could check that this commit is now available, I'd have my trick.
paul
Le 9 mai 2012 à 22:45, Otis Gospodnetic a écrit :
> Are you asking how to figure out the time between "add doc" and "see doc"?
>
> I suppose it could be useful to have Solr expose info about "how much time until the next autocommit" and then you could add that to the warmup time from previous warming and estimate.
>
> Otis
> ----
> Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm
>
>
>
>> ________________________________
>> From: Paul Libbrecht <pa...@hoplahup.net>
>> To: solr-user@lucene.apache.org
>> Sent: Wednesday, May 9, 2012 6:21 AM
>> Subject: anticipating the indexing completion
>>
>>
>> Hello SOLR experts,
>>
>> I have my own indexing web-application which talks in XML to SOLR. It works wonderfully well.
>> The queue is displayed in the indexer, so that experts can have a track that it went well into the index.
>>
>> However, i see no way currently to display that solr's searcher includes the changed resource.
>> The commit triggers after a few, then the warm-up time arrives, ... it can be as long as a few minutes.
>>
>> Well, I can read and check version numbers.
>> Wouldn't there be a better way?
>>
>> thanks in advance
>>
>> Paul
>>
Re: anticipating the indexing completion
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Paul,
Are you asking how to figure out the time between "add doc" and "see doc"?
I suppose it could be useful to have Solr expose info about "how much time until the next autocommit" and then you could add that to the warmup time from previous warming and estimate.
Otis
----
Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm
>________________________________
> From: Paul Libbrecht <pa...@hoplahup.net>
>To: solr-user@lucene.apache.org
>Sent: Wednesday, May 9, 2012 6:21 AM
>Subject: anticipating the indexing completion
>
>
>Hello SOLR experts,
>
>I have my own indexing web-application which talks in XML to SOLR. It works wonderfully well.
>The queue is displayed in the indexer, so that experts can have a track that it went well into the index.
>
>However, i see no way currently to display that solr's searcher includes the changed resource.
>The commit triggers after a few, then the warm-up time arrives, ... it can be as long as a few minutes.
>
>Well, I can read and check version numbers.
>Wouldn't there be a better way?
>
>thanks in advance
>
>Paul
>
>