You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Latter, Antoine" <An...@legis.wisconsin.gov> on 2011/11/14 17:39:45 UTC
Easy way to tell if there are pending documents
Hi Solr,
Does anyone know of an easy way to tell if there are pending documents waiting for commit?
Our application performs operations that are never safe to perform while commits are pending. We make this work by making sure that all indexing operations end in a commit, and stop the unsafe operations from running while a commit is running.
This works great most of the time, except when we have enough disk space to add documents to the pending area, but not enough disk space to do a commit - then the indexing operations only error out after they've done all of their adds.
It would be nice if the unsafe operation could somehow detect that there are pending documents and abort.
In the interim I'll have the unsafe operation perform a commit when it starts, but I've been weeding out useless commits from my app recently and I don't like them creeping back in.
Thanks,
Antoine
RE: Easy way to tell if there are pending documents
Posted by "Latter, Antoine" <An...@legis.wisconsin.gov>.
Excellent. It looks like I can drill down into exactly what I want without having to load up the rest of the statistics.
-----Original Message-----
From: Justin Caratzas [mailto:justin.caratzas@gmail.com]
Sent: Wednesday, November 16, 2011 10:41 AM
To: solr-user@lucene.apache.org
Subject: Re: Easy way to tell if there are pending documents
You can enable the stats handler
(https://issues.apache.org/jira/browse/SOLR-1750), and get inspect the json pragmatically.
-- Justin
"Latter, Antoine" <An...@legis.wisconsin.gov> writes:
> Thank you, that does help - but I am more looking for a way to get at this programmatically.
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Tuesday, November 15, 2011 11:22 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Easy way to tell if there are pending documents
>
> Antoine,
>
> On Solr Admin Stats page search for "docsPending". I think this is what you are looking for.
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene
> ecosystem search :: http://search-lucene.com/
>
>
>>________________________________
>>From: "Latter, Antoine" <An...@legis.wisconsin.gov>
>>To: "'solr-user@lucene.apache.org'" <so...@lucene.apache.org>
>>Sent: Monday, November 14, 2011 11:39 AM
>>Subject: Easy way to tell if there are pending documents
>>
>>Hi Solr,
>>
>>Does anyone know of an easy way to tell if there are pending documents waiting for commit?
>>
>>Our application performs operations that are never safe to perform
>>while commits are pending. We make this work by making sure that all
>>indexing operations end in a commit, and stop the unsafe operations
>>from running while a commit is running.
>>
>>This works great most of the time, except when we have enough disk
>>space to add documents to the pending area, but not enough disk space
>>to do a commit - then the indexing operations only error out after
>>they've done all of their adds.
>>
>>It would be nice if the unsafe operation could somehow detect that there are pending documents and abort.
>>
>>In the interim I'll have the unsafe operation perform a commit when it starts, but I've been weeding out useless commits from my app recently and I don't like them creeping back in.
>>
>>Thanks,
>>Antoine
>>
>>
>>
Re: Easy way to tell if there are pending documents
Posted by Justin Caratzas <ju...@gmail.com>.
You can enable the stats handler
(https://issues.apache.org/jira/browse/SOLR-1750), and get inspect the
json pragmatically.
-- Justin
"Latter, Antoine" <An...@legis.wisconsin.gov> writes:
> Thank you, that does help - but I am more looking for a way to get at this programmatically.
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Tuesday, November 15, 2011 11:22 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Easy way to tell if there are pending documents
>
> Antoine,
>
> On Solr Admin Stats page search for "docsPending". I think this is what you are looking for.
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/
>
>
>>________________________________
>>From: "Latter, Antoine" <An...@legis.wisconsin.gov>
>>To: "'solr-user@lucene.apache.org'" <so...@lucene.apache.org>
>>Sent: Monday, November 14, 2011 11:39 AM
>>Subject: Easy way to tell if there are pending documents
>>
>>Hi Solr,
>>
>>Does anyone know of an easy way to tell if there are pending documents waiting for commit?
>>
>>Our application performs operations that are never safe to perform
>> while commits are pending. We make this work by making sure that all
>> indexing operations end in a commit, and stop the unsafe operations
>> from running while a commit is running.
>>
>>This works great most of the time, except when we have enough disk
>> space to add documents to the pending area, but not enough disk
>> space to do a commit - then the indexing operations only error out
>> after they've done all of their adds.
>>
>>It would be nice if the unsafe operation could somehow detect that there are pending documents and abort.
>>
>>In the interim I'll have the unsafe operation perform a commit when it starts, but I've been weeding out useless commits from my app recently and I don't like them creeping back in.
>>
>>Thanks,
>>Antoine
>>
>>
>>
RE: Easy way to tell if there are pending documents
Posted by "Latter, Antoine" <An...@legis.wisconsin.gov>.
Thank you, that does help - but I am more looking for a way to get at this programmatically.
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Tuesday, November 15, 2011 11:22 AM
To: solr-user@lucene.apache.org
Subject: Re: Easy way to tell if there are pending documents
Antoine,
On Solr Admin Stats page search for "docsPending". I think this is what you are looking for.
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/
>________________________________
>From: "Latter, Antoine" <An...@legis.wisconsin.gov>
>To: "'solr-user@lucene.apache.org'" <so...@lucene.apache.org>
>Sent: Monday, November 14, 2011 11:39 AM
>Subject: Easy way to tell if there are pending documents
>
>Hi Solr,
>
>Does anyone know of an easy way to tell if there are pending documents waiting for commit?
>
>Our application performs operations that are never safe to perform while commits are pending. We make this work by making sure that all indexing operations end in a commit, and stop the unsafe operations from running while a commit is running.
>
>This works great most of the time, except when we have enough disk space to add documents to the pending area, but not enough disk space to do a commit - then the indexing operations only error out after they've done all of their adds.
>
>It would be nice if the unsafe operation could somehow detect that there are pending documents and abort.
>
>In the interim I'll have the unsafe operation perform a commit when it starts, but I've been weeding out useless commits from my app recently and I don't like them creeping back in.
>
>Thanks,
>Antoine
>
>
>
Re: Easy way to tell if there are pending documents
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Antoine,
On Solr Admin Stats page search for "docsPending". I think this is what you are looking for.
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/
>________________________________
>From: "Latter, Antoine" <An...@legis.wisconsin.gov>
>To: "'solr-user@lucene.apache.org'" <so...@lucene.apache.org>
>Sent: Monday, November 14, 2011 11:39 AM
>Subject: Easy way to tell if there are pending documents
>
>Hi Solr,
>
>Does anyone know of an easy way to tell if there are pending documents waiting for commit?
>
>Our application performs operations that are never safe to perform while commits are pending. We make this work by making sure that all indexing operations end in a commit, and stop the unsafe operations from running while a commit is running.
>
>This works great most of the time, except when we have enough disk space to add documents to the pending area, but not enough disk space to do a commit - then the indexing operations only error out after they've done all of their adds.
>
>It would be nice if the unsafe operation could somehow detect that there are pending documents and abort.
>
>In the interim I'll have the unsafe operation perform a commit when it starts, but I've been weeding out useless commits from my app recently and I don't like them creeping back in.
>
>Thanks,
>Antoine
>
>
>