You are viewing a plain text version of this content. The canonical link for it is here.

Posted to solr-user@lucene.apache.org by danben <da...@gmail.com> on 2009/07/08 17:58:41 UTC

SolrException - Lock obtain timed out, no leftover locks

Hi,

I'm running Solr 1.3.0 in multicore mode and feeding it data from which the
core name is inferred from a specific field. My service extracts the core
name and, if it has not seen it before, issues a create request for that
core before attempting to add the document (via SolrJ). I have a pool of
MyIndexers that run in parallel, taking documents from a queue and adding
them via the add method on the SolrServer instance corresponding to that
core (exactly one per core exists). Each core is in a separate data
directory. My timeouts are set as such:

I remove the index directories, start the server, check that no locks exist,
and generate ~500 documents spread across 5 cores for the MyIndexers to
handle. Each time, I see one or more exceptions with a message like

Lock_obtain_timed_out_SimpleFSLockmulticoreNewUser3dataindexlucenebd4994617386d14e2c8c29e23bcca719writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_...

When the indexers have completed, no lock is left over. There is no
discernible pattern as far as when the exception occurs (ie, it does not
tend to happen on the first or last or any particular document).

Interestingly, this problem does not happen when I have only a single
MyIndexer, or if I have a pool of MyIndexers and am running in single core
mode.

I've looked at the other posts from users getting this exception but it
always seemed to be a different case, such as the server having crashed
previously and a lock file being left over.

--
View this message in context: http://www.nabble.com/SolrException---Lock-obtain-timed-out%2C-no-leftover-locks-tp24393255p24393255.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrException - Lock obtain timed out, no leftover locks

Posted by danben <da...@gmail.com>.

Sorry, I thought I had removed this posting.  I am running Solr over HTTP,
but (as you surmised) I had a concurrency bug.  Thanks for the response.

Dan


hossman wrote:
> 
> 
> My only guess here is that you are using SolrJ in an embedded sense, not 
> via HTTP, and something about the code you have in your MyIndexers class 
> causes two differnet threads to attempt to create two differnet cores (or 
> perhaps the same core) using identical data directories at the same time.
> 
> either that: or maybe there is a bug in the CoreAdmin functionality for 
> creating/opening a new core resulting from improper synchronization.
> 
> it would help to have the full stack trace of hte Lock timed out 
> exception, and to know more details about how exactly your code goes about 
> creating new cores on the fly.
> 
> : I'm running Solr 1.3.0 in multicore mode and feeding it data from which
> the
> : core name is inferred from a specific field.  My service extracts the
> core
> : name and, if it has not seen it before, issues a create request for that
> : core before attempting to add the document (via SolrJ).  I have a pool
> of
> : MyIndexers that run in parallel, taking documents from a queue and
> adding
> : them via the add method on the SolrServer instance corresponding to that
> : core (exactly one per core exists).  Each core is in a separate data
> : directory.  My timeouts are set as such:
> : 
> : <writeLockTimeout>15000</writeLockTimeout>
> : <commitLockTimeout>25000</commitLockTimeout>
> : 
> : I remove the index directories, start the server, check that no locks
> exist,
> : and generate ~500 documents spread across 5 cores for the MyIndexers to
> : handle.  Each time, I see one or more exceptions with a message like 
> : 
> :
> Lock_obtain_timed_out_SimpleFSLockmulticoreNewUser3dataindexlucenebd4994617386d14e2c8c29e23bcca719writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_...
> : 
> : When the indexers have completed, no lock is left over.  There is no
> : discernible pattern as far as when the exception occurs (ie, it does not
> : tend to happen on the first or last or any particular document).
> : 
> : Interestingly, this problem does not happen when I have only a single
> : MyIndexer, or if I have a pool of MyIndexers and am running in single
> core
> : mode.  
> : 
> : I've looked at the other posts from users getting this exception but it
> : always seemed to be a different case, such as the server having crashed
> : previously and a lock file being left over.
> : 
> : -- 
> : View this message in context:
> http://www.nabble.com/SolrException---Lock-obtain-timed-out%2C-no-leftover-locks-tp24393255p24393255.html
> : Sent from the Solr - User mailing list archive at Nabble.com.
> : 
> 
> 
> 
> -Hoss
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/SolrException---Lock-obtain-timed-out%2C-no-leftover-locks-tp24393255p24616034.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SolrException - Lock obtain timed out, no leftover locks

Posted by Chris Hostetter <ho...@fucit.org>.

My only guess here is that you are using SolrJ in an embedded sense, not
via HTTP, and something about the code you have in your MyIndexers class
causes two differnet threads to attempt to create two differnet cores (or
perhaps the same core) using identical data directories at the same time.

either that: or maybe there is a bug in the CoreAdmin functionality for
creating/opening a new core resulting from improper synchronization.

it would help to have the full stack trace of hte Lock timed out
exception, and to know more details about how exactly your code goes about
creating new cores on the fly.

: I'm running Solr 1.3.0 in multicore mode and feeding it data from which the
: core name is inferred from a specific field. My service extracts the core
: name and, if it has not seen it before, issues a create request for that
: core before attempting to add the document (via SolrJ). I have a pool of
: MyIndexers that run in parallel, taking documents from a queue and adding
: them via the add method on the SolrServer instance corresponding to that
: core (exactly one per core exists). Each core is in a separate data
: directory. My timeouts are set as such:
:
: <writeLockTimeout>15000</writeLockTimeout>
: <commitLockTimeout>25000</commitLockTimeout>
:
: I remove the index directories, start the server, check that no locks exist,
: and generate ~500 documents spread across 5 cores for the MyIndexers to
: handle. Each time, I see one or more exceptions with a message like
:
: Lock_obtain_timed_out_SimpleFSLockmulticoreNewUser3dataindexlucenebd4994617386d14e2c8c29e23bcca719writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_...
:
: When the indexers have completed, no lock is left over. There is no
: discernible pattern as far as when the exception occurs (ie, it does not
: tend to happen on the first or last or any particular document).
:
: Interestingly, this problem does not happen when I have only a single
: MyIndexer, or if I have a pool of MyIndexers and am running in single core
: mode.
:
: I've looked at the other posts from users getting this exception but it
: always seemed to be a different case, such as the server having crashed
: previously and a lock file being left over.
:
: --
: View this message in context: http://www.nabble.com/SolrException---Lock-obtain-timed-out%2C-no-leftover-locks-tp24393255p24393255.html
: Sent from the Solr - User mailing list archive at Nabble.com.
:

-Hoss