You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Zhang, Lisheng" <Li...@BroadVision.com> on 2013/05/30 01:10:52 UTC
solr 4.3: write.lock is not removed
Hi,
I recently upgraded solr from 3.6.1 to 4.3, it works well, but I noticed that after finishing
indexing
write.lock
is NOT removed. Later if I index again it still works OK. Only after I shutdown Tomcat
then write.lock is removed. This behavior caused some problem like I could not use luke
to observe indexed data.
I did not see any error/warning messages.
Is this the designed behavior? Can I have the old behavior (after commit write.lock is
removed) through configuration?
Thanks very much for helps, Lisheng
RE: solr 4.3: write.lock is not removed
Posted by "Zhang, Lisheng" <Li...@BroadVision.com>.
I did more test and it seems that this is still a bug (previous issue 3/):
1/ Create a core by CURL command with dataDir=<some_folder>, core is created OK
and later indexing worked OK also.
2/ But in solr.xml, dadaDir is not defined in element "<core ...".
3/ After restart solr, dataDir information is lost and solr issued WARN.
4/ If I manually add dataDir attribute into core element in solr.xml after core
is created, restarting solr will be fine.
Thanks very much for helps, Lisheng
-----Original Message-----
From: Zhang, Lisheng
Sent: Thursday, May 30, 2013 11:22 AM
To: 'solr-user@lucene.apache.org'
Subject: RE: solr 4.3: write.lock is not removed
I did more tests and get more info: the basic setting is that we created core from PHP CURl
API where we define:
schema
config
instanceDir=<my_solr_home>
dataDir=<my_solr_home>/data/new_collection_name
In solr 3.6.1 we donot need to define schema/config because conf folder is not inside each
collection.
1/ Indexing works OK but write.lock is not removed (we use "/update?commit=true..")
2/ Shutdown tomcat, I saw write.lock is gone
3/ Restart Tomcat, indexed data was created at the instanceDir/data level, with some warning
messages. It seems that in solr.xml, dataDir is not defined?
Thanks very much for helps, Lisheng
-----Original Message-----
From: Zhang, Lisheng [mailto:Lisheng.Zhang@broadvision.com]
Sent: Thursday, May 30, 2013 10:57 AM
To: solr-user@lucene.apache.org
Subject: RE: solr 4.3: write.lock is not removed
Hi,
We just use CURL from PHP code to submit indexing request, like:
/update?commit=true..
This worked well in solr 3.6.1. I saw the link you showed and really appreciate
(if no other choice I will change java source code but hope there is a better
way..)?
Thanks very much for helps, Lisheng
-----Original Message-----
From: bbarani [mailto:bbarani@gmail.com]
Sent: Thursday, May 30, 2013 9:45 AM
To: solr-user@lucene.apache.org
Subject: Re: solr 4.3: write.lock is not removed
How are you indexing the documents? Are you using indexing program?
The below post discusses the same issue..
http://lucene.472066.n3.nabble.com/removing-write-lock-file-in-solr-after-indexing-td3699356.html
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-4-3-write-lock-is-not-removed-tp4066908p4067101.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.3: write.lock is not removed
Posted by "Zhang, Lisheng" <Li...@BroadVision.com>.
I did more tests and get more info: the basic setting is that we created core from PHP CURl
API where we define:
schema
config
instanceDir=<my_solr_home>
dataDir=<my_solr_home>/data/new_collection_name
In solr 3.6.1 we donot need to define schema/config because conf folder is not inside each
collection.
1/ Indexing works OK but write.lock is not removed (we use "/update?commit=true..")
2/ Shutdown tomcat, I saw write.lock is gone
3/ Restart Tomcat, indexed data was created at the instanceDir/data level, with some warning
messages. It seems that in solr.xml, dataDir is not defined?
Thanks very much for helps, Lisheng
-----Original Message-----
From: Zhang, Lisheng [mailto:Lisheng.Zhang@broadvision.com]
Sent: Thursday, May 30, 2013 10:57 AM
To: solr-user@lucene.apache.org
Subject: RE: solr 4.3: write.lock is not removed
Hi,
We just use CURL from PHP code to submit indexing request, like:
/update?commit=true..
This worked well in solr 3.6.1. I saw the link you showed and really appreciate
(if no other choice I will change java source code but hope there is a better
way..)?
Thanks very much for helps, Lisheng
-----Original Message-----
From: bbarani [mailto:bbarani@gmail.com]
Sent: Thursday, May 30, 2013 9:45 AM
To: solr-user@lucene.apache.org
Subject: Re: solr 4.3: write.lock is not removed
How are you indexing the documents? Are you using indexing program?
The below post discusses the same issue..
http://lucene.472066.n3.nabble.com/removing-write-lock-file-in-solr-after-indexing-td3699356.html
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-4-3-write-lock-is-not-removed-tp4066908p4067101.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.3: write.lock is not removed
Posted by "Zhang, Lisheng" <Li...@BroadVision.com>.
Hi,
We just use CURL from PHP code to submit indexing request, like:
/update?commit=true..
This worked well in solr 3.6.1. I saw the link you showed and really appreciate
(if no other choice I will change java source code but hope there is a better
way..)?
Thanks very much for helps, Lisheng
-----Original Message-----
From: bbarani [mailto:bbarani@gmail.com]
Sent: Thursday, May 30, 2013 9:45 AM
To: solr-user@lucene.apache.org
Subject: Re: solr 4.3: write.lock is not removed
How are you indexing the documents? Are you using indexing program?
The below post discusses the same issue..
http://lucene.472066.n3.nabble.com/removing-write-lock-file-in-solr-after-indexing-td3699356.html
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-4-3-write-lock-is-not-removed-tp4066908p4067101.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: solr 4.3: write.lock is not removed
Posted by bbarani <bb...@gmail.com>.
How are you indexing the documents? Are you using indexing program?
The below post discusses the same issue..
http://lucene.472066.n3.nabble.com/removing-write-lock-file-in-solr-after-indexing-td3699356.html
--
View this message in context: http://lucene.472066.n3.nabble.com/solr-4-3-write-lock-is-not-removed-tp4066908p4067101.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: solr 4.3: write.lock is not removed
Posted by "Zhang, Lisheng" <Li...@BroadVision.com>.
Hi,
Thanks very much for the explanation! Could we config to get to old behavior?
I asked this option because our app has many small cores so that we prefer
create/close writer on the fly (otherwise we may have memory issue quickly).
We also do not need NRT for now.
Thanks very much for helps, Lisheng
-----Original Message-----
From: Chris Hostetter [mailto:hossman_lucene@fucit.org]
Sent: Thursday, May 30, 2013 11:35 AM
To: solr-user@lucene.apache.org
Subject: Re: solr 4.3: write.lock is not removed
: I recently upgraded solr from 3.6.1 to 4.3, it works well, but I noticed that after finishing
: indexing
:
: write.lock
:
: is NOT removed. Later if I index again it still works OK. Only after I shutdown Tomcat
: then write.lock is removed. This behavior caused some problem like I could not use luke
: to observe indexed data.
IIRC, This was an intentional change. In older versions of Solr the
IndexWRiter was only opened if/when updates needed to be made, but that
made it impossible to safely take advantage of some internal optimizations
related to NRT IndexReader reloading, so the logic was modified to always
keep the IndexWriter open as lon as the SolrCore is loaded.
In general, your past behavior of pointing luke at a live solr index could
have also produced problems if updates came into solr while luke had the
write lock active.
-Hoss
Re: solr 4.3: write.lock is not removed
Posted by Chris Hostetter <ho...@fucit.org>.
: I recently upgraded solr from 3.6.1 to 4.3, it works well, but I noticed that after finishing
: indexing
:
: write.lock
:
: is NOT removed. Later if I index again it still works OK. Only after I shutdown Tomcat
: then write.lock is removed. This behavior caused some problem like I could not use luke
: to observe indexed data.
IIRC, This was an intentional change. In older versions of Solr the
IndexWRiter was only opened if/when updates needed to be made, but that
made it impossible to safely take advantage of some internal optimizations
related to NRT IndexReader reloading, so the logic was modified to always
keep the IndexWriter open as lon as the SolrCore is loaded.
In general, your past behavior of pointing luke at a live solr index could
have also produced problems if updates came into solr while luke had the
write lock active.
-Hoss