You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by wenjie zheng <we...@gmail.com> on 2006/03/05 02:36:34 UTC
Problem when deleting files from index
My program loops thru a list of email records, first searches in index for
its docid.
Once it gets the docid, it will try to remove the doc from the index.
The problem I ran into was:
4 out 10 times I ran the program, I got "Lock obtain timed out:
Lock@/tmp/lucene-6c36cae1cb524a61cdbb38a32f2ef684-write.lock" on the last
email record I tried to delete.
5 out 10 times it finished just fine.
1 out 10 times it got Lock error when there were more than one email records
left to delete (there are totally 29 docs in the index)
Here is the code:
int i = 0;
Iterator<EmailRecord> it = v.iterator(); //v is the vector contains email
records
while(it.hasNext()){
EmailRecord eml = (EmailRecord)it.next();
// local search function to get the docId, searcher and Directory are closed
after search is done
int docid = lSearcher.search(eml);
// if doc not found, change the status updator
if(docid == SystemConfig.ERROR_DOC_NOT_FOUND){
// update database entry
updator.setStatus(i, SystemConfig.ERROR_DOC_NOT_FOUND);
}else{
Directory indexDir = null;
IndexModifier modifier = null;
try{
indexDir = FSDirectory.getDirectory(index, false);
modifier = new IndexModifier(indexDir, new StandardAnalyzer(), false);
modifier.deleteDocument(docid);
modifier.flush(); // make sure all the changes pushed to the disk
modifier.close();
indexDir.close();
}catch(IOException e){
logger.error("Cannot delete " + docid + " th doc from the index: " +
eml.getUsername() + " " + e.getMessage());
updator.setStatus(i, SystemConfig.ERROR_CANNOT_DELETE);
}
// try to sleep between two deletions
try{
Thread.sleep(100);
}catch(InterruptedException e){
logger.error("Cannot sleep after one deletion... " + e.getMessage());
}
}
i++;
}
Thanks
Wenjie
Re: Problem when deleting files from index
Posted by wenjie zheng <we...@gmail.com>.
Thanks for the hint.
It's my bad! My code is supposed to create a single thread for each user to
delete all his email records from the user's index.
But there was a bug in my code causing it to spawn more than one threads
trying to modify the same index.
I think that's why sometimes it worked but sometimes it failed.
Thanks
Wenjie
On 3/5/06, Daniel Naber <lu...@danielnaber.de> wrote:
>
> On Sonntag 05 März 2006 02:36, wenjie zheng wrote:
>
> > a list of email records, first searches in index for
> > its docid.
> > Once it gets the docid, it will try to remove the doc from the index.
> >
> > The problem I ran into was:
> > 4 out 10 times I ran the program, I got "Lock obtain timed out:
>
> Any reason you create a new IndexModifier for each item? Anyway, please
> post a bug report (http://issues.apache.org/jira/browse/LUCENE) and attach
> a self-contained test case so I can try to reproduce this.
>
> Regards
> Daniel
>
> --
> http://www.danielnaber.de
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
Re: Problem when deleting files from index
Posted by Daniel Naber <lu...@danielnaber.de>.
On Sonntag 05 März 2006 02:36, wenjie zheng wrote:
> a list of email records, first searches in index for
> its docid.
> Once it gets the docid, it will try to remove the doc from the index.
>
> The problem I ran into was:
> 4 out 10 times I ran the program, I got "Lock obtain timed out:
Any reason you create a new IndexModifier for each item? Anyway, please
post a bug report (http://issues.apache.org/jira/browse/LUCENE) and attach
a self-contained test case so I can try to reproduce this.
Regards
Daniel
--
http://www.danielnaber.de
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: refetching interval
Posted by Grant Ingersoll <gs...@syr.edu>.
Try the Nutch mailing list
Michael Ji wrote:
> Hi,
>
> I am using nutch 07 and found the following code in
> FetchListTool.java
>
> private static final long FETCH_GENERATION_DELAY_MS =
> 7 * 24 * 60 * 60 * 1000;
>
> that means next refetching time is always 7 days
> later, no matter what fetch interval setting in
> nutch-site.xml,
>
> I feel puzzled. Could any one give me a hint?
>
> thanks,
>
> Michael,
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam? Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
>
--
Grant Ingersoll
Sr. Software Engineer
Center for Natural Language Processing
Syracuse University
School of Information Studies
335 Hinds Hall
Syracuse, NY 13244
http://www.cnlp.org
Voice: 315-443-5484
Fax: 315-443-6886
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
refetching interval
Posted by Michael Ji <fj...@yahoo.com>.
Hi,
I am using nutch 07 and found the following code in
FetchListTool.java
private static final long FETCH_GENERATION_DELAY_MS =
7 * 24 * 60 * 60 * 1000;
that means next refetching time is always 7 days
later, no matter what fetch interval setting in
nutch-site.xml,
I feel puzzled. Could any one give me a hint?
thanks,
Michael,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: compile search.jsp
Posted by Chris Hostetter <ho...@fucit.org>.
nutch-user@lucene.apache.org is the appropriate email list to consult with
questions about using/configuring/customizing nutch.
java-*@lucene.apache.org is for discussing the core lucene java library.
: Date: Sat, 4 Mar 2006 18:36:25 -0800 (PST)
: From: Michael Ji <fj...@yahoo.com>
: Reply-To: java-dev@lucene.apache.org
: To: java-dev@lucene.apache.org
: Subject: compile search.jsp
:
: Hi,
:
: I made change in search.jsp under /nutch/src/web/jsp
: and hope the change could reflect to the skin of nutch
: search page.
:
: I tried to run "ant war" and replace ROOT.war in
: tomcat/webapp
:
: also I tried to shutdown and restart tomcat;
:
: But seems the nutch search page keeps the same, also
: the bean.LOG.info keeps the same as before even I am
: writing new information.
:
: I wonder if any compiling steps I missed.
:
: thanks your help,
:
: Michael,
:
:
: __________________________________________________
: Do You Yahoo!?
: Tired of spam? Yahoo! Mail has the best spam
: protection around
: http://mail.yahoo.com
:
: __________________________________________________
: Do You Yahoo!?
: Tired of spam? Yahoo! Mail has the best spam protection around
: http://mail.yahoo.com
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-dev-help@lucene.apache.org
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
RE: compile search.jsp
Posted by Richard Braman <rb...@bramantax.com>.
To change the skin, goto TOMCAT_HOME/webapps/ROOT(this is where nutch
web should be installed if you did it right)/en/
and edit search.html, help, faq, etc. Also edit the header in the
footer in the include directory. It's pretty confusing. I had to spend
hours looking though the mailing list archives to find the answer to
this one.
-----Original Message-----
From: Michael Ji [mailto:fji_00@yahoo.com]
Sent: Saturday, March 04, 2006 10:04 PM
To: nutch-dev@lucene.apache.org
Subject: compile search.jsp
Hi,
I made change in search.jsp under /nutch/src/web/jsp
and hope the change could reflect to the skin of
nutch search page.
I tried to run "ant war" and replace ROOT.war in
tomcat/webapp
also I tried to shutdown and restart tomcat;
But seems the nutch search page keeps the same, also
the bean.LOG.info keeps the same as before even I am
writing new information.
I wonder if any compiling steps I missed.
thanks your help,
Michael,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
compile search.jsp
Posted by Michael Ji <fj...@yahoo.com>.
Hi,
I made change in search.jsp under /nutch/src/web/jsp
and hope the change could reflect to the skin of
nutch search page.
I tried to run "ant war" and replace ROOT.war in
tomcat/webapp
also I tried to shutdown and restart tomcat;
But seems the nutch search page keeps the same, also
the bean.LOG.info keeps the same as before even I am
writing new information.
I wonder if any compiling steps I missed.
thanks your help,
Michael,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
compile search.jsp
Posted by Michael Ji <fj...@yahoo.com>.
Hi,
I made change in search.jsp under /nutch/src/web/jsp
and hope the change could reflect to the skin of nutch
search page.
I tried to run "ant war" and replace ROOT.war in
tomcat/webapp
also I tried to shutdown and restart tomcat;
But seems the nutch search page keeps the same, also
the bean.LOG.info keeps the same as before even I am
writing new information.
I wonder if any compiling steps I missed.
thanks your help,
Michael,
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam
protection around
http://mail.yahoo.com
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org