You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Michael McCandless (JIRA)" <ji...@apache.org> on 2009/10/07 11:39:31 UTC
[jira] Created: (LUCENE-1950) Remove autoCommit from IndexWriter
Remove autoCommit from IndexWriter
----------------------------------
Key: LUCENE-1950
URL: https://issues.apache.org/jira/browse/LUCENE-1950
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Reporter: Michael McCandless
Assignee: Michael McCandless
Priority: Minor
Fix For: 3.0
IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: Fwd: [jira] Created: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by Mark Miller <ma...@gmail.com>.
Me more than you. I don't know if its ext4 or 2.6.31+ext4, but commit is
slow on a my machine - making autocommit ridiculously slow.
Check out the SQLite inserts bench - the Lucene fsync stuff appears to
be similar:
http://www.phoronix.com/scan.php?page=article&item=freebsd8_ubuntu910&num=7
Michael McCandless wrote:
> Just want to confirm: is anyone (Uwe?) already working on this? If
> not I'm quite happy to take it. Been looking forward to this for a
> loooong time :)
>
> Mike
>
> ---------- Forwarded message ----------
> From: Michael McCandless (JIRA) <ji...@apache.org>
> Date: Wed, Oct 7, 2009 at 5:39 AM
> Subject: [jira] Created: (LUCENE-1950) Remove autoCommit from IndexWriter
> To: java-dev@lucene.apache.org
>
>
> Remove autoCommit from IndexWriter
> ----------------------------------
>
> Key: LUCENE-1950
> URL: https://issues.apache.org/jira/browse/LUCENE-1950
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.0
>
>
> IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
>
--
- Mark
http://www.lucidimagination.com
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Fwd: [jira] Created: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by Michael McCandless <lu...@mikemccandless.com>.
Just want to confirm: is anyone (Uwe?) already working on this? If
not I'm quite happy to take it. Been looking forward to this for a
loooong time :)
Mike
---------- Forwarded message ----------
From: Michael McCandless (JIRA) <ji...@apache.org>
Date: Wed, Oct 7, 2009 at 5:39 AM
Subject: [jira] Created: (LUCENE-1950) Remove autoCommit from IndexWriter
To: java-dev@lucene.apache.org
Remove autoCommit from IndexWriter
----------------------------------
Key: LUCENE-1950
URL: https://issues.apache.org/jira/browse/LUCENE-1950
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Reporter: Michael McCandless
Assignee: Michael McCandless
Priority: Minor
Fix For: 3.0
IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763701#action_12763701 ]
Michael McCandless commented on LUCENE-1950:
--------------------------------------------
OK I committed phase 1.
> Remove autoCommit from IndexWriter
> ----------------------------------
>
> Key: LUCENE-1950
> URL: https://issues.apache.org/jira/browse/LUCENE-1950
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-1950.patch
>
>
> IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12770290#action_12770290 ]
Uwe Schindler commented on LUCENE-1950:
---------------------------------------
Can we close this, Mike? It seems to be committed.
> Remove autoCommit from IndexWriter
> ----------------------------------
>
> Key: LUCENE-1950
> URL: https://issues.apache.org/jira/browse/LUCENE-1950
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-1950.patch, LUCENE-1950.patch
>
>
> IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless resolved LUCENE-1950.
----------------------------------------
Resolution: Fixed
Woops, yes this is done.
> Remove autoCommit from IndexWriter
> ----------------------------------
>
> Key: LUCENE-1950
> URL: https://issues.apache.org/jira/browse/LUCENE-1950
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-1950.patch, LUCENE-1950.patch
>
>
> IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1950:
---------------------------------------
Attachment: LUCENE-1950.patch
Attached patch. All tests pass. This is just the first step, which
is to remove autoCommit from IW's public APIs, and fix all places that
were using those APIs.
Second step is to remove autoCommit from inside IW.
When there were tests testing both aC=false and true, I just remove
the "true" case. When the test was only testing aC=true, I switched
it to false, and sometimes had to tweak the test (eg to insert calls
to IW.commit) to make it happy. For one test, when I did this
(TestIndexWriter.testImmediateDiskFullWithThreads) it uncovered a
latent bug in IW where if an IOException is hit on building the cfx
file for flushed doc stores, we were missing a call to
docWriter.abort().
I also removed autoCommit from the alg files in
contrib/benchmark/conf, fixed up the javadocs, and removed some unused
imports.
I plan to commit soon!
> Remove autoCommit from IndexWriter
> ----------------------------------
>
> Key: LUCENE-1950
> URL: https://issues.apache.org/jira/browse/LUCENE-1950
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-1950.patch
>
>
> IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1950) Remove autoCommit from IndexWriter
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless updated LUCENE-1950:
---------------------------------------
Attachment: LUCENE-1950.patch
Patch to remove autoCommit inside IW. Will commit shortly.
> Remove autoCommit from IndexWriter
> ----------------------------------
>
> Key: LUCENE-1950
> URL: https://issues.apache.org/jira/browse/LUCENE-1950
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Reporter: Michael McCandless
> Assignee: Michael McCandless
> Priority: Minor
> Fix For: 3.0
>
> Attachments: LUCENE-1950.patch, LUCENE-1950.patch
>
>
> IndexWriter's autoCommit is deprecated; in 3.0 it will be hardwired to false.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org