You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Jayson Minard (JIRA)" <ji...@apache.org> on 2009/05/08 23:10:45 UTC
[jira] Created: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
-------------------------------------------------------------------------
Key: SOLR-1155
URL: https://issues.apache.org/jira/browse/SOLR-1155
Project: Solr
Issue Type: Improvement
Components: search
Affects Versions: 1.3
Reporter: Jayson Minard
Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by Ryan McKinley <ry...@gmail.com>.
Seems ok now...
On Fri, May 29, 2009 at 7:47 PM, Mike Klaas <mi...@gmail.com> wrote:
> I'd like to take a look at this but JIRA seems to be down. Is anyone else
> experiencing this?
>
> -Mike
>
>
> On Wed, May 13, 2009 at 7:41 AM, Jayson Minard (JIRA) <ji...@apache.org>wrote:
>
>>
>> [
>> https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>>
>> Jayson Minard updated SOLR-1155:
>> --------------------------------
>>
>> Attachment: Solr-1155.patch
>>
>> Resolve TODO for commitWithin, and updated AutoCommitTrackerTest to
>> validate the fix.
>>
>> > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
>> > -------------------------------------------------------------------------
>> >
>> > Key: SOLR-1155
>> > URL: https://issues.apache.org/jira/browse/SOLR-1155
>> > Project: Solr
>> > Issue Type: Improvement
>> > Components: search
>> > Affects Versions: 1.3
>> > Reporter: Jayson Minard
>> > Attachments: Solr-1155.patch, Solr-1155.patch
>> >
>> >
>> > Currently DirectUpdateHandler2 will block adds during a commit, and it
>> seems to be possible with recent changes to Lucene to allow them to run
>> concurrently.
>> > See:
>> http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> You can reply to this email to add a comment to the issue online.
>>
>>
>
Re: [jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by Mike Klaas <mi...@gmail.com>.
I'd like to take a look at this but JIRA seems to be down. Is anyone else
experiencing this?
-Mike
On Wed, May 13, 2009 at 7:41 AM, Jayson Minard (JIRA) <ji...@apache.org>wrote:
>
> [
> https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Jayson Minard updated SOLR-1155:
> --------------------------------
>
> Attachment: Solr-1155.patch
>
> Resolve TODO for commitWithin, and updated AutoCommitTrackerTest to
> validate the fix.
>
> > Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> > -------------------------------------------------------------------------
> >
> > Key: SOLR-1155
> > URL: https://issues.apache.org/jira/browse/SOLR-1155
> > Project: Solr
> > Issue Type: Improvement
> > Components: search
> > Affects Versions: 1.3
> > Reporter: Jayson Minard
> > Attachments: Solr-1155.patch, Solr-1155.patch
> >
> >
> > Currently DirectUpdateHandler2 will block adds during a commit, and it
> seems to be possible with recent changes to Lucene to allow them to run
> concurrently.
> > See:
> http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Updated unit test to compile (was missing method change from previous patch that changed an interface it implemented)
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Knocked down a few TODO and cleaned up resetting of counts.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707572#action_12707572 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
DirectUpdateHandlerWithAutoCommit.java
DirectUpdateHandler3.java
Updated file, and associated interface used to help DirectUpdateHandler2 and 3 co-exist with tests, SnapPuller and ReplicationHandler classes.
A larger patch with all changes to tests, config files, and other items makeing DirectUpdateHandler3 the default (purely for running all of the unit tests using it) is attached as well.
Patch includes fix for SOLR-1157 which got in the way of running unit tests.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12708795#action_12708795 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
TODO: fix commitWithin, it isn't respected by the DirectUpdateHandler3
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: (was: SOLR-1155.patch)
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707602#action_12707602 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
Preparing a first patch, calling it DirectUpdateHandler3 as there is a lot of reshuffling and a new commit tracker implementation that is lighter-weight.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: DirectUpdateHandler3.java
I created DirectUpdateHandler3 as a copy of DirectUpdateHandler2 and then did a pass across it rebuilding the locking, stats, and tracker. I am basically at the "IDE show no errors" stage and going to read it a few more times, and start testing. Feedback is welcome as I'm in the dark on a few areas of this code and I am working from a lot of assumptions. Some event placeholder methods remain while I think through the issues, will trim it back when done.
Events to the auto commit tracker are within the locks to help avoid cases where we set the counts to 0 after other work has concurrently been done. The stats information has similar concerns and is not yet dealt with, so some stats might reset to 0 on commit when there was further work being done (so they should not go to 0).
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: DirectUpdateHandler3.java
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: (was: DirectUpdateHandlerWithAutoCommit.java)
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-1155:
----------------------------------------
Fix Version/s: 1.5
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3, 1.4
> Reporter: Jayson Minard
> Fix For: 1.5
>
> Attachments: SOLR-1155-release1.4-rev834789.patch, SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707701#action_12707701 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
Does anyone know if the UpdateHandler.close() method is called when everything else is shut down and no transactions can come through? If so, I can remove the blocking added to prevent a writer from re-opening after close is called.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707636#action_12707636 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
All tests pass with the attached patch file and DirectUpdateHandler3 as the handler for each solrconfig variant in the test-files directory. Needs review for locking holes and some of the TODO comments answered still. And needs a lot of concurrent update testing.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: DirectUpdateHandler3.java
Updated with different locking around the post commit/optimize handlers. There is a gap in time between commit and these calls, is that safe? left TODO's in the code at places of concern.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
New patch resolves the serialization of commit/optimize/close/rollback actions that should not be concurrent with each other. Also protect methods after close to keep other threads from accidentally re-opening writer.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: (was: SOLR-1155.patch)
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
minor change to rollback to only drop and re-open writer if there was one to begin wtih.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: (was: DirectUpdateHandler3.java)
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707692#action_12707692 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
new TODO...
Right now auto commit will eventually fire off a commit that might run into a manual commit or optimize and sit at the syncrhonization block, then fire off on the tail of the other action. I would rather it wait until those running commands clear and THEN evaluate if it needs to do a commit or not since its information it used to make the decision could be moot due to the other action that beat it in. So if a commit or optimize is running it will not evaluate for auto commit until they complete.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: Solr-1155.patch
Resolve TODO for commitWithin, and updated AutoCommitTrackerTest to validate the fix.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1155.patch, Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
move wait for searcher out of locks in optimize and commit
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Refactored the AutoCommitTracker out to separate top-level class rather than inner class. Enhanced the interface used by unit tests and other classes to talk to the update handler so make this work. Interface is not final, just a temp answer to the decoupling.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Bug fixing, and added a new test that test just the AutoCommitTracker in isolation rather than through the DirectUpdateHandlerX
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707600#action_12707600 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
Thinking about pending counts as well. If adds are ongoing during a commit, there has to be a point in time where the counters clear the number of documents committed. Currently they are set to 0 which is safe since nothing was ongoing while the commit happened. Something to remember...
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Fixed some of the issue mentioned in the comment above. Split commit and optimize work out since optimize has a stronger need to lock and be consistent.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155-release1.4-rev834789.patch
Added patch for release version of 1.4 (but this is most likely identical to the last trunk patch anyway)
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3, 1.4
> Reporter: Jayson Minard
> Attachments: SOLR-1155-release1.4-rev834789.patch, SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: Solr-1155.patch
Cleaning the patch so that RequestHandlerBase only has the required change and not the accidental reformatting.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: Solr-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Change openWriter to do a check of null writer before locking to avoid artificial contention between threads due to ensuring a writer is open.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707672#action_12707672 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
Things likely needing further work:
* serialize commits? (two shouldn't happen at same time)
* optimize should us iwManageWriter lock from optimize through call-back call, so break it away from commit which only needs iwUseWriter?) This also takes care of the contract that optimize callback is still on an optimized index.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandler3.java, DirectUpdateHandlerWithAutoCommit.java, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-1155) Change
DirectUpdateHandler2 to allow concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707572#action_12707572 ]
Jayson Minard edited comment on SOLR-1155 at 5/8/09 7:52 PM:
-------------------------------------------------------------
This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking at least for deletes.
was (Author: jminard):
This will involve a change to locking as seen in the thread mentioned above between add/delete/commit methods, but there are also synchronized methods on the CommitTracker scehduleCommitWithin method, run method, and getCommitCount method that will cause similar blocking.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12707575#action_12707575 ]
Jayson Minard commented on SOLR-1155:
-------------------------------------
Yonik mentioned: "We'd need to think about what type of synchronization may be needed
for postCommit and postOptimize hooks too."
Is the expectation for those callbacks that nothing will change until the callback completes? (i.e. a script might run a process that scans the index after optimize expecting it to remain optimized during the script execution) Or is that not the contract of those callbacks?
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Affects Version/s: 1.4
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3, 1.4
> Reporter: Jayson Minard
> Attachments: SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155-trunk-rev834706.patch
Updated patch to trunk rev 834706 and up to date with current DirectUpdateHandler2 functionality
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3, 1.4
> Reporter: Jayson Minard
> Attachments: SOLR-1155-trunk-rev834706.patch, Solr-1155.patch, Solr-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-1155) Change DirectUpdateHandler2 to allow
concurrent adds during an autocommit
Posted by "Jayson Minard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-1155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayson Minard updated SOLR-1155:
--------------------------------
Attachment: SOLR-1155.patch
Changed external commit interface to not have locking, but rather query if a commit/rollback/optimize is already running to accomplish the same goal. Also ensure no locking around waiting for searcher after commit or optimize.
> Change DirectUpdateHandler2 to allow concurrent adds during an autocommit
> -------------------------------------------------------------------------
>
> Key: SOLR-1155
> URL: https://issues.apache.org/jira/browse/SOLR-1155
> Project: Solr
> Issue Type: Improvement
> Components: search
> Affects Versions: 1.3
> Reporter: Jayson Minard
> Attachments: SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch, SOLR-1155.patch
>
>
> Currently DirectUpdateHandler2 will block adds during a commit, and it seems to be possible with recent changes to Lucene to allow them to run concurrently.
> See: http://www.nabble.com/Autocommit-blocking-adds---AutoCommit-Speedup--td23435224.html
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.