You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Radim Kolar (JIRA)" <ji...@apache.org> on 2012/11/05 00:32:12 UTC
[jira] [Created] (SOLR-4030) Use Lucene segment merge throttling
Radim Kolar created SOLR-4030:
---------------------------------
Summary: Use Lucene segment merge throttling
Key: SOLR-4030
URL: https://issues.apache.org/jira/browse/SOLR-4030
Project: Solr
Issue Type: Improvement
Affects Versions: 5.0
Reporter: Radim Kolar
Attachments: solr-mergeratelimit.txt
add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505790#comment-13505790 ]
Radim Kolar commented on SOLR-4030:
-----------------------------------
it still merges into trunk cleanly. just do git pull
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller reassigned SOLR-4030:
---------------------------------
Assignee: Mark Miller
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller reassigned SOLR-4030:
---------------------------------
Assignee: (was: Mark Miller)
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt, solr-ratelimit2.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505866#comment-13505866 ]
Radim Kolar commented on SOLR-4030:
-----------------------------------
LUCENE-4537 is cause. I will rework it but its not clear if new rate limiter is used for ordinary writes (not just merges) as well.
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mark Miller updated SOLR-4030:
------------------------------
Priority: Minor (was: Major)
Affects Version/s: (was: 5.0)
Fix Version/s: 5.0
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505161#comment-13505161 ]
Mark Miller commented on SOLR-4030:
-----------------------------------
This would be nice to add - could you update this patch to 5x trunk Radmin?
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508813#comment-13508813 ]
Radim Kolar commented on SOLR-4030:
-----------------------------------
any progress on this? its trivial patch.
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt, solr-ratelimit2.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radim Kolar updated SOLR-4030:
------------------------------
Attachment: solr-ratelimit2.txt
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt, solr-ratelimit2.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505892#comment-13505892 ]
Radim Kolar commented on SOLR-4030:
-----------------------------------
added limits for reading, merging and writing.
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt, solr-ratelimit2.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radim Kolar updated SOLR-4030:
------------------------------
Attachment: solr-mergeratelimit.txt
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 5.0
> Reporter: Radim Kolar
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505855#comment-13505855 ]
Radim Kolar commented on SOLR-4030:
-----------------------------------
but it failed to build with 5.x in jenkins. i will take a more detailed look on it
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radim Kolar updated SOLR-4030:
------------------------------
Fix Version/s: 4.1
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 5.0
> Reporter: Radim Kolar
> Labels: patch
> Fix For: 4.1
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Radim Kolar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radim Kolar updated SOLR-4030:
------------------------------
Labels: patch (was: )
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Affects Versions: 5.0
> Reporter: Radim Kolar
> Labels: patch
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505840#comment-13505840 ]
Mark Miller commented on SOLR-4030:
-----------------------------------
bq. it still merges into trunk cleanly.
Sure, it applies - it does not compile.
bq. just do git pull
I'm using your patch and an svn checkout - no git to be found.
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-4030) Use Lucene segment merge throttling
Posted by "Mark Miller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505908#comment-13505908 ]
Mark Miller commented on SOLR-4030:
-----------------------------------
Looks good. I'll look at adding a unit test. I'd also rather detect when no settings have been entered (all unlimited) and not wrap the dir with a rate limiting wrapper in that case.
We also do an instance of check in 4x against fsdir - so we should make that work with this.
> Use Lucene segment merge throttling
> -----------------------------------
>
> Key: SOLR-4030
> URL: https://issues.apache.org/jira/browse/SOLR-4030
> Project: Solr
> Issue Type: Improvement
> Reporter: Radim Kolar
> Assignee: Mark Miller
> Priority: Minor
> Labels: patch
> Fix For: 4.1, 5.0
>
> Attachments: solr-mergeratelimit.txt, solr-ratelimit2.txt
>
>
> add argument "maxMergeWriteMBPerSec" to Solr directory factories.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org