You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2010/04/28 21:31:49 UTC
[jira] Created: (LUCENE-2419) Improve parallel tests
Improve parallel tests
----------------------
Key: LUCENE-2419
URL: https://issues.apache.org/jira/browse/LUCENE-2419
Project: Lucene - Java
Issue Type: Improvement
Components: Build
Affects Versions: 3.1
Reporter: Robert Muir
Assignee: Robert Muir
As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
(I emailed the blog author and received permission to use this code)
This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2419) Improve parallel tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2419:
--------------------------------
Fix Version/s: 3.1
4.0
> Improve parallel tests
> ----------------------
>
> Key: LUCENE-2419
> URL: https://issues.apache.org/jira/browse/LUCENE-2419
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2419.patch
>
>
> As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
> It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
> Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
> Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
> (I emailed the blog author and received permission to use this code)
> This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2419) Improve parallel tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-2419.
---------------------------------
Resolution: Fixed
backported to 3.x, rev 941677
> Improve parallel tests
> ----------------------
>
> Key: LUCENE-2419
> URL: https://issues.apache.org/jira/browse/LUCENE-2419
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2419.patch
>
>
> As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
> It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
> Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
> Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
> (I emailed the blog author and received permission to use this code)
> This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2419) Improve parallel tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12861960#action_12861960 ]
Robert Muir commented on LUCENE-2419:
-------------------------------------
I plan to commit soon.
While there might be some issue with the NativeFSLockFactory, I cannot reproduce it, and hopefully this will create less test locks in the meantime... besides it speeds up tests.
feel free to reopen if you have problems.
> Improve parallel tests
> ----------------------
>
> Key: LUCENE-2419
> URL: https://issues.apache.org/jira/browse/LUCENE-2419
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1
> Reporter: Robert Muir
> Assignee: Robert Muir
> Attachments: LUCENE-2419.patch
>
>
> As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
> It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
> Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
> Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
> (I emailed the blog author and received permission to use this code)
> This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2419) Improve parallel tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-2419.
---------------------------------
Resolution: Fixed
Committed revision 939111.
> Improve parallel tests
> ----------------------
>
> Key: LUCENE-2419
> URL: https://issues.apache.org/jira/browse/LUCENE-2419
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1
> Reporter: Robert Muir
> Assignee: Robert Muir
> Attachments: LUCENE-2419.patch
>
>
> As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
> It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
> Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
> Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
> (I emailed the blog author and received permission to use this code)
> This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-2419) Improve parallel tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir reopened LUCENE-2419:
---------------------------------
> Improve parallel tests
> ----------------------
>
> Key: LUCENE-2419
> URL: https://issues.apache.org/jira/browse/LUCENE-2419
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2419.patch
>
>
> As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
> It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
> Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
> Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
> (I emailed the blog author and received permission to use this code)
> This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2419) Improve parallel tests
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2419:
--------------------------------
Attachment: LUCENE-2419.patch
attached is a patch, all tests pass.
Additionally i fixed the problem that backwards tests did not respect the threadsPerProcessor variable.
> Improve parallel tests
> ----------------------
>
> Key: LUCENE-2419
> URL: https://issues.apache.org/jira/browse/LUCENE-2419
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Build
> Affects Versions: 3.1
> Reporter: Robert Muir
> Assignee: Robert Muir
> Attachments: LUCENE-2419.patch
>
>
> As mentioned on the dev@ mailing list here: http://www.lucidimagination.com/search/document/93432a677917b9bd/lucenejunitresultformatter_sometimes_fails_to_lock
> It would be useful to not create a lockfactory for each test suite (As they are run sequentially in the same separate JVM).
> Additionally, we create a lot of JVMs (26) for each batch, because we have to run one for each letter.
> Instead, we use a technique here to divide up the tests with a custom selector: http://blog.code-cop.org/2009/09/parallel-junit.html
> (I emailed the blog author and received permission to use this code)
> This gives a nice boost to the speed of overall tests, especially Solr tests, as many start with an "S", but this is no longer a problem.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org