You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shai Erera (JIRA)" <ji...@apache.org> on 2010/04/27 12:36:31 UTC
[jira] Created: (LUCENE-2416) Some improvements to Benchmark
Some improvements to Benchmark
------------------------------
Key: LUCENE-2416
URL: https://issues.apache.org/jira/browse/LUCENE-2416
Project: Lucene - Java
Issue Type: Improvement
Components: contrib/benchmark
Reporter: Shai Erera
Assignee: Shai Erera
Priority: Minor
Fix For: 3.1
I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-2416) Some improvements to Benchmark
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera reopened LUCENE-2416:
--------------------------------
Backport to 3.1
> Some improvements to Benchmark
> ------------------------------
>
> Key: LUCENE-2416
> URL: https://issues.apache.org/jira/browse/LUCENE-2416
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 4.0
>
> Attachments: LUCENE-2416.patch, LUCENE-2416.patch
>
>
> I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
> While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2416) Some improvements to Benchmark
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera updated LUCENE-2416:
-------------------------------
Fix Version/s: 3.1
> Some improvements to Benchmark
> ------------------------------
>
> Key: LUCENE-2416
> URL: https://issues.apache.org/jira/browse/LUCENE-2416
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2416.patch, LUCENE-2416.patch
>
>
> I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
> While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2416) Some improvements to Benchmark
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera updated LUCENE-2416:
-------------------------------
Attachment: LUCENE-2416.patch
LongToEnglishContentSource did not synchronize properly - fixed !
On the way I've noticed it only outputs negative numbers, and starts from the odd value Long.MIN_VAL + 10 (why the "+10"?). Unless someone creates THAT MANY documents, you'll never get positive numbers w/ that content source.
> Some improvements to Benchmark
> ------------------------------
>
> Key: LUCENE-2416
> URL: https://issues.apache.org/jira/browse/LUCENE-2416
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2416.patch, LUCENE-2416.patch
>
>
> I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
> While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2416) Some improvements to Benchmark
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera resolved LUCENE-2416.
--------------------------------
Resolution: Fixed
Committed revision 941396.
> Some improvements to Benchmark
> ------------------------------
>
> Key: LUCENE-2416
> URL: https://issues.apache.org/jira/browse/LUCENE-2416
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 3.1, 4.0
>
> Attachments: LUCENE-2416.patch, LUCENE-2416.patch
>
>
> I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
> While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2416) Some improvements to Benchmark
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera resolved LUCENE-2416.
--------------------------------
Resolution: Fixed
Committed revision 938582.
> Some improvements to Benchmark
> ------------------------------
>
> Key: LUCENE-2416
> URL: https://issues.apache.org/jira/browse/LUCENE-2416
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2416.patch, LUCENE-2416.patch
>
>
> I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
> While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2416) Some improvements to Benchmark
Posted by "Shai Erera (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shai Erera updated LUCENE-2416:
-------------------------------
Attachment: LUCENE-2416.patch
* Removed unrelated jdoc in DirContentSource
* LongToEnglishContentSource - added jdoc, improved synchronization, added ASF license
* WriteLineDocTask is now threadsafe - uses PrintWriter and a ThreadLocal for threads buffer
* Added relevant test to WriteLineDocTaskTest
* Deleted Format.simpleName(Class) and replaced by class.getSimpleName()
* Handled 3.0 TODOs:
** Renamed StringBufferReader to StringBuilderReader
** Changed to use StringBuilder in TrecContentSource
In order to apply the patch you need to first:
svn mv lucene/contrib/benchmark/src/java/org/apache/lucene/benchmark/byTask/utils/StringBufferReader.java lucene/contrib/benchmark/src/java/org/apache/lucene/benchmark/byTask/utils/StringBuilderReader.java
All benchmark tests pass. I plan to commit shortly.
> Some improvements to Benchmark
> ------------------------------
>
> Key: LUCENE-2416
> URL: https://issues.apache.org/jira/browse/LUCENE-2416
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/benchmark
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2416.patch
>
>
> I've noticed that WriteLineDocTask declares it does not support multi-threading, but taking a closer look I think this is really for no good reason. Most of the work is done by reading from the ContentSource and constructing the document. If those two are mult-threaded (and I think all ContentSources are), then we can synchronize only around writing the actual document to the line file.
> While investigating that, I've noticed some 1.5 TODOs and some other minor improvements that can be made. If you've wanted to make some minor improvements to benchmark, let me know :). I intend to include only minor and trivial ones.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org