You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Grant Ingersoll (JIRA)" <ji...@apache.org> on 2007/03/20 02:41:32 UTC
[jira] Created: (LUCENE-837) QueryMaker Implementations
QueryMaker Implementations
--------------------------
Key: LUCENE-837
URL: https://issues.apache.org/jira/browse/LUCENE-837
Project: Lucene - Java
Issue Type: Bug
Components: contrib/benchmark
Reporter: Grant Ingersoll
Assigned To: Grant Ingersoll
Priority: Minor
Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
Add in a new QueryMaker for reading queries from a file that is specified in the properties.
Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated LUCENE-837:
-----------------------------------
Attachment: field-selector-bench.patch
Here's my changes. Am going to commit shortly
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch, field-selector-bench.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Closed: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll closed LUCENE-837.
----------------------------------
Resolution: Fixed
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assignee: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch, field-selector-bench.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated LUCENE-837:
-----------------------------------
Summary: contrib/benchmark QueryMaker and Task Refactorings (was: QueryMaker Implementations)
The patch I have is slightly broader than just QueryMaker, as it also refactors some common code in the Search*TravTask.
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-837) contrib/benchmark QueryMaker and
Task Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483388 ]
Grant Ingersoll commented on LUCENE-837:
----------------------------------------
Committed field-selector-bench.patch on revision 521569
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch, field-selector-bench.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-837) contrib/benchmark QueryMaker and
Task Refactorings
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483413 ]
Doron Cohen commented on LUCENE-837:
------------------------------------
Hi, I like the new field selector stuff.
Few comments:
- copyright notice missing in the new ***Selector class
- The bytes storing in BasicDocMaker - perhaps better to extract the bytes in
BasicDocMaker, - just before the closing '}' of
if (docData.body!=null && docData.body.length()>0) {
just taking bdy.getBytes("UTF-8").
This way this too would respect the doc size limitation parameter.
This would actually allow all doc makers to support this, and you can undo the changes to DocData and to ReutersDocMaker.
Hope I got it right?
- Would you like to add the new task to list in TestPerfTasksParse, with no parameters and with some parameters?
(it should be anyhow parse-tested now automatically, but with no params.)
- the meaning of the new reuters.doc.maker.store.bytes property is: also store the
entire doc content as a raw bytes field, right? (It was not clear to from the javadoc.)
Regards,
Doron
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch, field-selector-bench.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll resolved LUCENE-837.
------------------------------------
Resolution: Fixed
Lucene Fields: (was: [New])
committed revision 520890
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll updated LUCENE-837:
-----------------------------------
Attachment: benchmark-updates.patch
See changes.txt in contrib/benchmark.
Patch applies in contrib/benchmark, NOT Lucene root!
Does some refactoring of QueryMaker implementations to share some common code and introduces AbstractQueryMaker as parent class.
SearchTravRetTask now extends SearchTravTask.
ReadTask now implements traversalSize() method to allow us to specify number of hits to traverse (SearchTravTask now supports a command parameter). This will allow us to simulate user scenario of traversing 10 docs at a time.
Added new FileBasedQueryMaker which allows for the specification of a file or resource containing a list of queries that can be parsed by the QP, assuming one per line.
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-updates.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-837) contrib/benchmark QueryMaker and
Task Refactorings
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483682 ]
Doron Cohen commented on LUCENE-837:
------------------------------------
I updated the docs to be clearer abut the records counting, and also list the new task and property in the package docs.
Committed.
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch, field-selector-bench.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll reopened LUCENE-837:
------------------------------------
Lucene Fields: [Patch Available]
more updates coming shortly. I will attach patch, but am also going to commit.
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Updated: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by Doron Cohen <DO...@il.ibm.com>.
Grant Ingersoll <gr...@gmail.com> wrote on 20/03/2007 05:10:47:
> Thanks, Doron. If you good w/ all the changes I will commit tonight.
Yes please..
>
> We might want to start thinking about Unit Tests... :-) Seems kind
> of weird to have tests for tests, but this is becoming sufficiently
> complex that it should have some tests.
I had the same feeling, I think I'll give it a try.
>
> Also, +1 for deprecating and eventually removing the original
> benchmark stuff that kicked this all off. The byTask frameworks is
> very nice and I don't see the point of maintaining the old one when
> the task-based approach is a superset of it.
+1
Doron
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
Re: [jira] Updated: (LUCENE-837) contrib/benchmark QueryMaker and Task Refactorings
Posted by Grant Ingersoll <gr...@gmail.com>.
Thanks, Doron. If you good w/ all the changes I will commit tonight.
We might want to start thinking about Unit Tests... :-) Seems kind
of weird to have tests for tests, but this is becoming sufficiently
complex that it should have some tests.
Also, +1 for deprecating and eventually removing the original
benchmark stuff that kicked this all off. The byTask frameworks is
very nice and I don't see the point of maintaining the old one when
the task-based approach is a superset of it.
On Mar 20, 2007, at 6:09 AM, Doron Cohen (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/LUCENE-837?
> page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
>
> Doron Cohen updated LUCENE-837:
> -------------------------------
>
> Attachment: benchmark-more-updates.patch
>
> The changes looks good.
> I integrated in my changes:
>
> - Modified query-maker generation for read related tasks to make
> further read tasks addition simpler and safer.
> - Changed Taks' setParams() to throw UnsupportedOperationException
> if that task does not suppot command line param.
> - Improved javadoc to specify all properties command line params
> currently supported.
> - Refactored ReportTasks so that it is easy/possible now to create
> new report tasks.
>
> If you feel comfortable with committng them together, that would be
> great. Or, if you prefer to commit yours first, I'll reintegrate
> mine later. The changes are supposed to allow you now to easily add
> new report tasks.
>
> I also modified the micro-standard.alg to use the new traverse params.
> (I didn't try the new file based query makers.)
>
> Attached file (benchmark-more-updates.patch) contains the changes
> from benchmark-updates.patch.
>
>> contrib/benchmark QueryMaker and Task Refactorings
>> --------------------------------------------------
>>
>> Key: LUCENE-837
>> URL: https://issues.apache.org/jira/browse/LUCENE-837
>> Project: Lucene - Java
>> Issue Type: Bug
>> Components: contrib/benchmark
>> Reporter: Grant Ingersoll
>> Assigned To: Grant Ingersoll
>> Priority: Minor
>> Attachments: benchmark-more-updates.patch, benchmark-
>> updates.patch
>>
>>
>> Introduce an abstract QueryMaker implementation that shares much
>> of the common code between the various QueryMaker implementations.
>> Add in a new QueryMaker for reading queries from a file that is
>> specified in the properties.
>> Patch shortly, and if no concerns, will commit tomorrow or Wed.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-dev-help@lucene.apache.org
>
------------------------------------------------------
Grant Ingersoll
http://www.grantingersoll.com/
http://lucene.grantingersoll.com
http://www.paperoftheweek.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-837) contrib/benchmark QueryMaker and Task
Refactorings
Posted by "Doron Cohen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doron Cohen updated LUCENE-837:
-------------------------------
Attachment: benchmark-more-updates.patch
The changes looks good.
I integrated in my changes:
- Modified query-maker generation for read related tasks to make further read tasks addition simpler and safer.
- Changed Taks' setParams() to throw UnsupportedOperationException if that task does not suppot command line param.
- Improved javadoc to specify all properties command line params currently supported.
- Refactored ReportTasks so that it is easy/possible now to create new report tasks.
If you feel comfortable with committng them together, that would be great. Or, if you prefer to commit yours first, I'll reintegrate mine later. The changes are supposed to allow you now to easily add new report tasks.
I also modified the micro-standard.alg to use the new traverse params.
(I didn't try the new file based query makers.)
Attached file (benchmark-more-updates.patch) contains the changes from benchmark-updates.patch.
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-837) contrib/benchmark QueryMaker and
Task Refactorings
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12483586 ]
Grant Ingersoll commented on LUCENE-837:
----------------------------------------
Thanks. Making the changes now. I moved the bytes storage up to BasicDocMaker.java. Added the copyright and am adding a test.
We may eventually want to store arbitrary bytes, too, but no need to now.
I changed the property to be
doc.store.body.bytes which means to store the bytes of the body as a field.
The testing has already paid off. I wasn't setting the params var correctly.
Committed
> contrib/benchmark QueryMaker and Task Refactorings
> --------------------------------------------------
>
> Key: LUCENE-837
> URL: https://issues.apache.org/jira/browse/LUCENE-837
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/benchmark
> Reporter: Grant Ingersoll
> Assigned To: Grant Ingersoll
> Priority: Minor
> Attachments: benchmark-more-updates.patch, benchmark-updates.patch, field-selector-bench.patch
>
>
> Introduce an abstract QueryMaker implementation that shares much of the common code between the various QueryMaker implementations.
> Add in a new QueryMaker for reading queries from a file that is specified in the properties.
> Patch shortly, and if no concerns, will commit tomorrow or Wed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org