You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2009/08/22 14:36:14 UTC
[jira] Created: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
----------------------------------------------------------------------------------------------
Key: LUCENE-1843
URL: https://issues.apache.org/jira/browse/LUCENE-1843
Project: Lucene - Java
Issue Type: Improvement
Components: Analysis
Affects Versions: 2.9
Reporter: Uwe Schindler
Assignee: Uwe Schindler
Fix For: 2.9
Attachments: LUCENE-1843.patch
This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1843:
----------------------------------
Attachment: LUCENE-1843.patch
Patch - Will commit soon.
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1843) Convert some tests to new
TokenStream API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746638#action_12746638 ]
Uwe Schindler commented on LUCENE-1843:
---------------------------------------
>From a private mail with Robert Muir:
yes, all of what you mentioned are problems, and testing for
attributes that should be there is good in my opinion too.
I noticed the shingle problem as well, it was strange to test
termAtt.toString() and expect position increments or types to appear
:/
one reason I asked about this, is at some point it would be nice to
try to factor test cases in lucene contrib. currently, they all have
same helper methods such as assertAnalyzesTo and this is silly in my
opinion.
On Sun, Aug 23, 2009 at 12:57 PM, Uwe Schindler<uw...@thetaphi.de> wrote:
> There are more problems. The test with getAttribute is good, if you are
> really sure, if the attribute is really available and want assert this. In
> all other cases addAttribute should be used to consume a TokenStream. The
> changed ones were problematic, because they used foreign TokenStreams, do
> not for sure have all these attributes.
>
> I thought, all tests in contrib use LuceneTestCase as superclass, but they
> use the standard junit class. Because of that I did not notice when I put
> setOnlyUseNewAPI(true) into LuceneTestCase.setUp(), that they are run with
> the default false setting.
>
> Other problems in these tests are, that some (shingle tests) use
> TermAttribute.toString() which looks different if the attribute is
> implemented by TermAttributeImpl (newAPI=true) or Token (newAPI=false).
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1843) Convert some tests to new
TokenStream API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-1843.
-----------------------------------
Resolution: Fixed
Committed revision: 807190
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch, LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1843) Convert some tests to
new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12746694#action_12746694 ]
Uwe Schindler edited comment on LUCENE-1843 at 8/23/09 4:35 PM:
----------------------------------------------------------------
Patch that makes all contrib/analyzer tests that work with TokenStreams subclasses of BaseTokenStreamTestCase. This superclass now has a lot of utility methods to check TokenStreams using arrays of strings/ints.
The patch also contains a better version of SingleTokenTokenStream, using the Token.copyTo() function and a Token/TokenWrapper instance as attribute implementation.
This patch may still include some unused imports, had no time to check this manually (I am the person, that codes with Notepad...)
was (Author: thetaphi):
Patch that makes all contrib/analyzer tests that work with TokenStreams subclasses of BaseTokenStreamTestCase. This superclass now has a lot of utility methods to check TokenStreams using arrays of strings/ints.
This patch may still include some unused imports, had no time to check this manually (I am the person, that codes with Notepad...)
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1843:
----------------------------------
Attachment: LUCENE-1846.patch
Patch that makes all contrib/analyzer tests that work with TokenStreams subclasses of BaseTokenStreamTestCase. This superclass now has a lot of utility methods to check TokenStreams using arrays of strings/ints.
This patch may still include some unused imports, had no time to check this manually (I am the person, that codes with Notepad...)
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1843:
----------------------------------
Attachment: LUCENE-1843.patch
- Small updates
- forget conversion of two filters in contrib/memory
Hope this is the last patch.
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-1843) Convert some tests to new
TokenStream API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler resolved LUCENE-1843.
-----------------------------------
Resolution: Fixed
Committed revision: 806847
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Reopened: (LUCENE-1843) Convert some tests to new
TokenStream API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler reopened LUCENE-1843:
-----------------------------------
There are some more tests, that fail with onlyUseNewAPI in contrib/analyzers.
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1843:
----------------------------------
Attachment: (was: LUCENE-1846.patch)
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1843:
----------------------------------
Attachment: LUCENE-1843.patch
Now the right file.
Will commit tomorrow.
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1843) Convert some tests to new TokenStream
API, better support of cross-impl AttributeImpl.copyTo()
Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Uwe Schindler updated LUCENE-1843:
----------------------------------
Attachment: LUCENE-1843.patch
Some more tests converted. Also optimized Token.copyTo() to also respect TokenWrapper as target.
> Convert some tests to new TokenStream API, better support of cross-impl AttributeImpl.copyTo()
> ----------------------------------------------------------------------------------------------
>
> Key: LUCENE-1843
> URL: https://issues.apache.org/jira/browse/LUCENE-1843
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Analysis
> Affects Versions: 2.9
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 2.9
>
> Attachments: LUCENE-1843.patch, LUCENE-1843.patch, LUCENE-1843.patch, LUCENE-1843.patch
>
>
> This patch converts some remaining tests to the new TokenStream API and non-deprecated classes.
> This patch also enhances AttributeImpl.copyTo() of Token and TokenWrapper to also support copying e.g. TermAttributeImpl into Token. The target impl must only support all interfaces but must not be of the same type. Token and TokenWrapper use optimized coping without casting to 6 interfaces where possible.
> Maybe the special tokenizers in contrib (shingle matrix and so on using tokens to cache may be enhanced by that). Also Yonik's request for optimized copying of states between incompatible AttributeSources may be enhanced by that (possibly a new issue).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org