You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Robert Muir (JIRA)" <ji...@apache.org> on 2010/01/09 19:56:54 UTC
[jira] Created: (LUCENE-2201) more performance improvements for
snowball
more performance improvements for snowball
------------------------------------------
Key: LUCENE-2201
URL: https://issues.apache.org/jira/browse/LUCENE-2201
Project: Lucene - Java
Issue Type: Improvement
Components: contrib/analyzers
Reporter: Robert Muir
Priority: Minor
Attachments: LUCENE-2201.patch
i took a more serious look at snowball after LUCENE-2194.
This gives greatly improved performance, but note it has some minor breaks to snowball internals:
* Among.s becomes a char[] instead of a string
* SnowballProgram.current becomes a char[] instead of a StringBuilder
* SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
* same as the above with eq_s_b and eq_v_b
* replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798478#action_12798478 ]
Robert Muir commented on LUCENE-2201:
-------------------------------------
if no one objects I will commit this in a few days, nothing that technically some parts of the api have changed, even though nothing uses it directly and why would you manually subclass SnowballProgram...
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Resolved: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir resolved LUCENE-2201.
---------------------------------
Resolution: Fixed
Fix Version/s: 3.1
Committed revision 898976.
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Fix For: 3.1
>
> Attachments: LUCENE-2201.patch, LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2201:
--------------------------------
Attachment: LUCENE-2201.patch
patch to make snowball work on char[]
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799706#action_12799706 ]
Robert Muir commented on LUCENE-2201:
-------------------------------------
all tests pass (and also LUCENE-2203 tests) with a clean build or binary .class files...
would like to commit this one at the end of today also.
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch, LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir updated LUCENE-2201:
--------------------------------
Attachment: LUCENE-2201.patch
here is a patch showing what i mean, it seems almost silly but appears to work with old SnowballProgram class files.
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch, LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robert Muir reassigned LUCENE-2201:
-----------------------------------
Assignee: Robert Muir
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12799335#action_12799335 ]
Robert Muir commented on LUCENE-2201:
-------------------------------------
Hello, I was looking at this, and it causes problems for any old binary SnowballPrograms because of String -> CharSequence signature changes, etc.
So, are we worried about this? it looks fixable by adding overloaded String-based methods to all of these, but is messy.
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Assignee: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-2201) more performance improvements for
snowball
Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12798748#action_12798748 ]
Robert Muir commented on LUCENE-2201:
-------------------------------------
all tests from LUCENE-2203 pass with this patch... (it does not change any snowball behavior).
I will also update the patch to additionally make member variables in Among final, consistent with what has already happened in Snowball: http://svn.tartarus.org/snowball/trunk/snowball/java/org/tartarus/snowball/Among.java?view=diff&r1=267&r2=502&diff_format=h
> more performance improvements for snowball
> ------------------------------------------
>
> Key: LUCENE-2201
> URL: https://issues.apache.org/jira/browse/LUCENE-2201
> Project: Lucene - Java
> Issue Type: Improvement
> Components: contrib/analyzers
> Reporter: Robert Muir
> Priority: Minor
> Attachments: LUCENE-2201.patch
>
>
> i took a more serious look at snowball after LUCENE-2194.
> This gives greatly improved performance, but note it has some minor breaks to snowball internals:
> * Among.s becomes a char[] instead of a string
> * SnowballProgram.current becomes a char[] instead of a StringBuilder
> * SnowballProgram.eq_s(int, String) becomes eq_s(int, CharSequence), so that eq_v(StringBuilder) doesnt need to create an extra string.
> * same as the above with eq_s_b and eq_v_b
> * replace_s(int, int, String) becomes replace_s(int, int, CharSequence), so that StringBuilder-based slice and insertion methods don't need to create an extra string.
> all of these "breaks" imho are only theoretical, the problem is just that pretty much everything is public or protected in the snowball internals.
> the performance improvement here depends heavily upon the snowball language in use, but its way more significant than LUCENE-2194.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org