You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2010/04/18 22:43:49 UTC

[jira] Created: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
----------------------------------------------------------------------------------

                 Key: LUCENE-2401
                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
             Project: Lucene - Java
          Issue Type: Improvement
          Components: Analysis
            Reporter: Uwe Schindler
            Assignee: Uwe Schindler
             Fix For: 3.1


The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.

This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Updated: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2401:
----------------------------------

    Attachment: LUCENE-2401.patch

Added an test for completely unknown (long) CharSequences, that are not catched by instanceof checks in the general append() method. Clover now reports 100% in this part.

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch, LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Commented: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Robert Muir (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858348#action_12858348 ] 

Robert Muir commented on LUCENE-2401:
-------------------------------------

+1. This all seems great to me! Only one question: do we need the specialized case for CharTermAttribute really? Seems a little specialized to append chartermatt's to each other, but I think the String/StringBuffer methods are good to add and great to have working fast!



> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Commented: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858346#action_12858346 ] 

Uwe Schindler commented on LUCENE-2401:
---------------------------------------

One thing:
I did not add methods taking start and end as parameter for the specialized cases. This is not even supported in StringBuilder or other java classes implementing Appendable.

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Commented: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858351#action_12858351 ] 

Uwe Schindler commented on LUCENE-2401:
---------------------------------------

bq. do we need the specialized case for CharTermAttribute really

Why not, maybe you do copyAttributes to put tokens away and then you want to append them? In general, StringBuilder also has a specialized append() method for itsself. So it does not hurt and it is tested. Its somehow a duplicate to copyTo() [except it appends], but this one only copies the buffer and not the whole attribute (die, Token, die!).

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch, LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Updated: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2401:
----------------------------------

    Attachment: LUCENE-2401.patch

Attached the patch.

This also makes all implementation methods in the implementation class final (the non-deprecated, new ones), to help the hotspot optimizer.

Also the tests were partly rewriten, and one test bug removed.

All tests pass.

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Updated: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2401:
----------------------------------

    Attachment: LUCENE-2401.patch

Improved patch:

- I added a (commented out) perf test that was used to find the best maximum CharSequence length, that is always faster than the instanceof checks in Appendable.append(CSQ).
- Because of above tests I changed the max length to 4

I will commit this soon.

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch, LUCENE-2401.patch, LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Commented: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Earwin Burrfoot (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858358#action_12858358 ] 

Earwin Burrfoot commented on LUCENE-2401:
-----------------------------------------

I think at least I will hit that specialized case :)
My filters glue some tokens together. (Die, hard-iterative-API, die!)

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch, LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[jira] Resolved: (LUCENE-2401) Improve performance of CharTermAttribute(Impl) and also fully implement Appendable

Posted by "Uwe Schindler (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/LUCENE-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler resolved LUCENE-2401.
-----------------------------------

    Lucene Fields: [New, Patch Available]  (was: [New])
       Resolution: Fixed

Committed revision: 935521

> Improve performance of CharTermAttribute(Impl) and also fully implement Appendable
> ----------------------------------------------------------------------------------
>
>                 Key: LUCENE-2401
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2401
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Analysis
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.1
>
>         Attachments: LUCENE-2401.patch, LUCENE-2401.patch, LUCENE-2401.patch
>
>
> The Appendable.append(CharSequence) method in CharTermAttributes is good for general use. But like StringBuilder has for some common use cases specialized methods, this does the same and adds separate append methods for String, StringBuilder and CharTermAttribute itsself. This methods enable the compiler to directly link the specialized methods and don't use the instanceof checks. The unspecialized method only does the instanceof checks for longer CharSequences (>8 chars), else it simply iterates.
> This patch also fixes the required special "null" handling. append() methods are required by Appendable to append "null", if the argument is null. I dont like this, but its required. Maybe we should document, that we dont dont support it. Otherwise, JDK's formatter fails with formatting null.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org