[jira] [Comment Edited] (COLLECTIONS-899) CaseInsensitiveMap incorrectly converts U+03D0 (ϐ) to U+03B2 (β), causing distinct keys to collide

["Bernd Eckenfels (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/COLLECTIONS-899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18109655#comment-18109655 ] 

Bernd Eckenfels edited comment on COLLECTIONS-899 at 8/30/26 12:52 PM:
-----------------------------------------------------------------------

I think because of COLLECTIOM-294 it tries to emulate case insensitive Match (hoewever Not very successful ,) 

I would suggest rather than fixing it completely (not sure its possible at all. Would Need to hash the casefolded intermediary), I would suggest to fix the doc (especially the missleading „uses Unicode files“.

BTW when I changed it tostring.toLowerCase(ROOT) the testLocaleIndepemdemce fails for en:1 aka U+03C2 03C3. That’s the Final sigma rule 


was (Author: b.eckenfels):
I think because of Collection-294 it tries to Emulate case insensitive Match (jowerver Not very successful ,) 

I would suggest rather than fixing it completely, I would just change it toßlowerCase and document it’s quirk?

> CaseInsensitiveMap incorrectly converts U+03D0 (ϐ) to U+03B2 (β), causing distinct keys to collide
> --------------------------------------------------------------------------------------------------
>
>                 Key: COLLECTIONS-899
>                 URL: https://issues.apache.org/jira/browse/COLLECTIONS-899
>             Project: Commons Collections
>          Issue Type: Bug
>          Components: Map
>    Affects Versions: 4.6.0
>         Environment: Apache Commons Collections 4.6.0
> Java: 17
> OS: Windows 11
>            Reporter: 尹茂椿萱
>            Priority: Major
>              Labels: case-insensitive-map, regression,, unicode,
>
> {{CaseInsensitiveMap}} performs an incorrect Unicode lowercase conversion for some characters.
> The class-level Javadoc states that keys are converted to all lowercase in a locale-independent fashion using information from the Unicode data file. However, the current implementation converts each character using:
>  
> {{Character.toLowerCase(Character.toUpperCase(chars[i]))}}
> This is not equivalent to Unicode lowercase conversion.
> For example, the Unicode character U+03D0 (GREEK BETA SYMBOL, {{{}ϐ{}}}) is unchanged by lowercase conversion:
>  
> {{"\u03D0".toLowerCase(Locale.ROOT)}}
> returns {{{}ϐ{}}}.
> However, the current implementation converts it as follows:
>  
> {{ϐ (U+03D0)
> -> Character.toUpperCase()
> β (U+03B2)
> -> Character.toLowerCase()
> β (U+03B2)}}
> As a result, {{ϐ}} and {{β}} are incorrectly normalized to the same key.
> This causes observable incorrect behavior in the public {{CaseInsensitiveMap}} API. For example:
>  
> {{CaseInsensitiveMap<String, String> map = new CaseInsensitiveMap<>();
> map.put("\u03D0", "beta-symbol");
> map.put("\u03B2", "greek-beta");
> assertEquals(2, map.size());}}
> The expected size is {{{}2{}}}, because the two input keys are distinct and are not lowercase-equivalent according to Unicode lowercase mapping.
> With the current implementation, the actual size is {{1}} because both keys are converted to {{{}β{}}}.
> The previous implementation used:
>  
> {{key.toString().toLowerCase()}}
> and preserved {{ϐ}} as {{{}ϐ{}}}, so this collision did not occur.
> This appears to be a regression introduced by the character-wise case conversion implementation.
> *Expected behavior*
> {{CaseInsensitiveMap}} should convert keys according to locale-independent Unicode lowercase semantics. In particular:
>  
> {{U+03D0 (ϐ) -> U+03D0 (ϐ)
> U+03B2 (β) -> U+03B2 (β)}}
> Therefore, inserting both keys should retain two distinct entries.
> *Actual behavior*
> The current implementation converts:
>  
> {{U+03D0 (ϐ) -> U+03B2 (β)
> U+03B2 (β) -> U+03B2 (β)}}
> Consequently, inserting both keys results in only one map entry.
> *Steps to reproduce*
>  # Create a {{{}CaseInsensitiveMap{}}}.
>  # Insert {{"\u03D0"}} as a key.
>  # Insert {{"\u03B2"}} as a key.
>  # Check the map size.
> Reproducer:
>  
> {{CaseInsensitiveMap<String, String> map = new CaseInsensitiveMap<>();
> map.put("\u03D0", "beta-symbol");
> map.put("\u03B2", "greek-beta");
> System.out.println(map.size());}}
> Actual result:
>  
> {{1}}
> Expected result:
>  
> {{2}}
> *Possible fix*
> The issue appears to be caused by performing case conversion character-by-character using {{{}Character.toLowerCase(Character.toUpperCase(char)){}}}.
> A possible fix would be to use Java's string-level lowercase conversion with an explicit locale, for example:
>  
> {{return key.toString().toLowerCase(Locale.ROOT);}}
> This preserves the intended locale-independent Unicode lowercase semantics described by the class-level documentation.
> If the character-wise implementation is retained for performance reasons, it should nevertheless ensure that its behavior is consistent with the documented Unicode lowercase mapping, including characters such as U+03D0 ({{{}ϐ{}}}).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)