You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Shuo Cheng (Jira)" <ji...@apache.org> on 2026/07/30 07:40:00 UTC
[jira] [Updated] (FLINK-40265) BinaryStringData.compareTo uses inconsistent UTF-16 and UTF-8 ordering for SMP characters
[ https://issues.apache.org/jira/browse/FLINK-40265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shuo Cheng updated FLINK-40265:
-------------------------------
Description:
h3. Description
{{BinaryStringData.compareTo}} uses two different comparison paths:
* If both values have a non-null {{javaObject}}, it delegates to {{String.compareTo}}, which compares UTF-16 code units.
* Otherwise, it materializes both values and compares their UTF-8 bytes as unsigned bytes.
These orders are not equivalent for comparisons between an SMP character and BMP characters in the range U+E000-U+FFFF. As a result, the comparison result depends on whether a {{BinaryStringData}} is Java-backed or binary-backed and can change after a serializer round trip.
For example, compare GRINNING FACE (😀, U+1F600) with FULLWIDTH EXCLAMATION MARK (！, U+FF01):
* UTF-16: D83D DE00 < FF01, so 😀 < ！
* UTF-8: F0 9F 98 80 > EF BC 81, so 😀 > ！
h3. Reproduce
{code:java}
@Test
void compareToIsConsistentAfterSerialization() throws IOException {
BinaryStringData grinningFace = BinaryStringData.fromString("😀");
BinaryStringData fullWidthExclamationMark = BinaryStringData.fromString("！");
DataOutputSerializer output = new DataOutputSerializer(16);
StringDataSerializer.INSTANCE.serialize(grinningFace, output);
BinaryStringData deserializedGrinningFace =
(BinaryStringData)
StringDataSerializer.INSTANCE.deserialize(
new DataInputDeserializer(output.getCopyOfBuffer()));
assertThat(deserializedGrinningFace).isEqualTo(grinningFace);
assertThat(deserializedGrinningFace.compareTo(fullWidthExclamationMark)).isPositive();
assertThat(grinningFace.compareTo(fullWidthExclamationMark))
.isEqualTo(deserializedGrinningFace.compareTo(fullWidthExclamationMark));
}
{code}
h3. Actual behavior
After serialization:
* {{grinningFace.compareTo(fullWidthExclamationMark)}} is negative because both values retain Java objects and use {{String.compareTo}}.
* {{deserializedGrinningFace.compareTo(fullWidthExclamationMark)}} is positive because the deserialized value is binary-backed and comparison uses UTF-8 bytes.
The final assertion fails.
h3. Expected behavior
{{BinaryStringData.compareTo}} should produce the same ordering regardless of the backing representation and before/after serialization. The class Javadoc and {{SortUtil.putStringNormalizedKey}} already describe/use UTF-8 byte ordering, so comparison should follow one canonical ordering.
h3. Impact
This violates the {{Comparable}} contract. The original and deserialized values compare equal to each other, but have opposite ordering relative to the same third value. It can cause representation-dependent results in generated string comparisons, sorting, MIN/MAX, or other operations using {{compareTo}}.
was:
h3. Description
{{BinaryStringData.compareTo}} uses two different comparison paths:
* If both values have a non-null {{javaObject}}, it delegates to {{String.compareTo}}, which compares UTF-16 code units.
* Otherwise, it materializes both values and compares their UTF-8 bytes as unsigned bytes.
These orders are not equivalent for comparisons between an SMP character and BMP characters in the range U+E000-U+FFFF. As a result, the comparison result depends on whether a {{BinaryStringData}} is Java-backed or binary-backed and can change after a serializer round trip.
For example, compare GRINNING FACE (😀, U+1F600) with FULLWIDTH EXCLAMATION MARK (！, U+FF01):
* UTF-16: D83D DE00 < FF01, so 😀 < ！
* UTF-8: F0 9F 98 80 > EF BC 81, so 😀 > ！
h3. Reproduce
{code:ajava}
@Test
void compareToIsConsistentAfterSerialization() throws IOException {
BinaryStringData grinningFace = BinaryStringData.fromString("😀");
BinaryStringData fullWidthExclamationMark = BinaryStringData.fromString("！");
DataOutputSerializer output = new DataOutputSerializer(16);
StringDataSerializer.INSTANCE.serialize(grinningFace, output);
BinaryStringData deserializedGrinningFace =
(BinaryStringData)
StringDataSerializer.INSTANCE.deserialize(
new DataInputDeserializer(output.getCopyOfBuffer()));
assertThat(deserializedGrinningFace).isEqualTo(grinningFace);
assertThat(deserializedGrinningFace.compareTo(fullWidthExclamationMark)).isPositive();
assertThat(grinningFace.compareTo(fullWidthExclamationMark))
.isEqualTo(deserializedGrinningFace.compareTo(fullWidthExclamationMark));
}
{code}
h3. Actual behavior
After serialization:
* {{grinningFace.compareTo(fullWidthExclamationMark)}} is negative because both values retain Java objects and use {{String.compareTo}}.
* {{deserializedGrinningFace.compareTo(fullWidthExclamationMark)}} is positive because the deserialized value is binary-backed and comparison uses UTF-8 bytes.
The final assertion fails.
h3. Expected behavior
{{BinaryStringData.compareTo}} should produce the same ordering regardless of the backing representation and before/after serialization. The class Javadoc and {{SortUtil.putStringNormalizedKey}} already describe/use UTF-8 byte ordering, so comparison should follow one canonical ordering.
h3. Impact
This violates the {{Comparable}} contract. The original and deserialized values compare equal to each other, but have opposite ordering relative to the same third value. It can cause representation-dependent results in generated string comparisons, sorting, MIN/MAX, or other operations using {{compareTo}}.
> BinaryStringData.compareTo uses inconsistent UTF-16 and UTF-8 ordering for SMP characters
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-40265
> URL: https://issues.apache.org/jira/browse/FLINK-40265
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Runtime
> Reporter: Shuo Cheng
> Priority: Major
>
> h3. Description
> {{BinaryStringData.compareTo}} uses two different comparison paths:
> * If both values have a non-null {{javaObject}}, it delegates to {{String.compareTo}}, which compares UTF-16 code units.
> * Otherwise, it materializes both values and compares their UTF-8 bytes as unsigned bytes.
> These orders are not equivalent for comparisons between an SMP character and BMP characters in the range U+E000-U+FFFF. As a result, the comparison result depends on whether a {{BinaryStringData}} is Java-backed or binary-backed and can change after a serializer round trip.
> For example, compare GRINNING FACE (😀, U+1F600) with FULLWIDTH EXCLAMATION MARK (！, U+FF01):
> * UTF-16: D83D DE00 < FF01, so 😀 < ！
> * UTF-8: F0 9F 98 80 > EF BC 81, so 😀 > ！
> h3. Reproduce
> {code:java}
> @Test
> void compareToIsConsistentAfterSerialization() throws IOException {
> BinaryStringData grinningFace = BinaryStringData.fromString("😀");
> BinaryStringData fullWidthExclamationMark = BinaryStringData.fromString("！");
> DataOutputSerializer output = new DataOutputSerializer(16);
> StringDataSerializer.INSTANCE.serialize(grinningFace, output);
> BinaryStringData deserializedGrinningFace =
> (BinaryStringData)
> StringDataSerializer.INSTANCE.deserialize(
> new DataInputDeserializer(output.getCopyOfBuffer()));
> assertThat(deserializedGrinningFace).isEqualTo(grinningFace);
> assertThat(deserializedGrinningFace.compareTo(fullWidthExclamationMark)).isPositive();
> assertThat(grinningFace.compareTo(fullWidthExclamationMark))
> .isEqualTo(deserializedGrinningFace.compareTo(fullWidthExclamationMark));
> }
> {code}
> h3. Actual behavior
> After serialization:
> * {{grinningFace.compareTo(fullWidthExclamationMark)}} is negative because both values retain Java objects and use {{String.compareTo}}.
> * {{deserializedGrinningFace.compareTo(fullWidthExclamationMark)}} is positive because the deserialized value is binary-backed and comparison uses UTF-8 bytes.
> The final assertion fails.
> h3. Expected behavior
> {{BinaryStringData.compareTo}} should produce the same ordering regardless of the backing representation and before/after serialization. The class Javadoc and {{SortUtil.putStringNormalizedKey}} already describe/use UTF-8 byte ordering, so comparison should follow one canonical ordering.
> h3. Impact
> This violates the {{Comparable}} contract. The original and deserialized values compare equal to each other, but have opposite ordering relative to the same third value. It can cause representation-dependent results in generated string comparisons, sorting, MIN/MAX, or other operations using {{compareTo}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)