You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "raminqaf (via GitHub)" <gi...@apache.org> on 2026/09/04 10:10:52 UTC
[PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
raminqaf opened a new pull request, #29101:
URL: https://github.com/apache/flink/pull/29101
## What is the purpose of the change
This is a sub-task of [FLINK-40485](https://issues.apache.org/jira/browse/FLINK-40485) / FLIP-604 (Complete VARIANT Primitive Coverage with UUID and Timestamps).
It adds explicit `CAST` and `TRY_CAST` support between the `UUID` type and character strings and binary. The `UUID` type was introduced in FLINK-40486 and its runtime in FLINK-40490. Until now a `UUID` could only be produced as a literal and round-tripped through sources and sinks. It could not be converted to or from any other type. This change makes `UUID` usable together with the string and binary types that carry it in practice.
## Brief change log
- Add `UuidCastUtils` in `flink-table-runtime` with the runtime helpers: render a `UUID` to its canonical string, parse a string to the 16-byte encoding, and validate a 16-byte binary value.
- Add four cast rules in `flink-table-planner` and register them in `CastRuleProvider`: `UuidToStringCastRule`, `UuidToBinaryCastRule`, `StringToUuidCastRule`, `BinaryToUuidCastRule`.
- Route `UUID` casts through `LogicalTypeCasts` in `SqlCastFunction#canCastFrom`, in both directions. Calcite's own checker only allows a `UUID` to be cast from another `UUID`, so the type-level matrix has to govern it.
- Extend `LogicalTypeCasts` with the `UUID` cast matrix: `UUID` to `CHAR`/`VARCHAR`, `UUID` to `BINARY(16)`/`BYTES` only, and `CHAR`/`VARCHAR`/`BINARY`/`VARBINARY` to `UUID`.
## Cast semantics
All casts are explicit. There are no implicit coercions to or from `UUID`. `VARIANT` and `UUID` conversion is out of scope and handled by a separate issue.
**`UUID` to `STRING`.** Renders the canonical lower-case 8-4-4-4-12 form.
**`STRING` to `UUID`.** Parsed leniently, matching PostgreSQL's `uuid` input (`string_to_uuid` in `adt/uuid.c`). The digits may be upper or lower case, the value may be wrapped in a single pair of braces, and a single hyphen may follow any group of four digits. A malformed value fails `CAST` and returns `NULL` for `TRY_CAST`.
| Input | Result |
|-------------------------------------------|-------------------------------------|
| `A0EEBC99-9C0B-4EF8-BB6D-6BB9BD380A11` | passes (upper case) |
| `{a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11}` | passes (braces) |
| `a0eebc999c0b4ef8bb6d6bb9bd380a11` | passes (no hyphens) |
| `a0ee-bc99-9c0b-4ef8-bb6d-6bb9-bd38-0a11` | passes (a hyphen after every group) |
| `{a0eebc99-9c0b4ef8-bb6d6bb9-bd380a11}` | passes (braces, hyphens at 8/16/24) |
| `a0e-ebc999c0b4ef8bb6d6bb9bd380a11` | fails (hyphen inside a group) |
| `-a0eebc999c0b4ef8bb6d6bb9bd380a11` | fails (leading hyphen) |
| `a0eebc99--9c0b4ef8bb6d6bb9bd380a11` | fails (doubled hyphen) |
| `{a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11` | fails (unmatched brace) |
| `a0eebc999c0b4ef8bb6d6bb9bd380a1` | fails (31 digits) |
| `g0eebc999c0b4ef8bb6d6bb9bd380a11` | fails (non-hex digit) |
The five passing examples all decode to the same value `a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11`.
**`UUID` to `BINARY(16)` / `BYTES`.** Produces the 16 big-endian bytes. Any other target width is rejected during validation, since padding or trimming would corrupt the value.
**`BINARY` / `BYTES` to `UUID`.** Reinterprets the bytes as a `UUID`. The value has to be exactly 16 bytes, which is checked at runtime. A different length fails `CAST` and returns `NULL` for `TRY_CAST`.
## Verifying this change
This change added tests and can be verified as follows:
- `CastRulesTest` covers the exhaustive value and lenient-parse matrix at the cast-executor level.
- `LogicalTypeCastsTest` covers the type-level cast matrix in both directions.
- `CastFunctionITCase#uuidCasts` covers the end-to-end SQL and Table API paths, validation failures, and `TRY_CAST` returning `NULL`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes
- If yes, how is the feature documented? The `UUID` type documentation, including casting, is tracked in FLINK-40494 and is not part of this PR.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3941120173
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java:
##########
@@ -162,6 +164,18 @@ class CastRulesTest {
private static final Bitmap DEFAULT_BITMAP = Bitmap.fromArray(new int[] {0, 1, 2});
+ private static final String UUID_STRING = "550e8400-e29b-41d4-a716-446655440000";
+ private static final byte[] UUID_BYTES = uuidBytes(UUID_STRING);
+
+ private static byte[] uuidBytes(String uuid) {
+ final java.util.UUID value = java.util.UUID.fromString(uuid);
+ final byte[] result = new byte[16];
+ java.nio.ByteBuffer.wrap(result)
Review Comment:
absolute name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3943506963
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java:
##########
@@ -162,6 +164,18 @@ class CastRulesTest {
private static final Bitmap DEFAULT_BITMAP = Bitmap.fromArray(new int[] {0, 1, 2});
+ private static final String UUID_STRING = "550e8400-e29b-41d4-a716-446655440000";
+ private static final byte[] UUID_BYTES = uuidBytes(UUID_STRING);
+
+ private static byte[] uuidBytes(String uuid) {
+ final java.util.UUID value = java.util.UUID.fromString(uuid);
+ final byte[] result = new byte[16];
+ java.nio.ByteBuffer.wrap(result)
Review Comment:
Removed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3941115202
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -687,6 +698,15 @@ private static boolean supportsCasting(
// BITMAP can only be cast to BYTES (unbounded VARBINARY), because trimming or padding
// would corrupt the serialized bitmap data.
return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
+ } else if (sourceRoot == UUID && targetRoot == BINARY) {
Review Comment:
I feel like we miss vise versa case...
as a result
```sql
SELECT CAST(UUID '550e8400-e29b-41d4-a716-446655440000' AS BINARY(10)); -- fails with validation exception
```
and this
```sql
SELECT CAST(CAST(x'01020304050607080910' AS BINARY(10)) AS UUID); -- fail with runtime exception
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933585255
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java:
##########
@@ -2136,6 +2157,28 @@ private CastTestSpecBuilder fail(
return this;
}
+ private CastTestSpecBuilder fail(
+ DataType dataType,
+ Object src,
+ Class<? extends Throwable> exception,
+ String messageSubstring) {
+ this.inputTypes.add(dataType);
+ this.assertionExecutors.add(
+ executor ->
+ assertThatThrownBy(() -> executor.cast(src))
+ .isInstanceOf(exception)
+ .hasStackTraceContaining(messageSubstring));
+ this.descriptions.add("{" + src + " => " + exception.getName() + "}");
+ this.castContexts.add(
+ CastRule.Context.create(
Review Comment:
nit: overload the existing fail instead of creating a new context
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin merged PR #29101:
URL: https://github.com/apache/flink/pull/29101
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #29101:
URL: https://github.com/apache/flink/pull/29101#issuecomment-5539079006
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "9dd6a2128841115033da99ddd89112caef347717",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "9dd6a2128841115033da99ddd89112caef347717",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 9dd6a2128841115033da99ddd89112caef347717 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933503958
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/UuidToBinaryCastRule.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions.casting;
+
+import org.apache.flink.table.types.logical.LogicalType;
+import org.apache.flink.table.types.logical.LogicalTypeRoot;
+import org.apache.flink.table.types.logical.VarBinaryType;
+import org.apache.flink.table.types.logical.utils.LogicalTypeChecks;
+
+/**
+ * {@link LogicalTypeRoot#UUID} to binary cast rule.
+ *
+ * <p>A {@code UUID} is stored as its 16-byte big-endian encoding, so the cast returns those bytes
+ * unchanged. Only {@code BINARY(16)} and {@code BYTES} are supported targets: any other width would
+ * pad or trim the value and thereby corrupt it.
+ *
+ * <p>The input is already the internal {@code byte[]}, so the generated expression is the input
+ * term itself, with no transformation. Example generated code:
+ *
+ * <pre>
+ * result$0 = uuid$0;
+ * </pre>
+ */
+class UuidToBinaryCastRule extends AbstractExpressionCodeGeneratorCastRule<byte[], byte[]> {
+
+ static final UuidToBinaryCastRule INSTANCE = new UuidToBinaryCastRule();
+
+ private UuidToBinaryCastRule() {
+ super(
+ CastRulePredicate.builder()
+ .predicate(
+ (input, target) ->
+ input.is(LogicalTypeRoot.UUID) && isSupportedTarget(target))
+ .build());
+ }
+
+ private static boolean isSupportedTarget(LogicalType target) {
+ if (target.is(LogicalTypeRoot.BINARY)) {
+ return LogicalTypeChecks.getLength(target) == 16;
+ }
+ if (target.is(LogicalTypeRoot.VARBINARY)) {
+ return LogicalTypeChecks.getLength(target) == VarBinaryType.MAX_LENGTH;
Review Comment:
https://github.com/apache/flink/pull/29101#discussion_r3933499611
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java:
##########
@@ -1571,6 +1584,50 @@ Stream<CastTestSpecBuilder> testCases() {
.fromCase(BITMAP(), DEFAULT_BITMAP, DEFAULT_BITMAP.toBytes())
.fromCase(BITMAP(), Bitmap.empty(), Bitmap.empty().toBytes())
.fromCase(BITMAP(), null, null),
+ // UUID cast rules. A UUID renders as its canonical lower-case 8-4-4-4-12 form and
+ // maps to its 16-byte big-endian encoding.
+ CastTestSpecBuilder.testCastTo(STRING())
+ .fromCase(UUID(), UUID_BYTES, fromString(UUID_STRING))
+ .fromCase(UUID(), null, null),
+ // a bounded character target trims, and CHAR pads to its fixed width
+ CastTestSpecBuilder.testCastTo(VARCHAR(8))
+ .fromCase(UUID(), UUID_BYTES, fromString("550e8400")),
+ CastTestSpecBuilder.testCastTo(CHAR(38))
+ .fromCase(UUID(), UUID_BYTES, fromString(UUID_STRING + " ")),
+ CastTestSpecBuilder.testCastTo(BINARY(16))
+ .fromCase(UUID(), UUID_BYTES, UUID_BYTES)
+ .fromCase(UUID(), null, null),
+ CastTestSpecBuilder.testCastTo(BYTES())
+ .fromCase(UUID(), UUID_BYTES, UUID_BYTES)
+ .fromCase(UUID(), null, null),
+ // a UUID is parsed leniently from a string, following PostgreSQL conventions
+ CastTestSpecBuilder.testCastTo(UUID())
+ .fromCase(STRING(), fromString(UUID_STRING), UUID_BYTES)
+ .fromCase(STRING(), fromString(UUID_STRING.toUpperCase()), UUID_BYTES)
+ .fromCase(STRING(), fromString("{" + UUID_STRING + "}"), UUID_BYTES)
+ .fromCase(STRING(), fromString(UUID_STRING.replace("-", "")), UUID_BYTES)
+ // a hyphen may follow any group of four digits, PostgreSQL style
+ .fromCase(
+ STRING(),
+ fromString("550e-8400-e29b-41d4-a716-4466-5544-0000"),
+ UUID_BYTES)
+ .fromCase(STRING(), null, null)
+ // a malformed value fails, and yields null for TRY_CAST
+ .fail(STRING(), fromString("not-a-uuid"), TableRuntimeException.class)
+ .fail(STRING(), fromString("550e8400"), TableRuntimeException.class)
+ // too many hexadecimal digits
+ .fail(STRING(), fromString(UUID_STRING + "00"), TableRuntimeException.class)
+ // a hyphen inside a four-digit group is rejected
+ .fail(
+ STRING(),
+ fromString("5-50e8400e29b41d4a716446655440000"),
+ TableRuntimeException.class),
Review Comment:
Added error messages
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3941141545
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -687,6 +698,15 @@ private static boolean supportsCasting(
// BITMAP can only be cast to BYTES (unbounded VARBINARY), because trimming or padding
// would corrupt the serialized bitmap data.
return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
+ } else if (sourceRoot == UUID && targetRoot == BINARY) {
Review Comment:
i would expect both failing with validation exception since info about types is known during planning here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933501407
##########
flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java:
##########
@@ -297,7 +298,31 @@ private static Stream<Arguments> testData() {
new VariantType(),
new MapType(new IntType(), new CharType()),
false,
- false));
+ false),
+ // UUID casts are explicit only, in both directions
+ Arguments.of(new UuidType(), VarCharType.STRING_TYPE, false, true),
+ Arguments.of(new UuidType(), new CharType(), false, true),
+ Arguments.of(VarCharType.STRING_TYPE, new UuidType(), false, true),
+ Arguments.of(new CharType(), new UuidType(), false, true),
+ // UUID maps to its 16-byte encoding: only BINARY(16) and BYTES, any other width is
+ // rejected because it would pad or trim the value
+ Arguments.of(new UuidType(), new BinaryType(16), false, true),
+ Arguments.of(new UuidType(), new BinaryType(10), false, false),
+ Arguments.of(
+ new UuidType(), new VarBinaryType(VarBinaryType.MAX_LENGTH), false, true),
+ Arguments.of(new UuidType(), new VarBinaryType(16), false, false),
Review Comment:
Yes, it is indeed wrong. Updated it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933475083
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -218,7 +219,7 @@ public final class LogicalTypeCasts {
castTo(VARCHAR)
.implicitFromFamily(CHARACTER_STRING)
.explicitFromFamily(PREDEFINED, CONSTRUCTED)
- .explicitFrom(RAW, NULL, STRUCTURED_TYPE, BITMAP, VARIANT)
+ .explicitFrom(RAW, NULL, STRUCTURED_TYPE, BITMAP, VARIANT, UUID)
Review Comment:
Yes, intentional. FLIP-604 specifies `UUID` has no implicit coercions. It matches Calcite (UUID is assignable only from UUID in `SqlTypeAssignmentRule`) and PostgreSQL, where uuid has no implicit casts.
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -218,7 +219,7 @@ public final class LogicalTypeCasts {
castTo(VARCHAR)
.implicitFromFamily(CHARACTER_STRING)
.explicitFromFamily(PREDEFINED, CONSTRUCTED)
- .explicitFrom(RAW, NULL, STRUCTURED_TYPE, BITMAP, VARIANT)
+ .explicitFrom(RAW, NULL, STRUCTURED_TYPE, BITMAP, VARIANT, UUID)
Review Comment:
Yes, intentional. FLIP-604 specifies `UUID` has no implicit coercions. It matches Calcite (`UUID` is assignable only from `UUID` in `SqlTypeAssignmentRule`) and PostgreSQL, where uuid has no implicit casts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933499611
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -679,6 +689,13 @@ private static boolean supportsCasting(
// BITMAP can only be cast to BYTES (unbounded VARBINARY), because trimming or padding
// would corrupt the serialized bitmap data.
return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
+ } else if (sourceRoot == UUID && targetRoot == BINARY) {
+ // A UUID is exactly 16 bytes, so it maps to BINARY(16). Any other width would pad or
+ // trim the value and thereby corrupt it.
+ return allowExplicit && getLength(targetType) == 16;
+ } else if (sourceRoot == UUID && targetRoot == VARBINARY) {
+ // A UUID can only be cast to BYTES (unbounded VARBINARY), for the same reason.
+ return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
Review Comment:
That is true... Changed it to:
```java
getLength(targetType) >= 16
```
Now the targetType needs to be just bigger than 16 bytes so the UUID fits in
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3943556729
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -687,6 +698,15 @@ private static boolean supportsCasting(
// BITMAP can only be cast to BYTES (unbounded VARBINARY), because trimming or padding
// would corrupt the serialized bitmap data.
return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
+ } else if (sourceRoot == UUID && targetRoot == BINARY) {
Review Comment:
Changed and now it throws
```
Caused by: org.apache.calcite.sql.validate.SqlValidatorException: Cast function cannot convert value of type BINARY(10) to type UUID
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933020862
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -218,7 +219,7 @@ public final class LogicalTypeCasts {
castTo(VARCHAR)
.implicitFromFamily(CHARACTER_STRING)
.explicitFromFamily(PREDEFINED, CONSTRUCTED)
- .explicitFrom(RAW, NULL, STRUCTURED_TYPE, BITMAP, VARIANT)
+ .explicitFrom(RAW, NULL, STRUCTURED_TYPE, BITMAP, VARIANT, UUID)
Review Comment:
Is this in sync with Calcite? No implicit cast from UUID? I'm fine with the current, I just wanted to double check here.
##########
flink-table/flink-table-common/src/main/java/org/apache/flink/table/types/logical/utils/LogicalTypeCasts.java:
##########
@@ -679,6 +689,13 @@ private static boolean supportsCasting(
// BITMAP can only be cast to BYTES (unbounded VARBINARY), because trimming or padding
// would corrupt the serialized bitmap data.
return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
+ } else if (sourceRoot == UUID && targetRoot == BINARY) {
+ // A UUID is exactly 16 bytes, so it maps to BINARY(16). Any other width would pad or
+ // trim the value and thereby corrupt it.
+ return allowExplicit && getLength(targetType) == 16;
+ } else if (sourceRoot == UUID && targetRoot == VARBINARY) {
+ // A UUID can only be cast to BYTES (unbounded VARBINARY), for the same reason.
+ return allowExplicit && getLength(targetType) == VarBinaryType.MAX_LENGTH;
Review Comment:
Doesn't need to be max length right? I guess BITMAP is var-length which makes it tricky, but UUID is not.
##########
flink-table/flink-table-common/src/test/java/org/apache/flink/table/types/LogicalTypeCastsTest.java:
##########
@@ -297,7 +298,31 @@ private static Stream<Arguments> testData() {
new VariantType(),
new MapType(new IntType(), new CharType()),
false,
- false));
+ false),
+ // UUID casts are explicit only, in both directions
+ Arguments.of(new UuidType(), VarCharType.STRING_TYPE, false, true),
+ Arguments.of(new UuidType(), new CharType(), false, true),
+ Arguments.of(VarCharType.STRING_TYPE, new UuidType(), false, true),
+ Arguments.of(new CharType(), new UuidType(), false, true),
+ // UUID maps to its 16-byte encoding: only BINARY(16) and BYTES, any other width is
+ // rejected because it would pad or trim the value
+ Arguments.of(new UuidType(), new BinaryType(16), false, true),
+ Arguments.of(new UuidType(), new BinaryType(10), false, false),
+ Arguments.of(
+ new UuidType(), new VarBinaryType(VarBinaryType.MAX_LENGTH), false, true),
+ Arguments.of(new UuidType(), new VarBinaryType(16), false, false),
Review Comment:
wrong test?
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/functions/casting/UuidToBinaryCastRule.java:
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.table.planner.functions.casting;
+
+import org.apache.flink.table.types.logical.LogicalType;
+import org.apache.flink.table.types.logical.LogicalTypeRoot;
+import org.apache.flink.table.types.logical.VarBinaryType;
+import org.apache.flink.table.types.logical.utils.LogicalTypeChecks;
+
+/**
+ * {@link LogicalTypeRoot#UUID} to binary cast rule.
+ *
+ * <p>A {@code UUID} is stored as its 16-byte big-endian encoding, so the cast returns those bytes
+ * unchanged. Only {@code BINARY(16)} and {@code BYTES} are supported targets: any other width would
+ * pad or trim the value and thereby corrupt it.
+ *
+ * <p>The input is already the internal {@code byte[]}, so the generated expression is the input
+ * term itself, with no transformation. Example generated code:
+ *
+ * <pre>
+ * result$0 = uuid$0;
+ * </pre>
+ */
+class UuidToBinaryCastRule extends AbstractExpressionCodeGeneratorCastRule<byte[], byte[]> {
+
+ static final UuidToBinaryCastRule INSTANCE = new UuidToBinaryCastRule();
+
+ private UuidToBinaryCastRule() {
+ super(
+ CastRulePredicate.builder()
+ .predicate(
+ (input, target) ->
+ input.is(LogicalTypeRoot.UUID) && isSupportedTarget(target))
+ .build());
+ }
+
+ private static boolean isSupportedTarget(LogicalType target) {
+ if (target.is(LogicalTypeRoot.BINARY)) {
+ return LogicalTypeChecks.getLength(target) == 16;
+ }
+ if (target.is(LogicalTypeRoot.VARBINARY)) {
+ return LogicalTypeChecks.getLength(target) == VarBinaryType.MAX_LENGTH;
Review Comment:
see comment above.
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java:
##########
@@ -1571,6 +1584,50 @@ Stream<CastTestSpecBuilder> testCases() {
.fromCase(BITMAP(), DEFAULT_BITMAP, DEFAULT_BITMAP.toBytes())
.fromCase(BITMAP(), Bitmap.empty(), Bitmap.empty().toBytes())
.fromCase(BITMAP(), null, null),
+ // UUID cast rules. A UUID renders as its canonical lower-case 8-4-4-4-12 form and
+ // maps to its 16-byte big-endian encoding.
+ CastTestSpecBuilder.testCastTo(STRING())
+ .fromCase(UUID(), UUID_BYTES, fromString(UUID_STRING))
+ .fromCase(UUID(), null, null),
+ // a bounded character target trims, and CHAR pads to its fixed width
+ CastTestSpecBuilder.testCastTo(VARCHAR(8))
+ .fromCase(UUID(), UUID_BYTES, fromString("550e8400")),
+ CastTestSpecBuilder.testCastTo(CHAR(38))
+ .fromCase(UUID(), UUID_BYTES, fromString(UUID_STRING + " ")),
+ CastTestSpecBuilder.testCastTo(BINARY(16))
+ .fromCase(UUID(), UUID_BYTES, UUID_BYTES)
+ .fromCase(UUID(), null, null),
+ CastTestSpecBuilder.testCastTo(BYTES())
+ .fromCase(UUID(), UUID_BYTES, UUID_BYTES)
+ .fromCase(UUID(), null, null),
+ // a UUID is parsed leniently from a string, following PostgreSQL conventions
+ CastTestSpecBuilder.testCastTo(UUID())
+ .fromCase(STRING(), fromString(UUID_STRING), UUID_BYTES)
+ .fromCase(STRING(), fromString(UUID_STRING.toUpperCase()), UUID_BYTES)
+ .fromCase(STRING(), fromString("{" + UUID_STRING + "}"), UUID_BYTES)
+ .fromCase(STRING(), fromString(UUID_STRING.replace("-", "")), UUID_BYTES)
+ // a hyphen may follow any group of four digits, PostgreSQL style
+ .fromCase(
+ STRING(),
+ fromString("550e-8400-e29b-41d4-a716-4466-5544-0000"),
+ UUID_BYTES)
+ .fromCase(STRING(), null, null)
+ // a malformed value fails, and yields null for TRY_CAST
+ .fail(STRING(), fromString("not-a-uuid"), TableRuntimeException.class)
+ .fail(STRING(), fromString("550e8400"), TableRuntimeException.class)
+ // too many hexadecimal digits
+ .fail(STRING(), fromString(UUID_STRING + "00"), TableRuntimeException.class)
+ // a hyphen inside a four-digit group is rejected
+ .fail(
+ STRING(),
+ fromString("5-50e8400e29b41d4a716446655440000"),
+ TableRuntimeException.class),
Review Comment:
We should also at least test against a substring of the error message to ensure the error is correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40487][table] CAST between UUID and STRING / BINARY(16) [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29101:
URL: https://github.com/apache/flink/pull/29101#discussion_r3933727083
##########
flink-table/flink-table-planner/src/test/java/org/apache/flink/table/planner/functions/casting/CastRulesTest.java:
##########
@@ -2136,6 +2157,28 @@ private CastTestSpecBuilder fail(
return this;
}
+ private CastTestSpecBuilder fail(
+ DataType dataType,
+ Object src,
+ Class<? extends Throwable> exception,
+ String messageSubstring) {
+ this.inputTypes.add(dataType);
+ this.assertionExecutors.add(
+ executor ->
+ assertThatThrownBy(() -> executor.cast(src))
+ .isInstanceOf(exception)
+ .hasStackTraceContaining(messageSubstring));
+ this.descriptions.add("{" + src + " => " + exception.getName() + "}");
+ this.castContexts.add(
+ CastRule.Context.create(
Review Comment:
Added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org