You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "dylanhz (via GitHub)" <gi...@apache.org> on 2026/09/08 16:02:32 UTC
[PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
dylanhz opened a new pull request, #29133:
URL: https://github.com/apache/flink/pull/29133
## What is the purpose of the change
Fix [FLINK-40597](https://issues.apache.org/jira/browse/FLINK-40597): a DataStream created with explicit `Types.UUID` can fail input type validation for a non-lambda `MapFunction<UUID, String>` with `Input mismatch: Basic type expected.`
## Brief change log
- Recognize `java.util.UUID` when validating explicitly specified basic type information.
- Leave UUID extraction without explicit type information unchanged (`GenericTypeInfo<UUID>` with Kryo by default). Automatic extraction as `Types.UUID` is tracked separately in [FLINK-40546](https://issues.apache.org/jira/browse/FLINK-40546).
## Verifying this change
- Added a `fromData(Types.UUID, ...).map(...)` regression test and a check that `fromData(...)` still infers `GenericTypeInfo<UUID>`.
- Confirmed the regression test fails without the fix and passes with it.
- `DataStreamTest` and `TypeExtractorTest`: 102 tests passed after rebasing onto master.
- Full `./mvnw clean verify` was attempted. After resolving a local Python executable issue and resuming, verification stopped at `flink-s3-fs-base`: `SeaweedFsTestContainerTest` requires Docker, which is not installed locally. Subsequent modules have not been verified.
## Does this pull request potentially affect one of the following parts:
- Dependencies: no
- Public API: yes, fixes validation in the existing `@Public` `TypeExtractor`; no API signatures change
- Serializers: no
- Runtime per-record code paths: no
- Deployment or recovery: no
- S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Codex 0.147.0 (model: gpt-6)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29133:
URL: https://github.com/apache/flink/pull/29133#discussion_r3987926034
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/DataStreamTest.java:
##########
@@ -836,6 +837,31 @@ public Long reduce(Long value1, Long value2) throws Exception {
.isEqualTo(preferredResource7);
}
+ @Test
+ void testMapWithExplicitUuidType() {
+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+ // A lambda would bypass the input validation exercised by this test.
+ final MapFunction<UUID, String> mapper =
+ new MapFunction<UUID, String>() {
+ @Override
+ public String map(UUID value) {
+ return value.toString();
+ }
+ };
Review Comment:
Can be simplified to:
```suggestion
final MapFunction<UUID, String> mapper = UUID::toString;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29133:
URL: https://github.com/apache/flink/pull/29133#discussion_r3987922486
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/DataStreamTest.java:
##########
@@ -836,6 +837,31 @@ public Long reduce(Long value1, Long value2) throws Exception {
.isEqualTo(preferredResource7);
}
+ @Test
+ void testMapWithExplicitUuidType() {
+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+ // A lambda would bypass the input validation exercised by this test.
+ final MapFunction<UUID, String> mapper =
+ new MapFunction<UUID, String>() {
+ @Override
+ public String map(UUID value) {
+ return value.toString();
+ }
+ };
Review Comment:
Can the `MapFunction` return the UUID type as `byte[]` or `Byte`? Let's add a test like this
```java
@Test
void testMapToBytesWithExplicitUuidType() {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// A lambda would bypass the input validation exercised by this test.
final MapFunction<UUID, byte[]> byteMapper = value -> {
ByteBuffer buffer = ByteBuffer.wrap(new byte[16]);
buffer.putLong(value.getMostSignificantBits());
buffer.putLong(value.getLeastSignificantBits());
return buffer.array();
};
final DataStream<byte[]> stream = env.fromData(Types.UUID, new UUID(0L, 0L)).map(byteMapper);
assertThat(stream.getType()).isEqualTo(PrimitiveArrayTypeInfo.BYTE_PRIMITIVE_ARRAY_TYPE_INFO);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "dylanhz (via GitHub)" <gi...@apache.org>.
dylanhz commented on PR #29133:
URL: https://github.com/apache/flink/pull/29133#issuecomment-5594380286
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "dylanhz (via GitHub)" <gi...@apache.org>.
dylanhz commented on PR #29133:
URL: https://github.com/apache/flink/pull/29133#issuecomment-5595142022
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "twalthr (via GitHub)" <gi...@apache.org>.
twalthr merged PR #29133:
URL: https://github.com/apache/flink/pull/29133
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29133:
URL: https://github.com/apache/flink/pull/29133#discussion_r3987995353
##########
flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java:
##########
@@ -1498,10 +1499,16 @@ private static void validateInfo(
// check for Java Basic Types
if (typeInfo instanceof BasicTypeInfo) {
- TypeInformation<?> actual;
// check if basic type at all
- if (!(type instanceof Class<?>)
- || (actual = BasicTypeInfo.getInfoFor((Class<?>) type)) == null) {
+ if (!(type instanceof Class<?>)) {
+ throw new InvalidTypesException("Basic type expected.");
+ }
+ // UUID is not registered for automatic extraction to preserve existing serializers.
+ final TypeInformation<?> actual =
+ type == UUID.class
+ ? BasicTypeInfo.UUID_TYPE_INFO
+ : BasicTypeInfo.getInfoFor((Class<?>) type);
Review Comment:
Should we add a test in `TypeExtractorTest`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29133:
URL: https://github.com/apache/flink/pull/29133#discussion_r3987981468
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/DataStreamTest.java:
##########
@@ -836,6 +837,31 @@ public Long reduce(Long value1, Long value2) throws Exception {
.isEqualTo(preferredResource7);
}
+ @Test
+ void testMapWithExplicitUuidType() {
+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+ // A lambda would bypass the input validation exercised by this test.
+ final MapFunction<UUID, String> mapper =
+ new MapFunction<UUID, String>() {
+ @Override
+ public String map(UUID value) {
+ return value.toString();
+ }
+ };
Review Comment:
Or even having a test for Mapping String -> UUID
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "dylanhz (via GitHub)" <gi...@apache.org>.
dylanhz commented on code in PR #29133:
URL: https://github.com/apache/flink/pull/29133#discussion_r4001918601
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/DataStreamTest.java:
##########
@@ -836,6 +837,31 @@ public Long reduce(Long value1, Long value2) throws Exception {
.isEqualTo(preferredResource7);
}
+ @Test
+ void testMapWithExplicitUuidType() {
+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+ // A lambda would bypass the input validation exercised by this test.
+ final MapFunction<UUID, String> mapper =
+ new MapFunction<UUID, String>() {
+ @Override
+ public String map(UUID value) {
+ return value.toString();
+ }
+ };
Review Comment:
UUID → byte[] seems to exercise the same input validation as UUID → String, while String → UUID covers output extraction. Is there a specific scenario you’d like these tests to cover?
##########
flink-core/src/main/java/org/apache/flink/api/java/typeutils/TypeExtractor.java:
##########
@@ -1498,10 +1499,16 @@ private static void validateInfo(
// check for Java Basic Types
if (typeInfo instanceof BasicTypeInfo) {
- TypeInformation<?> actual;
// check if basic type at all
- if (!(type instanceof Class<?>)
- || (actual = BasicTypeInfo.getInfoFor((Class<?>) type)) == null) {
+ if (!(type instanceof Class<?>)) {
+ throw new InvalidTypesException("Basic type expected.");
+ }
+ // UUID is not registered for automatic extraction to preserve existing serializers.
+ final TypeInformation<?> actual =
+ type == UUID.class
+ ? BasicTypeInfo.UUID_TYPE_INFO
+ : BasicTypeInfo.getInfoFor((Class<?>) type);
Review Comment:
Agreed. Moved both tests to `TypeExtractorTest`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "raminqaf (via GitHub)" <gi...@apache.org>.
raminqaf commented on code in PR #29133:
URL: https://github.com/apache/flink/pull/29133#discussion_r3987922486
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/DataStreamTest.java:
##########
@@ -836,6 +837,31 @@ public Long reduce(Long value1, Long value2) throws Exception {
.isEqualTo(preferredResource7);
}
+ @Test
+ void testMapWithExplicitUuidType() {
+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+ // A lambda would bypass the input validation exercised by this test.
+ final MapFunction<UUID, String> mapper =
+ new MapFunction<UUID, String>() {
+ @Override
+ public String map(UUID value) {
+ return value.toString();
+ }
+ };
Review Comment:
Can the `MapFunction` return the UUID type as `byte[]` or `Byte`?
```suggestion
final MapFunction<UUID, byte[]> byteMapper = new MapFunction<UUID, byte[]>() {
@Override
public byte[] map(final UUID value) throws Exception {
ByteBuffer buffer = ByteBuffer.wrap(new byte[16]);
buffer.putLong(value.getMostSignificantBits());
buffer.putLong(value.getLeastSignificantBits());
return buffer.array();
}
};
```
##########
flink-streaming-java/src/test/java/org/apache/flink/streaming/api/DataStreamTest.java:
##########
@@ -836,6 +837,31 @@ public Long reduce(Long value1, Long value2) throws Exception {
.isEqualTo(preferredResource7);
}
+ @Test
+ void testMapWithExplicitUuidType() {
+ final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
+ // A lambda would bypass the input validation exercised by this test.
+ final MapFunction<UUID, String> mapper =
+ new MapFunction<UUID, String>() {
+ @Override
+ public String map(UUID value) {
+ return value.toString();
+ }
+ };
Review Comment:
Can be simplified to:
```suggestion
final MapFunction<UUID, String> mapper = UUID::toString;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-40597][core] Fix input type validation for Types.UUID [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #29133:
URL: https://github.com/apache/flink/pull/29133#issuecomment-5588344939
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "38498754b2d3aa6eee8bcdaa790881bf91d8217a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "38498754b2d3aa6eee8bcdaa790881bf91d8217a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 38498754b2d3aa6eee8bcdaa790881bf91d8217a UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org