You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "snuyanzin (via GitHub)" <gi...@apache.org> on 2026/10/17 22:57:17 UTC
[PR] [FLINK-41017][table] Use `DecimalData#isCompact` while casting [flink]
snuyanzin opened a new pull request, #29507:
URL: https://github.com/apache/flink/pull/29507
## What is the purpose of the change
The PR makes using `DecimalData#isCompact` while casting
which speeds up casting for DECIMAL (compact)
## Brief change log
DecimalData
## Verifying this change
existing tests + extra tests added
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): ( no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: ( no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? ( no)
- If yes, how is the feature documented? (not applicable)
---
##### Was generative AI tooling used to co-author this PR?
<!--
If generative AI tooling has been used in the process of authoring this PR, please
change the checkbox below to `[X]` and replace the placeholder in the "Generated-by"
line with the tool name and version. Otherwise remove the "Generated-by" line.
See the ASF Generative Tooling Guidance for details:
https://www.apache.org/legal/generative-tooling.html
You are responsible for the quality and correctness of every change in this PR
regardless of the tooling used. Low-effort AI-generated PRs will be closed. See
AGENTS.md for the full guidance.
-->
- [ ] Yes (please specify the tool below)
Generated-by: [Tool Name and Version]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-41017][table] Use `DecimalData#isCompact` while casting [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #29507:
URL: https://github.com/apache/flink/pull/29507#issuecomment-6242747211
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e43a260c6c07b6313909a465f480ef7e8b4e21b2",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e43a260c6c07b6313909a465f480ef7e8b4e21b2",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e43a260c6c07b6313909a465f480ef7e8b4e21b2 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-41017][table] Use `DecimalData#isCompact` while casting [flink]
Posted by "RocMarshal (via GitHub)" <gi...@apache.org>.
RocMarshal commented on code in PR #29507:
URL: https://github.com/apache/flink/pull/29507#discussion_r4305876834
##########
flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/data/DecimalDataTest.java:
##########
@@ -45,10 +49,77 @@
import static org.apache.flink.table.data.DecimalDataUtils.sround;
import static org.apache.flink.table.data.DecimalDataUtils.subtract;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
/** Test for {@link DecimalData}. */
class DecimalDataTest {
+ private static Stream<Arguments> compactCastCases() {
+ return Stream.of(
+ // unscaled, srcPrecision, srcScale, targetPrecision, targetScale
+ arguments(0L, 5, 2, 3, 1), // zero
+ arguments(123L, 5, 2, 6, 2), // equal scale
+ arguments(12345L, 5, 0, 10, 2), // exact scale-up
+ arguments(24L, 2, 1, 2, 0), // scale-down, round half down
+ arguments(25L, 2, 1, 2, 0), // scale-down, round half up (+)
+ arguments(-25L, 2, 1, 2, 0), // scale-down, round half up (-)
+ arguments(995L, 3, 2, 2, 1), // rounds into precision overflow -> null
+ arguments(-995L, 3, 2, 2, 1), // rounds into precision overflow -> null (-)
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 2), // long-overflow scale-up -> null
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary
+ arguments(-999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary (-)
+ arguments(12345L, 5, 2, 3, 2)); // value exceeds target precision -> null
+ }
+
+ private static Stream<Arguments> longCastCases() {
+ return Stream.of(
+ // val, targetPrecision, targetScale
+ arguments(0L, 3, 1),
+ arguments(123L, 5, 2), // exact scale-up
+ arguments(-123L, 5, 2),
+ arguments(Long.MAX_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(Long.MIN_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(999_999_999_999_999_999L, 18, 0), // max 18-digit boundary
+ arguments(99L, 2, 1)); // scale-up overflow of precision -> null
+ }
+
+ @ParameterizedTest(name = "{0} -> DECIMAL({1},{2})")
+ @MethodSource("longCastCases")
+ void testCastFromLongCompactMatchesBigDecimal(long val, int targetPrecision, int targetScale) {
+ final DecimalData expected =
+ DecimalData.fromBigDecimal(BigDecimal.valueOf(val), targetPrecision, targetScale);
+ final DecimalData actual = castFrom(val, targetPrecision, targetScale);
+
+ if (expected == null) {
+ assertThat(actual).isNull();
+ } else {
+ assertThat(actual).isNotNull();
+ assertThat(actual.precision()).isEqualTo(expected.precision());
+ assertThat(actual.scale()).isEqualTo(expected.scale());
+ assertThat(actual.toBigDecimal()).isEqualTo(expected.toBigDecimal());
+ }
Review Comment:
anchor-A
##########
flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/data/DecimalDataTest.java:
##########
@@ -45,10 +49,77 @@
import static org.apache.flink.table.data.DecimalDataUtils.sround;
import static org.apache.flink.table.data.DecimalDataUtils.subtract;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
/** Test for {@link DecimalData}. */
class DecimalDataTest {
+ private static Stream<Arguments> compactCastCases() {
+ return Stream.of(
+ // unscaled, srcPrecision, srcScale, targetPrecision, targetScale
+ arguments(0L, 5, 2, 3, 1), // zero
+ arguments(123L, 5, 2, 6, 2), // equal scale
+ arguments(12345L, 5, 0, 10, 2), // exact scale-up
+ arguments(24L, 2, 1, 2, 0), // scale-down, round half down
+ arguments(25L, 2, 1, 2, 0), // scale-down, round half up (+)
+ arguments(-25L, 2, 1, 2, 0), // scale-down, round half up (-)
+ arguments(995L, 3, 2, 2, 1), // rounds into precision overflow -> null
+ arguments(-995L, 3, 2, 2, 1), // rounds into precision overflow -> null (-)
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 2), // long-overflow scale-up -> null
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary
+ arguments(-999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary (-)
+ arguments(12345L, 5, 2, 3, 2)); // value exceeds target precision -> null
+ }
+
+ private static Stream<Arguments> longCastCases() {
+ return Stream.of(
+ // val, targetPrecision, targetScale
+ arguments(0L, 3, 1),
+ arguments(123L, 5, 2), // exact scale-up
+ arguments(-123L, 5, 2),
+ arguments(Long.MAX_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(Long.MIN_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(999_999_999_999_999_999L, 18, 0), // max 18-digit boundary
+ arguments(99L, 2, 1)); // scale-up overflow of precision -> null
+ }
+
+ @ParameterizedTest(name = "{0} -> DECIMAL({1},{2})")
+ @MethodSource("longCastCases")
+ void testCastFromLongCompactMatchesBigDecimal(long val, int targetPrecision, int targetScale) {
+ final DecimalData expected =
+ DecimalData.fromBigDecimal(BigDecimal.valueOf(val), targetPrecision, targetScale);
+ final DecimalData actual = castFrom(val, targetPrecision, targetScale);
+
+ if (expected == null) {
+ assertThat(actual).isNull();
+ } else {
+ assertThat(actual).isNotNull();
+ assertThat(actual.precision()).isEqualTo(expected.precision());
+ assertThat(actual.scale()).isEqualTo(expected.scale());
+ assertThat(actual.toBigDecimal()).isEqualTo(expected.toBigDecimal());
+ }
+ }
+
+ @ParameterizedTest(name = "unscaled={0} DECIMAL({1},{2}) -> DECIMAL({3},{4})")
+ @MethodSource("compactCastCases")
+ void testCastToDecimalCompactMatchesBigDecimal(
+ long unscaled, int srcPrecision, int srcScale, int targetPrecision, int targetScale) {
+ final DecimalData src = DecimalData.fromUnscaledLong(unscaled, srcPrecision, srcScale);
+ final DecimalData expected =
+ DecimalData.fromBigDecimal(src.toBigDecimal(), targetPrecision, targetScale);
+ final DecimalData actual = castToDecimal(src, targetPrecision, targetScale);
+
+ if (expected == null) {
+ assertThat(actual).isNull();
+ } else {
+ assertThat(actual).isNotNull();
+ assertThat(actual.precision()).isEqualTo(expected.precision());
+ assertThat(actual.scale()).isEqualTo(expected.scale());
+ // BigDecimal.equals is scale-sensitive, so this pins value *and* scale
+ assertThat(actual.toBigDecimal()).isEqualTo(expected.toBigDecimal());
+ }
Review Comment:
Just a nit comment.
May the common lines be extracted into a private method to reduce the redundant lines(anchor-A) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-41017][table] Use `DecimalData#isCompact` while casting [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin merged PR #29507:
URL: https://github.com/apache/flink/pull/29507
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-41017][table] Use `DecimalData#isCompact` while casting [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on code in PR #29507:
URL: https://github.com/apache/flink/pull/29507#discussion_r4306237247
##########
flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/data/DecimalDataTest.java:
##########
@@ -45,10 +49,77 @@
import static org.apache.flink.table.data.DecimalDataUtils.sround;
import static org.apache.flink.table.data.DecimalDataUtils.subtract;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
/** Test for {@link DecimalData}. */
class DecimalDataTest {
+ private static Stream<Arguments> compactCastCases() {
+ return Stream.of(
+ // unscaled, srcPrecision, srcScale, targetPrecision, targetScale
+ arguments(0L, 5, 2, 3, 1), // zero
+ arguments(123L, 5, 2, 6, 2), // equal scale
+ arguments(12345L, 5, 0, 10, 2), // exact scale-up
+ arguments(24L, 2, 1, 2, 0), // scale-down, round half down
+ arguments(25L, 2, 1, 2, 0), // scale-down, round half up (+)
+ arguments(-25L, 2, 1, 2, 0), // scale-down, round half up (-)
+ arguments(995L, 3, 2, 2, 1), // rounds into precision overflow -> null
+ arguments(-995L, 3, 2, 2, 1), // rounds into precision overflow -> null (-)
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 2), // long-overflow scale-up -> null
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary
+ arguments(-999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary (-)
+ arguments(12345L, 5, 2, 3, 2)); // value exceeds target precision -> null
+ }
+
+ private static Stream<Arguments> longCastCases() {
+ return Stream.of(
+ // val, targetPrecision, targetScale
+ arguments(0L, 3, 1),
+ arguments(123L, 5, 2), // exact scale-up
+ arguments(-123L, 5, 2),
+ arguments(Long.MAX_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(Long.MIN_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(999_999_999_999_999_999L, 18, 0), // max 18-digit boundary
+ arguments(99L, 2, 1)); // scale-up overflow of precision -> null
+ }
+
+ @ParameterizedTest(name = "{0} -> DECIMAL({1},{2})")
+ @MethodSource("longCastCases")
+ void testCastFromLongCompactMatchesBigDecimal(long val, int targetPrecision, int targetScale) {
+ final DecimalData expected =
+ DecimalData.fromBigDecimal(BigDecimal.valueOf(val), targetPrecision, targetScale);
+ final DecimalData actual = castFrom(val, targetPrecision, targetScale);
+
+ if (expected == null) {
+ assertThat(actual).isNull();
+ } else {
+ assertThat(actual).isNotNull();
+ assertThat(actual.precision()).isEqualTo(expected.precision());
+ assertThat(actual.scale()).isEqualTo(expected.scale());
+ assertThat(actual.toBigDecimal()).isEqualTo(expected.toBigDecimal());
+ }
+ }
+
+ @ParameterizedTest(name = "unscaled={0} DECIMAL({1},{2}) -> DECIMAL({3},{4})")
+ @MethodSource("compactCastCases")
+ void testCastToDecimalCompactMatchesBigDecimal(
+ long unscaled, int srcPrecision, int srcScale, int targetPrecision, int targetScale) {
+ final DecimalData src = DecimalData.fromUnscaledLong(unscaled, srcPrecision, srcScale);
+ final DecimalData expected =
+ DecimalData.fromBigDecimal(src.toBigDecimal(), targetPrecision, targetScale);
+ final DecimalData actual = castToDecimal(src, targetPrecision, targetScale);
+
+ if (expected == null) {
+ assertThat(actual).isNull();
+ } else {
+ assertThat(actual).isNotNull();
+ assertThat(actual.precision()).isEqualTo(expected.precision());
+ assertThat(actual.scale()).isEqualTo(expected.scale());
+ // BigDecimal.equals is scale-sensitive, so this pins value *and* scale
+ assertThat(actual.toBigDecimal()).isEqualTo(expected.toBigDecimal());
+ }
Review Comment:
extracted to one method
##########
flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/data/DecimalDataTest.java:
##########
@@ -45,10 +49,77 @@
import static org.apache.flink.table.data.DecimalDataUtils.sround;
import static org.apache.flink.table.data.DecimalDataUtils.subtract;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.junit.jupiter.params.provider.Arguments.arguments;
/** Test for {@link DecimalData}. */
class DecimalDataTest {
+ private static Stream<Arguments> compactCastCases() {
+ return Stream.of(
+ // unscaled, srcPrecision, srcScale, targetPrecision, targetScale
+ arguments(0L, 5, 2, 3, 1), // zero
+ arguments(123L, 5, 2, 6, 2), // equal scale
+ arguments(12345L, 5, 0, 10, 2), // exact scale-up
+ arguments(24L, 2, 1, 2, 0), // scale-down, round half down
+ arguments(25L, 2, 1, 2, 0), // scale-down, round half up (+)
+ arguments(-25L, 2, 1, 2, 0), // scale-down, round half up (-)
+ arguments(995L, 3, 2, 2, 1), // rounds into precision overflow -> null
+ arguments(-995L, 3, 2, 2, 1), // rounds into precision overflow -> null (-)
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 2), // long-overflow scale-up -> null
+ arguments(999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary
+ arguments(-999_999_999_999_999_999L, 18, 0, 18, 0), // max 18-digit boundary (-)
+ arguments(12345L, 5, 2, 3, 2)); // value exceeds target precision -> null
+ }
+
+ private static Stream<Arguments> longCastCases() {
+ return Stream.of(
+ // val, targetPrecision, targetScale
+ arguments(0L, 3, 1),
+ arguments(123L, 5, 2), // exact scale-up
+ arguments(-123L, 5, 2),
+ arguments(Long.MAX_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(Long.MIN_VALUE, 18, 0), // 19 digits, exceeds precision -> null
+ arguments(999_999_999_999_999_999L, 18, 0), // max 18-digit boundary
+ arguments(99L, 2, 1)); // scale-up overflow of precision -> null
+ }
+
+ @ParameterizedTest(name = "{0} -> DECIMAL({1},{2})")
+ @MethodSource("longCastCases")
+ void testCastFromLongCompactMatchesBigDecimal(long val, int targetPrecision, int targetScale) {
+ final DecimalData expected =
+ DecimalData.fromBigDecimal(BigDecimal.valueOf(val), targetPrecision, targetScale);
+ final DecimalData actual = castFrom(val, targetPrecision, targetScale);
+
+ if (expected == null) {
+ assertThat(actual).isNull();
+ } else {
+ assertThat(actual).isNotNull();
+ assertThat(actual.precision()).isEqualTo(expected.precision());
+ assertThat(actual.scale()).isEqualTo(expected.scale());
+ assertThat(actual.toBigDecimal()).isEqualTo(expected.toBigDecimal());
+ }
Review Comment:
reduced amount of comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-41017][table] Use `DecimalData#isCompact` while casting [flink]
Posted by "snuyanzin (via GitHub)" <gi...@apache.org>.
snuyanzin commented on PR #29507:
URL: https://github.com/apache/flink/pull/29507#issuecomment-6248342027
thanks for taking a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org