You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "liyubin117 (via GitHub)" <gi...@apache.org> on 2023/12/21 11:44:27 UTC
[PR] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
liyubin117 opened a new pull request, #23980:
URL: https://github.com/apache/flink/pull/23980
## What is the purpose of the change
Cleanup the usage of deprecated org.apache.flink.table.api.TableConfig#getMinIdleStateRetentionTime, org.apache.flink.table.api.TableConfig#getMaxIdleStateRetentionTime.
## Brief change log
use new api.
## Verifying this change
no need additional verify besides CI
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? no
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "xuyangzhong (via GitHub)" <gi...@apache.org>.
xuyangzhong commented on code in PR #23980:
URL: https://github.com/apache/flink/pull/23980#discussion_r1879409013
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/utils/TableConfigUtils.java:
##########
@@ -105,16 +104,6 @@ public static ZoneId getLocalTimeZone(ReadableConfig tableConfig) {
return ZoneId.of(zone);
}
- /**
- * Similar to {@link TableConfig#getMaxIdleStateRetentionTime()}.
- *
- * @see TableConfig#getMaxIdleStateRetentionTime()
- */
- @Deprecated
- public static long getMaxIdleStateRetentionTime(ReadableConfig tableConfig) {
Review Comment:
I believe that this utility method should be retained for the following reasons:
1. Based on the context of [FLINK-16835](https://issues.apache.org/jira/browse/FLINK-16835) and [FLINK-18555](https://issues.apache.org/jira/browse/FLINK-18555), some operators have not fully transitioned to using `StateTtlConfig` for min/max state retention time (e.g. over aggs and temporal joins). Therefore, the min/max state retention time needs to be temporarily retained as an internal variable.
2. Logic like `config.getStateRetentionTime() * 3 / 2` is scattered throughout the code, making it difficult to maintain.
3. Since `TableConfigUtils` is a non-public API, users won't notice its presence, so it's fine to keep it for now.
I would consider deleting this method only if we can completely eliminate the min/max state retention time across all code and fully replace it with `StateTtlConfig`. What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-2535219130
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "xuyangzhong (via GitHub)" <gi...@apache.org>.
xuyangzhong commented on code in PR #23980:
URL: https://github.com/apache/flink/pull/23980#discussion_r1881240233
##########
flink-table/flink-table-api-scala-bridge/src/test/scala/org/apache/flink/table/api/bridge/scala/internal/StreamTableEnvironmentImplTest.scala:
##########
@@ -48,8 +48,9 @@ class StreamTableEnvironmentImplTest {
val table = tEnv.fromDataStream(elements)
tEnv.toDataStream(table)
- assertThat(tEnv.getConfig.getMinIdleStateRetentionTime).isEqualTo(retention.toMillis)
- assertThat(tEnv.getConfig.getMaxIdleStateRetentionTime).isEqualTo(retention.toMillis * 3 / 2)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis).isEqualTo(retention.toMillis)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis * 3 / 2)
Review Comment:
nit: It seems that this assertion can be removed:
```
assertThat(tEnv.getConfig.getIdleStateRetention.toMillis * 3 / 2)...
```
##########
flink-table/flink-table-api-scala-bridge/src/test/scala/org/apache/flink/table/api/bridge/scala/internal/StreamTableEnvironmentImplTest.scala:
##########
@@ -63,8 +64,9 @@ class StreamTableEnvironmentImplTest {
val table = tEnv.fromDataStream(elements)
tEnv.toRetractStream[Row](table)
- assertThat(tEnv.getConfig.getMinIdleStateRetentionTime).isEqualTo(retention.toMillis)
- assertThat(tEnv.getConfig.getMaxIdleStateRetentionTime).isEqualTo(retention.toMillis * 3 / 2)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis).isEqualTo(retention.toMillis)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis * 3 / 2)
Review Comment:
ditto
##########
flink-python/pyflink/table/table_config.py:
##########
@@ -239,7 +239,7 @@ def get_min_idle_state_retention_time(self) -> int:
:return: The minimum time until state which was not updated will be retained.
"""
- return self._j_table_config.getMinIdleStateRetentionTime()
+ return self._j_table_config.getIdleStateRetention().toMillis()
Review Comment:
I think these python methods `get_min_idle_state_retention_time` and `get_max_idle_state_retention_time` can also be removed because they have been tagged `deprecated` in comments. cc @HuangXingBo can you help to double check it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on code in PR #23980:
URL: https://github.com/apache/flink/pull/23980#discussion_r1887951917
##########
flink-table/flink-table-api-scala-bridge/src/test/scala/org/apache/flink/table/api/bridge/scala/internal/StreamTableEnvironmentImplTest.scala:
##########
@@ -48,8 +48,9 @@ class StreamTableEnvironmentImplTest {
val table = tEnv.fromDataStream(elements)
tEnv.toDataStream(table)
- assertThat(tEnv.getConfig.getMinIdleStateRetentionTime).isEqualTo(retention.toMillis)
- assertThat(tEnv.getConfig.getMaxIdleStateRetentionTime).isEqualTo(retention.toMillis * 3 / 2)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis).isEqualTo(retention.toMillis)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis * 3 / 2)
Review Comment:
done :)
##########
flink-table/flink-table-api-scala-bridge/src/test/scala/org/apache/flink/table/api/bridge/scala/internal/StreamTableEnvironmentImplTest.scala:
##########
@@ -63,8 +64,9 @@ class StreamTableEnvironmentImplTest {
val table = tEnv.fromDataStream(elements)
tEnv.toRetractStream[Row](table)
- assertThat(tEnv.getConfig.getMinIdleStateRetentionTime).isEqualTo(retention.toMillis)
- assertThat(tEnv.getConfig.getMaxIdleStateRetentionTime).isEqualTo(retention.toMillis * 3 / 2)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis).isEqualTo(retention.toMillis)
+ assertThat(tEnv.getConfig.getIdleStateRetention.toMillis * 3 / 2)
Review Comment:
done :)
##########
flink-python/pyflink/table/table_config.py:
##########
@@ -239,7 +239,7 @@ def get_min_idle_state_retention_time(self) -> int:
:return: The minimum time until state which was not updated will be retained.
"""
- return self._j_table_config.getMinIdleStateRetentionTime()
+ return self._j_table_config.getIdleStateRetention().toMillis()
Review Comment:
done :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-2535859612
@xuyangzhong CI passed, please take a look :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-1866172964
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on code in PR #23980:
URL: https://github.com/apache/flink/pull/23980#discussion_r1879415457
##########
flink-table/flink-table-planner/src/main/java/org/apache/flink/table/planner/utils/TableConfigUtils.java:
##########
@@ -105,16 +104,6 @@ public static ZoneId getLocalTimeZone(ReadableConfig tableConfig) {
return ZoneId.of(zone);
}
- /**
- * Similar to {@link TableConfig#getMaxIdleStateRetentionTime()}.
- *
- * @see TableConfig#getMaxIdleStateRetentionTime()
- */
- @Deprecated
- public static long getMaxIdleStateRetentionTime(ReadableConfig tableConfig) {
Review Comment:
Wise consideration :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-1868876743
@LadyForest Hi, CI has passed, PTAL, thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "lincoln-lil (via GitHub)" <gi...@apache.org>.
lincoln-lil merged PR #23980:
URL: https://github.com/apache/flink/pull/23980
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-2547577934
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33921][table] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "liyubin117 (via GitHub)" <gi...@apache.org>.
liyubin117 commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-2533634146
@xuyangzhong I have also removed deprecated functions :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Cleanup usage of deprecated IdleStateRetentionTime related method in org.apache.flink.table.api.TableConfig [flink]
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #23980:
URL: https://github.com/apache/flink/pull/23980#issuecomment-1866115231
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "620ea94f2829ebc8c55ab0657261c65f6881054b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "620ea94f2829ebc8c55ab0657261c65f6881054b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 620ea94f2829ebc8c55ab0657261c65f6881054b UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org