You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2020/10/29 09:24:57 UTC
[GitHub] [flink] HuangXingBo opened a new pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
HuangXingBo opened a new pull request #13838:
URL: https://github.com/apache/flink/pull/13838
## What is the purpose of the change
*This pull request will add TypeHints and TypeCheck in Python API*
## Brief change log
- *Add Type hint*
- *Add TypeCheck tool mypy to check typehint*
## Verifying this change
This change added tests and can be verified as follows:
*Add a typehint check in CI*
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? (no)
- If yes, how is the feature documented? (not applicable)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718622856
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589",
"triggerID" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"triggerType" : "PUSH"
}, {
"hash" : "847d1128b89b4e8c625a992253d8bb23886e81eb",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8605",
"triggerID" : "847d1128b89b4e8c625a992253d8bb23886e81eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 847d1128b89b4e8c625a992253d8bb23886e81eb Azure: [SUCCESS](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8605)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu closed pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
dianfu closed pull request #13838:
URL: https://github.com/apache/flink/pull/13838
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-720411610
@dianfu Thanks a lot for the review. I have addressed the comments at the latest commit.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718622856
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589",
"triggerID" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"triggerType" : "PUSH"
}, {
"hash" : "847d1128b89b4e8c625a992253d8bb23886e81eb",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8605",
"triggerID" : "847d1128b89b4e8c625a992253d8bb23886e81eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589)
* 847d1128b89b4e8c625a992253d8bb23886e81eb Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8605)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] dianfu commented on a change in pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
dianfu commented on a change in pull request #13838:
URL: https://github.com/apache/flink/pull/13838#discussion_r516431377
##########
File path: flink-python/pyflink/common/configuration.py
##########
@@ -24,13 +28,12 @@ class Configuration:
Lightweight configuration object which stores key/value pairs.
"""
- def __init__(self, other=None, j_configuration=None):
+ def __init__(self, other: 'Configuration' = None, j_configuration: JavaObject = None):
"""
Creates a new configuration.
:param other: Optional, if this parameter exists, creates a new configuration with a
copy of the given configuration.
- :type other: Configuration
:param j_configuration: Optional, the py4j java configuration object, if this parameter
exists, creates a wrapper for it.
:type j_configuration: py4j.java_gateway.JavaObject
Review comment:
could be removed
##########
File path: flink-python/pyflink/common/execution_config.py
##########
@@ -92,7 +93,7 @@ def disable_closure_cleaner(self):
self._j_execution_config = self._j_execution_config.disableClosureCleaner()
return self
- def is_closure_cleaner_enabled(self):
+ def is_closure_cleaner_enabled(self) -> 'ExecutionConfig':
Review comment:
bool
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] HuangXingBo commented on a change in pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
HuangXingBo commented on a change in pull request #13838:
URL: https://github.com/apache/flink/pull/13838#discussion_r515775411
##########
File path: flink-python/pyflink/common/execution_config.py
##########
@@ -673,7 +674,7 @@ def is_use_snapshot_compression(self):
"""
return self._j_execution_config.isUseSnapshotCompression()
- def set_use_snapshot_compression(self, use_snapshot_compression):
+ def set_use_snapshot_compression(self, use_snapshot_compression: bool):
"""
Review comment:
Yes. Make sense.
##########
File path: flink-python/pyflink/common/execution_config.py
##########
@@ -673,7 +674,7 @@ def is_use_snapshot_compression(self):
"""
return self._j_execution_config.isUseSnapshotCompression()
- def set_use_snapshot_compression(self, use_snapshot_compression):
+ def set_use_snapshot_compression(self, use_snapshot_compression: bool):
"""
Control whether the compression (snappy) for keyed state in full checkpoints and savepoints
Review comment:
The implementation of these methods is aligned with the java side, if the implementation is changed, it may not be convenient for users
##########
File path: flink-python/pyflink/datastream/checkpoint_config.py
##########
@@ -268,15 +268,15 @@ def get_externalized_checkpoint_cleanup(self):
cleanup_mode = self._j_checkpoint_config.getExternalizedCheckpointCleanup()
return ExternalizedCheckpointCleanup._from_j_externalized_checkpoint_cleanup(cleanup_mode)
- def is_unaligned_checkpoints_enabled(self):
+ def is_unaligned_checkpoints_enabled(self) -> bool:
"""
Returns whether unaligned checkpoints are enabled.
:return: ``True`` if unaligned checkpoints are enabled.
"""
return self._j_checkpoint_config.isUnalignedCheckpointsEnabled()
- def enable_unaligned_checkpoints(self, enabled=True):
+ def enable_unaligned_checkpoints(self, enabled: bool = True):
"""
Enables unaligned checkpoints, which greatly reduce checkpointing times under backpressure.
Review comment:
Yes. Make sense
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718622856
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9 UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718622856
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589",
"triggerID" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"triggerType" : "PUSH"
}, {
"hash" : "847d1128b89b4e8c625a992253d8bb23886e81eb",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "847d1128b89b4e8c625a992253d8bb23886e81eb",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9 Azure: [FAILURE](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589)
* 847d1128b89b4e8c625a992253d8bb23886e81eb UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718622856
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589",
"triggerID" : "907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9 Azure: [PENDING](https://dev.azure.com/apache-flink/98463496-1af2-4620-8eab-a2ecc1a2e6fe/_build/results?buildId=8589)
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run travis` re-run the last Travis build
- `@flinkbot run azure` re-run the last Azure build
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot commented on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718538781
Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.
## Automated Checks
Last check on commit 907f4a8c989e52f8d9d82850e9f5fb1b1d4447b9 (Thu Oct 29 09:27:29 UTC 2020)
**Warnings:**
* No documentation files were touched! Remember to keep the Flink docs up to date!
<sub>Mention the bot in a comment to re-run the automated checks.</sub>
## Review Progress
* ❓ 1. The [description] looks good.
* ❓ 2. There is [consensus] that the contribution should go into to Flink.
* ❓ 3. Needs [attention] from.
* ❓ 4. The change fits into the overall [architecture].
* ❓ 5. Overall code [quality] is good.
Please see the [Pull Request Review Guide](https://flink.apache.org/contributing/reviewing-prs.html) for a full explanation of the review process.<details>
The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required <summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot approve description` to approve one or more aspects (aspects: `description`, `consensus`, `architecture` and `quality`)
- `@flinkbot approve all` to approve all aspects
- `@flinkbot approve-until architecture` to approve everything until `architecture`
- `@flinkbot attention @username1 [@username2 ..]` to require somebody's attention
- `@flinkbot disapprove architecture` to remove an approval you gave earlier
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot edited a comment on pull request #13838: [FLINK-16522][python] Add TypeHints and TypeCheck in Python API
Posted by GitBox <gi...@apache.org>.
flinkbot edited a comment on pull request #13838:
URL: https://github.com/apache/flink/pull/13838#issuecomment-718622856
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org