You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "gaborgsomogyi (via GitHub)" <gi...@apache.org> on 2023/09/14 09:11:45 UTC
[GitHub] [flink] gaborgsomogyi opened a new pull request, #23417: [FLINK-33030][python]Add python 3.11 support
gaborgsomogyi opened a new pull request, #23417:
URL: https://github.com/apache/flink/pull/23417
## What is the purpose of the change
At the moment python 3.11 is not supported. In this PR I've added this support.
## Brief change log
Added python 3.11 support.
## Verifying this change
* Existing python tests.
* Manually
```
dev/build-wheels.sh
```
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1763912555
When either the proto file or the tool version is changed then protobuf generation is required. The related howto can be found in the following place: https://github.com/apache/flink/blob/master/flink-python/README.md#protobuf-code-generation
Since the `grpcio-tools==1.50.0` has different parameter set and generates class files in an interface file it must be re-generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1762818432
Finally there is a green result including wheel builds: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53722&view=results
Reverting the azure hack agains and the PR is ready to review/merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1761097474
Fixed the notice file + reverted the temp azure hack to test wheel building.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1719081751
Facing the same `not able to test the Azure jobs` issue here...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1761088001
Wheel builds passed here: https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=53694&view=results
<img width="235" alt="Screenshot 2023-10-13 at 10 07 24" src="https://github.com/apache/flink/assets/18561820/085375f0-0990-4f5a-bc65-b87cce137a5c">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1761374491
`grpcio-tools` bump is also required so put back the wheel building test code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1759471969
@flinkbot run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] flinkbot commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "flinkbot (via GitHub)" <gi...@apache.org>.
flinkbot commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1719070154
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "721f303875656d69ae110b0e0cfb9d12ed40c2de",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "721f303875656d69ae110b0e0cfb9d12ed40c2de",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 721f303875656d69ae110b0e0cfb9d12ed40c2de UNKNOWN
<details>
<summary>Bot commands</summary>
The @flinkbot bot supports the following commands:
- `@flinkbot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1720944111
@HuangXingBo can the mentioned missing files added or new release is required?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1759282897
Thanks for the help!
Rebased to the latest master + bumped the version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1719424923
@HuangXingBo I've just double checked and seems like the maven central pemja is still showing `0.3.1` as latest:
https://central.sonatype.com/artifact/com.alibaba/pemja
I presume we need this since Flink uses this as dependency, right?
https://github.com/apache/flink/blob/c7f6470bb8cc314e7651b03e171af057f4edec1e/flink-python/pom.xml#L126-L132
Could you please help to release the new version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1719080987
cc @gyfora @mbalassi @morhidi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1720710015
Just added wheel building debug code to the PR which must be reverted back when Azure passed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1720942470
I've just analyzed through why `pemja 0.4.0` is not found and the reason is that:
* `0.3.0` has pom inside: https://repo.maven.apache.org/maven2/com/alibaba/pemja/0.3.0/
<img width="616" alt="Screenshot 2023-09-15 at 11 05 56" src="https://github.com/apache/flink/assets/18561820/19149d2b-6aeb-49db-9fe9-46b151615aa4">
* `0.4.0` has `no` pom inside: https://repo.maven.apache.org/maven2/com/alibaba/pemja/0.4.0/
<img width="612" alt="Screenshot 2023-09-15 at 11 06 36" src="https://github.com/apache/flink/assets/18561820/c1229b6e-71c7-4c14-88b9-d3cf27362e5f">
The end result is the following failure:
`06:13:53.385 [ERROR] Failed to execute goal on project flink-python: Could not resolve dependencies for project org.apache.flink:flink-python:jar:1.19-SNAPSHOT: Could not find artifact com.alibaba:pemja:jar:0.4.0 in alicloud-mvn-mirror (http://172.17.0.1:8888/repository/maven-central/) -> [Help 1]`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "HuangXingBo (via GitHub)" <gi...@apache.org>.
HuangXingBo commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1759117083
Sorry for the late reply. It's been a holiday in China for the past few days. I'll solve the problem of pemja asap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1740951700
Rebased to the latest master.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1763037841
We're good to go :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "mbalassi (via GitHub)" <gi...@apache.org>.
mbalassi merged PR #23417:
URL: https://github.com/apache/flink/pull/23417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [FLINK-33030][python]Add python 3.11 support [flink]
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on PR #23417:
URL: https://github.com/apache/flink/pull/23417#issuecomment-1763912833
Thanks for taking care!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink] gaborgsomogyi commented on a diff in pull request #23417: [FLINK-33030][python]Add python 3.11 support
Posted by "gaborgsomogyi (via GitHub)" <gi...@apache.org>.
gaborgsomogyi commented on code in PR #23417:
URL: https://github.com/apache/flink/pull/23417#discussion_r1325644523
##########
tools/releasing/create_binary_release.sh:
##########
@@ -129,8 +129,8 @@ make_python_release() {
cp ${pyflink_actual_name} "${PYTHON_RELEASE_DIR}/${pyflink_release_name}"
wheel_packages_num=0
- # py38,py39,py310 for mac 10.9, 11.0 and linux (9 wheel packages)
- EXPECTED_WHEEL_PACKAGES_NUM=9
+ # py38,py39,py310,py311 for mac 10.9, 11.0 and linux (12 wheel packages)
+ EXPECTED_WHEEL_PACKAGES_NUM=12
Review Comment:
+3 because mac 10.9, mac 11.0 and linux
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org