You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "thebozzcl (via GitHub)" <gi...@apache.org> on 2024/04/19 22:41:30 UTC

[PR] [#30789] Add support for Flink 1.18 [beam]

thebozzcl opened a new pull request, #31062:
URL: https://github.com/apache/beam/pull/31062

   This PR tries to solve #30789 by adding a runner for Flink 1.18. So far I haven't identified any changes that need to be done beyond the version bump.
   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [X] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1575300617


##########
CHANGES.md:
##########
@@ -57,7 +57,7 @@
 
 ## Highlights
 
-* Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. ([#29939](https://github.com/apache/beam/issues/29939))
+* Added FlinkRunner for Flink 1.17 and 1.18, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17 or 1.18, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17 or 1.18. ([#29939](https://github.com/apache/beam/issues/29939))

Review Comment:
   Sure thing!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik merged PR #31062:
URL: https://github.com/apache/beam/pull/31062


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2067417686

   Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment `assign set of reviewers`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2098501153

   Actually committer can "squash and merge" using GitHub website. No further action needed for PR author.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2098610454

   @Abacn Cool, I was not aware of that :+1:
   @thebozzcl Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2097735717

   Looks like this is all green and ready to merge. Can you please squash the commits to single commit? Thanks. :+1: 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1582611127


##########
sdks/typescript/package-lock.json:
##########
@@ -6,7 +6,7 @@
   "packages": {
     "": {
       "name": "apache-beam",
-      "version": "2.54.0-SNAPSHOT",
+      "version": "2.57.0-SNAPSHOT",

Review Comment:
   Strange, there is still this change, which might cause the failure of typescript check?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1583521120


##########
sdks/typescript/package-lock.json:
##########
@@ -6,7 +6,7 @@
   "packages": {
     "": {
       "name": "apache-beam",
-      "version": "2.54.0-SNAPSHOT",
+      "version": "2.57.0-SNAPSHOT",

Review Comment:
   I think I finally understood what happened here: there's two lines with the package version, and I failed to notice the second one.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108552430

   > @je-ik PostRelease test is different from other tests. It downloads the latest Nightly Snapshot and run some validation pipelines. It does not build beam on the current branch, and that is why you see "The test locally fails even before this commit".
   
   @je-ik, @Abacn: Given the task `:runners:flink:1.18:runQuickstartJavaFlinkLocal` is the failed task and it was introduced by this commit, can we revert it and see if the test is fixed first? We can look into other test issues if they emerge after that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108508891

   I took a "quick" look (it took a while to dl all the dependencies in a connection that's not much better than dial-up). I'm kind of baffled by this issue - I checked the `flink-core` library and `org.apache.flink.api.common.ExecutionConfig` hasn't been moved around at all since 1.14.
   
   We should probably create a separate ticket for this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2106103389

   The task `:runners:flink:1.18:runQuickstartJavaFlinkLocal` has failed to run due to this commit, which leads to the failure of our github action "https://github.com/apache/beam/actions/workflows/beam_PostRelease_NightlySnapshot.yml".
   
   Below is the error from the log:
   ```
   2024-05-10T16:28:07.4342882Z Caused by: java.lang.ClassNotFoundException: org.apache.flink.api.common.ExecutionConfig
   2024-05-10T16:28:07.4344273Z 	at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
   2024-05-10T16:28:07.4345556Z 	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
   2024-05-10T16:28:07.4346644Z 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
   2024-05-10T16:28:07.4347805Z 	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
   2024-05-10T16:28:07.4348942Z 	at java.lang.Class.forName0(Native Method)
   2024-05-10T16:28:07.4349661Z 	at java.lang.Class.forName(Class.java:348)
   2024-05-10T16:28:07.4350979Z 	at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:78)
   2024-05-10T16:28:07.4352850Z 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1988)
   2024-05-10T16:28:07.4354599Z 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1852)
   2024-05-10T16:28:07.4356870Z 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2186)
   2024-05-10T16:28:07.4358955Z 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1669)
   2024-05-10T16:28:07.4360298Z 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:503)
   2024-05-10T16:28:07.4362035Z 	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:461)
   2024-05-10T16:28:07.4364329Z 	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:539)
   2024-05-10T16:28:07.4367296Z 	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:527)
   2024-05-10T16:28:07.4368928Z 	at org.apache.flink.util.SerializedValue.deserializeValue(SerializedValue.java:67)
   2024-05-10T16:28:07.4370661Z 	at org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:101)
   2024-05-10T16:28:07.4373136Z 	at org.apache.flink.runtime.jobmaster.DefaultSlotPoolServiceSchedulerFactory.createScheduler(DefaultSlotPoolServiceSchedulerFactory.java:122)
   2024-05-10T16:28:07.4375581Z 	at org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:379)
   2024-05-10T16:28:07.4376991Z 	at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:356)
   2024-05-10T16:28:07.4379145Z 	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.internalCreateJobMasterService(DefaultJobMasterServiceFactory.java:128)
   2024-05-10T16:28:07.4382049Z 	at org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.lambda$createJobMasterService$0(DefaultJobMasterServiceFactory.java:100)
   2024-05-10T16:28:07.4384288Z 	at org.apache.flink.util.function.FunctionUtils.lambda$uncheckedSupplier$4(FunctionUtils.java:112)
   2024-05-10T16:28:07.4385674Z 	... 4 more
   ```
   
   +@damccorm 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1583520189


##########
.github/workflows/beam_PostCommit_Java_Tpcds_Flink.yml:
##########
@@ -101,5 +101,5 @@ jobs:
         with:
           gradle-command: :sdks:java:testing:tpcds:run
           arguments: |
-            -Ptpcds.runner=:runners:flink:1.17 \
+            -Ptpcds.runner=:runners:flink:1.18 \

Review Comment:
   I'm in no rush to get this merged, so I'd be okay with that.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1575286796


##########
CHANGES.md:
##########
@@ -57,7 +57,7 @@
 
 ## Highlights
 
-* Added FlinkRunner for Flink 1.17, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17. ([#29939](https://github.com/apache/beam/issues/29939))
+* Added FlinkRunner for Flink 1.17 and 1.18, removed support for Flink 1.12 and 1.13. Previous version of Pipeline running on Flink 1.16 and below can be upgraded to 1.17 or 1.18, if the Pipeline is first updated to Beam 2.56.0 with the same Flink version. After Pipeline runs with Beam 2.56.0, it should be possible to upgrade to FlinkRunner with Flink 1.17 or 1.18. ([#29939](https://github.com/apache/beam/issues/29939))

Review Comment:
   Beam 2.56.0 is already cut. 1.18 support will go to 2.57.0. Could add a new section in CHANGES.md copy-paste from the template above



##########
sdks/typescript/package-lock.json:
##########
@@ -1,12 +1,12 @@
 {
   "name": "apache-beam",
-  "version": "2.54.0-SNAPSHOT",
+  "version": "2.57.0-SNAPSHOT",

Review Comment:
   Though this is an unrelated change -- should this version bump be automated as part of cut-release-branch script? cc: @robertwb 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2084741655

   Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "shunping (via GitHub)" <gi...@apache.org>.
shunping commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108564178

   Looks like someone have reported this issue on Flink 1.18 before.
   - https://www.mail-archive.com/user@flink.apache.org/msg52035.html
   - https://www.mail-archive.com/user@flink.apache.org/msg52467.html


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2106783575

   I can confirm the failure locally, we should revert this until we can make the check pass.
   
   https://github.com/apache/beam/pull/31262


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2098829137

   @Abacn nice, I wasn't aware of that either! @je-ik thanks for taking care of it!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1580788419


##########
sdks/typescript/package-lock.json:
##########
@@ -1,12 +1,12 @@
 {
   "name": "apache-beam",
-  "version": "2.54.0-SNAPSHOT",
+  "version": "2.57.0-SNAPSHOT",

Review Comment:
   +1 to remove this unrelated change. We can merge this afterwards.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2084739314

   R: @je-ik 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2070828310

   Thanks for the work! Yeah the flaky go SDK test is irrelevant


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108528878

   @je-ik PostRelease test is different from other tests. It downloads the latest Nightly Snapshot and run some validation pipelines. It does not build beam on the current branch, and that is why you see "The test locally fails even before this commit".


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108623504

   I agree revert at the change moment
   
   `:runners:flink:1.17:runQuickstartJavaFlinkLocal` passed on Beam 2.56.0:
   
   - https://github.com/apache/beam/actions/workflows/beam_PostRelease_NightlySnapshot.yml
   
   `:runners:flink:1.17:runQuickstartJavaFlinkLocal` (as well as `:1.18:runQuickstartJavaFlinkLocal`) failed on master:
   
   - https://github.com/apache/beam/actions/runs/9068053305
   
   This means current master branch is failing Flink runner validation for both Flink 1.18 and under 1.18, possibly caused by this PR.
   
   If we revert the change and wait for another nightly snapshot built, and the PostRelease test is back green, we can confirm
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2106319331

   Hmm. I'm gonna try looking into it, but I have limited connectivity at the moment and might not be able to do much until this week. What's the preferred way to deal with this? Do we want to roll back?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "kennknowles (via GitHub)" <gi...@apache.org>.
kennknowles commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2176910219

   Just for posterity and everyone else getting notifications. As release manager I learned just now that we need a tickets such as this whenever we add a new version: https://issues.apache.org/jira/browse/INFRA-25880
   
   These can take a few days to get resolved, so it is good to do it right away as part of adding the new version support.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1581344167


##########
sdks/typescript/package-lock.json:
##########
@@ -1,12 +1,12 @@
 {
   "name": "apache-beam",
-  "version": "2.54.0-SNAPSHOT",
+  "version": "2.57.0-SNAPSHOT",

Review Comment:
   I'll remove it, but I didn't change it myself... won't it just get bumped again automatically?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2084714189

   Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
   
   R: @jrmccluskey for label go.
   R: @Abacn for label build.
   
   Available commands:
   - `stop reviewer notifications` - opt out of the automated review tooling
   - `remind me after tests pass` - tag the comment author after tests pass
   - `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
   
   The PR bot will only process comments in the main thread (not review comments).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2070816459

   As I thought, seems there's a flaky test: https://github.com/apache/beam/actions/runs/8760406645/job/24045305260
   
   ```
    --- FAIL: TestElementChan (0.00s)
       --- FAIL: TestElementChan/FillBufferThenAbortThenRead (0.00s)
           datamgr_test.go:412: got sum 5, count 5, want sum 20, count 20
   FAIL
   ```
   
   I'm gonna take a look at it to see if I can identify what caused the previous run to fail. That being said, Go is not my forte.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108626219

   if revert, there is another PR need to be reverted at the same time : #31217
   
   This one included a typo fix which shouldn't be reverted : https://github.com/apache/beam/pull/31217/files#diff-07d5f4d101410e8cf42b7241fe72074d3c8003866e2b077f51388e1283ebe442R337


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2106868372

   Correction. The test locally fails even before this commit (e.g. tested `49056fd7ed111c4c9ebb9236ab447d39f0aa86f3`). The check seems to have independent issues.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1581561448


##########
.github/workflows/beam_PostCommit_Java_Tpcds_Flink.yml:
##########
@@ -101,5 +101,5 @@ jobs:
         with:
           gradle-command: :sdks:java:testing:tpcds:run
           arguments: |
-            -Ptpcds.runner=:runners:flink:1.17 \
+            -Ptpcds.runner=:runners:flink:1.18 \

Review Comment:
   Sorry for not putting comments at once, but just realize that these github action changes will apply to both the master branch and the ongoing release process, but the current pending release-2.56.0 branch do not have `:runners:flink:1.18` project which will causing issue
   
   To avoid issue we can leave .github/ content as is



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "je-ik (via GitHub)" <gi...@apache.org>.
je-ik commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1582609876


##########
.github/workflows/beam_PostCommit_Java_Tpcds_Flink.yml:
##########
@@ -101,5 +101,5 @@ jobs:
         with:
           gradle-command: :sdks:java:testing:tpcds:run
           arguments: |
-            -Ptpcds.runner=:runners:flink:1.17 \
+            -Ptpcds.runner=:runners:flink:1.18 \

Review Comment:
   We might merge this after the release of 2.56.0?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "thebozzcl (via GitHub)" <gi...@apache.org>.
thebozzcl commented on code in PR #31062:
URL: https://github.com/apache/beam/pull/31062#discussion_r1581362391


##########
sdks/typescript/package-lock.json:
##########
@@ -1,12 +1,12 @@
 {
   "name": "apache-beam",
-  "version": "2.54.0-SNAPSHOT",
+  "version": "2.57.0-SNAPSHOT",

Review Comment:
   Odd, I pushed a commit but it's not showing directly in the PR's file changes. It's reflected properly in my branch and in the commit's changes, though.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2177226106

   A proposal to use single dockerhub repository for Flink job server container: #31631 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [#30789] Add support for Flink 1.18 [beam]

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #31062:
URL: https://github.com/apache/beam/pull/31062#issuecomment-2108629163

   Checking this PR again, it does not include non-trivial change. I doubt that revert it will make the flinkQuickstart back green again though


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org