You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "lukecwik (via GitHub)" <gi...@apache.org> on 2023/01/20 18:53:20 UTC

[GitHub] [beam] lukecwik opened a new pull request, #25104: [WIP] Migrate portable runners to second version of data channel code

lukecwik opened a new pull request, #25104:
URL: https://github.com/apache/beam/pull/25104

   
   ------------------------
   
   Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
   
    - [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
    - [ ] Update `CHANGES.md` with noteworthy changes.
    - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
   
   To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
   
   GitHub Actions Tests Status (on master branch)
   ------------------------------------------------------------------------------------------------
   [![Build python source distribution and wheels](https://github.com/apache/beam/workflows/Build%20python%20source%20distribution%20and%20wheels/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
   [![Python tests](https://github.com/apache/beam/workflows/Python%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Java tests](https://github.com/apache/beam/workflows/Java%20Tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
   [![Go tests](https://github.com/apache/beam/workflows/Go%20tests/badge.svg?branch=master&event=schedule)](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
   
   See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401050148

   Run Python Spark ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401049739

   Run Python Samza ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on a diff in pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on code in PR #25104:
URL: https://github.com/apache/beam/pull/25104#discussion_r1085704148


##########
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataGrpcMultiplexer2.java:
##########
@@ -253,15 +253,13 @@ public void onError(Throwable t) {
           "Failed to handle for {}",
           apiServiceDescriptor == null ? "unknown endpoint" : apiServiceDescriptor,
           t);
-      outboundObserver.onCompleted();
     }
 
     @Override
     public void onCompleted() {
       LOG.warn(
           "Hanged up for {}.",
           apiServiceDescriptor == null ? "unknown endpoint" : apiServiceDescriptor);
-      outboundObserver.onCompleted();
     }
   }
 }

Review Comment:
   https://github.com/apache/beam/pull/25144 is the follow-up



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1400858857

   Run Go Flink ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401003250

   Run Python Samza ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401011468

   Run Python Samza ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1400859304

   Run Python Flink ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401004688

   Run Java Flink PortableValidatesRunner Streaming


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401005512

   Run Go Spark ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401011139

   Run Go Samza ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401050433

   Run Java Flink PortableValidatesRunner Streaming


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401048958

   Run XVR_Flink PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on a diff in pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on code in PR #25104:
URL: https://github.com/apache/beam/pull/25104#discussion_r1085694824


##########
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataGrpcMultiplexer2.java:
##########
@@ -253,15 +253,13 @@ public void onError(Throwable t) {
           "Failed to handle for {}",
           apiServiceDescriptor == null ? "unknown endpoint" : apiServiceDescriptor,
           t);
-      outboundObserver.onCompleted();
     }
 
     @Override
     public void onCompleted() {
       LOG.warn(
           "Hanged up for {}.",
           apiServiceDescriptor == null ? "unknown endpoint" : apiServiceDescriptor);
-      outboundObserver.onCompleted();
     }
   }
 }

Review Comment:
   Yeah, I wanted to do that in a separate CL since the commit history would be easier to understand.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] y1chi commented on a diff in pull request #25104: Migrate portable runners to second version of data channel code

Posted by "y1chi (via GitHub)" <gi...@apache.org>.
y1chi commented on code in PR #25104:
URL: https://github.com/apache/beam/pull/25104#discussion_r1084709503


##########
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataGrpcMultiplexer2.java:
##########
@@ -253,15 +253,13 @@ public void onError(Throwable t) {
           "Failed to handle for {}",
           apiServiceDescriptor == null ? "unknown endpoint" : apiServiceDescriptor,
           t);
-      outboundObserver.onCompleted();
     }
 
     @Override
     public void onCompleted() {
       LOG.warn(
           "Hanged up for {}.",
           apiServiceDescriptor == null ? "unknown endpoint" : apiServiceDescriptor);
-      outboundObserver.onCompleted();
     }
   }
 }

Review Comment:
   Maybe we can rename this class to just BeamFnDataGrpcMultiplexer with a follow up?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1400858486

   Run XVR_Flink PostCommit


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401049493

   Run Python Flink ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401051434

   Run Go Spark ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401050587

   Run Java Samza PortableValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401005380

   Run Go Samza ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401050855

   Run Java Spark PortableValidatesRunner Batch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401049121

   Run Go Flink ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401051084

   Run Go Samza ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik merged pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik merged PR #25104:
URL: https://github.com/apache/beam/pull/25104


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401004015

   Run Python Spark ValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401005065

   Run Java Spark PortableValidatesRunner Batch


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401004875

   Run Java Samza PortableValidatesRunner


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] lukecwik commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "lukecwik (via GitHub)" <gi...@apache.org>.
lukecwik commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1401140149

   R: @y1chi @apilloud 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [beam] Abacn commented on pull request #25104: Migrate portable runners to second version of data channel code

Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #25104:
URL: https://github.com/apache/beam/pull/25104#issuecomment-1403091314

   python postcommit tests `apache_beam.io.external.xlang_debeziumio_it_test.CrossLanguageDebeziumIOTest.test_xlang_debezium_read` start failing likely related to this change:
   
   https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Python310/393/
   https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Python39/1381/
   https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Python38/3658/
   https://ci-beam.apache.org/view/PostCommit/job/beam_PostCommit_Python37/ (same test failed though not shown in jenkins, can be checked from log)
   
   ```
   Error Message
   RuntimeError: Service failed to start up with error 1
   Stacktrace
   self = <apache_beam.io.external.xlang_debeziumio_it_test.CrossLanguageDebeziumIOTest testMethod=test_xlang_debezium_read>
   ```
   
   looks like it fails to start the expansion service.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org