You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/09/08 21:25:15 UTC
[GitHub] [beam] chamikaramj opened a new pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
chamikaramj opened a new pull request #15484:
URL: https://github.com/apache/beam/pull/15484
This adds support for invoking Java transforms from Python.
Proto and Java updates were already added in https://github.com/apache/beam/pull/15343
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
`ValidatesRunner` compliance status (on master branch)
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/badge/icon?subject=Portable">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
</tbody>
</table>
Examples testing status on various runners
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
Post-Commit SDK/Transform Integration Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>Go</th>
<th>Java</th>
<th>Python</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
</a>
</td>
</tr>
</tbody>
</table>
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>---</th>
<th>Java</th>
<th>Python</th>
<th>Go</th>
<th>Website</th>
<th>Whitespace</th>
<th>Typescript</th>
</tr>
</thead>
<tbody>
<tr>
<td>Non-portable</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Portable</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f58547a) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **decrease** coverage by `0.00%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.74% 83.74% -0.01%
==========================================
Files 443 443
Lines 60077 60128 +51
==========================================
+ Hits 50312 50354 +42
- Misses 9765 9774 +9
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.37% <0.00%> (-4.66%)` | :arrow_down: |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.65% <0.00%> (-1.87%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (ø)` | |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...f58547a](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-916270745
CC: @robertwb
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f58547a) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **decrease** coverage by `0.00%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.74% 83.74% -0.01%
==========================================
Files 443 443
Lines 60077 60128 +51
==========================================
+ Hits 50312 50354 +42
- Misses 9765 9774 +9
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.37% <0.00%> (-4.66%)` | :arrow_down: |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.65% <0.00%> (-1.87%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (ø)` | |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...f58547a](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f50e89) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.02%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.78% -0.03%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50483 +91
- Misses 9736 9772 +36
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...7f50e89](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3808ea1) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **increase** coverage by `0.00%`.
> The diff coverage is `88.23%`.
> :exclamation: Current head 3808ea1 differs from pull request most recent head c79eeed. Consider uploading reports for the commit c79eeed to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
=======================================
Coverage 83.74% 83.75%
=======================================
Files 443 443
Lines 60077 60136 +59
=======================================
+ Hits 50312 50367 +55
- Misses 9765 9769 +4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...he\_beam/runners/interactive/sql/beam\_sql\_magics.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvYmVhbV9zcWxfbWFnaWNzLnB5) | `54.70% <0.00%> (-0.96%)` | :arrow_down: |
| [.../python/apache\_beam/portability/api/metrics\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyLnB5) | `100.00% <0.00%> (ø)` | |
| [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...n/apache\_beam/runners/interactive/cache\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoZV9tYW5hZ2VyLnB5) | `90.84% <0.00%> (ø)` | |
| [...eam/runners/interactive/caching/streaming\_cache.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL3N0cmVhbWluZ19jYWNoZS5weQ==) | `95.35% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.51% <0.00%> (+0.24%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...c79eeed](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r714119599
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Will send you a PR on this and we discuss in list as well if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c79eeed) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **increase** coverage by `0.09%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
+ Coverage 83.74% 83.84% +0.09%
==========================================
Files 443 443
Lines 60077 60386 +309
==========================================
+ Hits 50312 50630 +318
+ Misses 9765 9756 -9
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...he\_beam/runners/interactive/sql/beam\_sql\_magics.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvYmVhbV9zcWxfbWFnaWNzLnB5) | `54.70% <0.00%> (-0.96%)` | :arrow_down: |
| [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/gcp/bigquery.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5LnB5) | `75.53% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.81% <0.00%> (ø)` | |
| [.../python/apache\_beam/portability/api/metrics\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyLnB5) | `100.00% <0.00%> (ø)` | |
| [...hon/apache\_beam/portability/api/beam\_fn\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZm5fYXBpX3BiMi5weQ==) | `100.00% <0.00%> (ø)` | |
| [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...n/apache\_beam/runners/interactive/cache\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoZV9tYW5hZ2VyLnB5) | `90.84% <0.00%> (ø)` | |
| ... and [10 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...c79eeed](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-923275667
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f58547a) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **decrease** coverage by `0.00%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.74% 83.74% -0.01%
==========================================
Files 443 443
Lines 60077 60128 +51
==========================================
+ Hits 50312 50354 +42
- Misses 9765 9774 +9
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.37% <0.00%> (-4.66%)` | :arrow_down: |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.65% <0.00%> (-1.87%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (ø)` | |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...f58547a](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r710671611
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Added support for sending arbitrary field names through a well defined pattern that will be ignored.
Added support for positional args in Python side using above.
Updated Python documentation to clarify that args and kwargs for constructor/builder methods will be applied in the specified order.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6947fa2) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50479 +87
- Misses 9736 9776 +40
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c79eeed) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **increase** coverage by `0.03%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
+ Coverage 83.74% 83.77% +0.03%
==========================================
Files 443 443
Lines 60077 60170 +93
==========================================
+ Hits 50312 50408 +96
+ Misses 9765 9762 -3
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...he\_beam/runners/interactive/sql/beam\_sql\_magics.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvYmVhbV9zcWxfbWFnaWNzLnB5) | `54.70% <0.00%> (-0.96%)` | :arrow_down: |
| [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/gcp/bigquery.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5LnB5) | `75.53% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.81% <0.00%> (ø)` | |
| [.../python/apache\_beam/portability/api/metrics\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyLnB5) | `100.00% <0.00%> (ø)` | |
| [...hon/apache\_beam/portability/api/beam\_fn\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZm5fYXBpX3BiMi5weQ==) | `100.00% <0.00%> (ø)` | |
| [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...n/apache\_beam/runners/interactive/cache\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoZV9tYW5hZ2VyLnB5) | `90.84% <0.00%> (ø)` | |
| ... and [7 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...c79eeed](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-923137804
Robert, I believe I addressed your comments but please let me know if there is anything else.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (663c221) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `89.39%`.
> :exclamation: Current head 663c221 differs from pull request most recent head 7f50e89. Consider uploading reports for the commit 7f50e89 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50480 +88
- Misses 9736 9775 +39
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.38% <89.06%> (+2.98%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.24% <0.00%> (-4.88%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...7f50e89](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f50e89) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.02%`.
> The diff coverage is `90.90%`.
> :exclamation: Current head 7f50e89 differs from pull request most recent head 6947fa2. Consider uploading reports for the commit 6947fa2 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.78% -0.03%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50483 +91
- Misses 9736 9772 +36
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-917110923
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713397475
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Thanks.
Just to confirm, if the Java constructor/method does not have an argument named `foo` then `builderMethod(foo=x)` will fail, right? And if it does have an argument named `foo` and `bar` then `builderMethod(foo=x, bar=y)` will produce the same result as `builderMethod(bar=y, foo=x)` (or one of them rejected as the name order doesn't match)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0fc66aa) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `88.23%`.
> :exclamation: Current head 0fc66aa differs from pull request most recent head 663c221. Consider uploading reports for the commit 663c221 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 443
Lines 60128 60181 +53
==========================================
+ Hits 50392 50416 +24
- Misses 9736 9765 +29
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.70% <0.00%> (-1.68%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.14% <0.00%> (-0.63%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=) | `96.25% <0.00%> (-0.54%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...663c221](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6947fa2) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50479 +87
- Misses 9736 9776 +40
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6947fa2) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50479 +87
- Misses 9736 9776 +40
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6947fa2) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50479 +87
- Misses 9736 9776 +40
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-923412320
Run Java PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713500335
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
OK, if the parameter names are not available, let's fail if trying to use named parameters rather than silently hope they match. Types are not enough to disambiguate, e.g. imagine `CopyTransform(String src, String dest)` vs. `CopyTransform(String dest, String src)`. Maybe this'd fail, but something like `ComputeDiffTransform(String newLoc, String prevLoc)` would give the wrong results if someone called `ComputeDiffTransform(prevLoc="old", newLoc="new")`. True one can make this mistake with positional arguments, but at least in that case one is aware that the ordering should be checked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj closed pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj closed pull request #15484:
URL: https://github.com/apache/beam/pull/15484
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c79eeed) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **increase** coverage by `0.03%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
+ Coverage 83.74% 83.77% +0.03%
==========================================
Files 443 443
Lines 60077 60170 +93
==========================================
+ Hits 50312 50408 +96
+ Misses 9765 9762 -3
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...he\_beam/runners/interactive/sql/beam\_sql\_magics.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvYmVhbV9zcWxfbWFnaWNzLnB5) | `54.70% <0.00%> (-0.96%)` | :arrow_down: |
| [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/io/gcp/bigquery.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZ2NwL2JpZ3F1ZXJ5LnB5) | `75.53% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.81% <0.00%> (ø)` | |
| [.../python/apache\_beam/portability/api/metrics\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyLnB5) | `100.00% <0.00%> (ø)` | |
| [...hon/apache\_beam/portability/api/beam\_fn\_api\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZm5fYXBpX3BiMi5weQ==) | `100.00% <0.00%> (ø)` | |
| [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...n/apache\_beam/runners/interactive/cache\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoZV9tYW5hZ2VyLnB5) | `90.84% <0.00%> (ø)` | |
| ... and [7 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...c79eeed](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r708785353
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
Review comment:
Done. We already fail for more than one non-trivial invocation of with_constructor or with_constructor_method.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
I think there are multiple issues with supporting that.
(1) Java args have to be ordered but Python requires args to be specified before kwargs. So we'll somehow have to preserve the order between args and kwargs or accept only one. That will probably end up being confusing to users.
(2) Schema requires field names to be specified. So we'll have to define a special set of field names that get ignored by Java side.
Also, Java parameter names should be already known (for example, available in Java docs) and it should not be too difficult for Python users to specify these.
WDYT ?
##########
File path: sdks/python/apache_beam/portability/common_urns.py
##########
@@ -66,3 +67,5 @@
requirements = StandardRequirements.Enum
displayData = StandardDisplayData.DisplayData
+
+java_class_lookup = ExpansionMethods.JAVA_CLASS_LOOKUP
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r714342679
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
cc: @lostluck
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-916270579
R: @ihji
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f50e89) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.02%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.78% -0.03%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50483 +91
- Misses 9736 9772 +36
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...7f50e89](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r709677951
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
> Ordering
Right. The issue goes away if we always use *args before *kwargs (in the order they are specified).
> One issue is that the java argument names are not part of the signature's spec
This should be fine. We ignore argument names at Java side when the are not available through reflection.
> As for defining names, a simple scheme that uses "illegal" parameter names (e.g. #N) should be sufficient.
Sounds good I can add that and update the spec. I assume you meant that certain parameter names that will be ignored by the expansion service and will not be used for validation.
> Do we correctly handle builderMethod(a=1, b=2) vs. builderMethod(b=2, a=1) or otherwise enforce the names today?
We cannot reorder args since that could lead to ambiguities in Java side. Hence we send the args exactly in the order they are specified. May be we can clarify this in the documentation ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713413214
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
The argument get added as a field name of the schema for the method/constructor. I don't think the field name can be empty for a schema.
Note that we now include a single schema for a method/constructor which simplified things quite a bit and allowed us to support more types in a generic way: https://github.com/apache/beam/blob/919fdef0188f808d1d460bf26c1ca7db1db8648b/model/pipeline/src/main/proto/external_transforms.proto#L102
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3808ea1) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **increase** coverage by `0.00%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
=======================================
Coverage 83.74% 83.75%
=======================================
Files 443 443
Lines 60077 60136 +59
=======================================
+ Hits 50312 50367 +55
- Misses 9765 9769 +4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...he\_beam/runners/interactive/sql/beam\_sql\_magics.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvYmVhbV9zcWxfbWFnaWNzLnB5) | `54.70% <0.00%> (-0.96%)` | :arrow_down: |
| [.../python/apache\_beam/portability/api/metrics\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyLnB5) | `100.00% <0.00%> (ø)` | |
| [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...n/apache\_beam/runners/interactive/cache\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoZV9tYW5hZ2VyLnB5) | `90.84% <0.00%> (ø)` | |
| [...eam/runners/interactive/caching/streaming\_cache.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL3N0cmVhbWluZ19jYWNoZS5weQ==) | `95.35% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.51% <0.00%> (+0.24%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...3808ea1](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r709669197
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Kwargs always come after args, so the ordering between them is well-defined.
One issue is that the java argument names are not part of the signature's spec, so may not be stable, or may not even be in the .class files at all. As for defining names, a simple scheme that uses "illegal" parameter names (e.g. `#N`) should be sufficient.
Do we correctly handle `builderMethod(a=1, b=2)` vs. `builderMethod(b=2, a=1)` or otherwise enforce the names today? If not, perhaps we should only allow positional arguments, as one would expect those two to be exactly the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713397475
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Thanks.
Just to confirm, if the Java constructor/method does not have an argument named `foo` then `builderMethod(foo=x)` will fail, right? And if it does have an argument named `foo` and `bar` then `builderMethod(foo=x, bar=y)` will produce the same result as `builderMethod(bar=y, foo=x)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713466822
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
It's not available by default. Preserving parameter names requires specifying the Java compiler parameter "-parameters". Beam sets this by default.
I don't think this is a major issue though.
We use parameter names to avoid any ambiguities when mapping a Python method/constructor invocation to Java side. If available information is inadequate to unambiguously determine the constructor/method we fail.
https://github.com/apache/beam/blob/587f58e1a1c3f5caa14f9b47144b5a5296cff9b5/sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java#L238
To perform the actual invocation, we just use the values (similar to a local Java invocation) hence if provided value types do not match, that will result in a failure.
https://github.com/apache/beam/blob/587f58e1a1c3f5caa14f9b47144b5a5296cff9b5/sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java#L168
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (663c221) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `89.39%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50480 +88
- Misses 9736 9775 +39
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.38% <89.06%> (+2.98%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.24% <0.00%> (-4.88%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...663c221](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (663c221) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `89.39%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50480 +88
- Misses 9736 9775 +39
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.38% <89.06%> (+2.98%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.24% <0.00%> (-4.88%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...663c221](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c79eeed) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **increase** coverage by `0.03%`.
> The diff coverage is `88.23%`.
> :exclamation: Current head c79eeed differs from pull request most recent head 0fc66aa. Consider uploading reports for the commit 0fc66aa to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
+ Coverage 83.80% 83.84% +0.03%
==========================================
Files 443 443
Lines 60128 60386 +258
==========================================
+ Hits 50392 50630 +238
- Misses 9736 9756 +20
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.70% <0.00%> (-1.68%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=) | `96.25% <0.00%> (-0.54%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.51% <0.00%> (-0.25%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.81% <0.00%> (-0.17%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/combiners.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb21iaW5lcnMucHk=) | `95.79% <0.00%> (+1.91%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...0fc66aa](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (663c221) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `89.39%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50480 +88
- Misses 9736 9775 +39
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.38% <89.06%> (+2.98%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.24% <0.00%> (-4.88%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...663c221](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3808ea1) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **increase** coverage by `0.00%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
=======================================
Coverage 83.74% 83.75%
=======================================
Files 443 443
Lines 60077 60136 +59
=======================================
+ Hits 50312 50367 +55
- Misses 9765 9769 +4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...he\_beam/runners/interactive/sql/beam\_sql\_magics.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvYmVhbV9zcWxfbWFnaWNzLnB5) | `54.70% <0.00%> (-0.96%)` | :arrow_down: |
| [.../python/apache\_beam/portability/api/metrics\_pb2.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyLnB5) | `100.00% <0.00%> (ø)` | |
| [...on/apache\_beam/portability/api/metrics\_pb2\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL21ldHJpY3NfcGIyX3VybnMucHk=) | `100.00% <0.00%> (ø)` | |
| [...n/apache\_beam/runners/interactive/cache\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoZV9tYW5hZ2VyLnB5) | `90.84% <0.00%> (ø)` | |
| [...eam/runners/interactive/caching/streaming\_cache.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9jYWNoaW5nL3N0cmVhbWluZ19jYWNoZS5weQ==) | `95.35% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.51% <0.00%> (+0.24%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...3808ea1](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713410868
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Another idea, rather than having an ignored prefix, is to allow any number of unnamed (empty string) arguments at the beginning which would be considered "positional" arguments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r707825421
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Here (and elsewhere) should we accept args as well as kwargs, as java arguments are unnamed?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r705811530
##########
File path: sdks/python/apache_beam/portability/common_urns.py
##########
@@ -66,3 +67,5 @@
requirements = StandardRequirements.Enum
displayData = StandardDisplayData.DisplayData
+
+java_class_lookup = ExpansionMethods.JAVA_CLASS_LOOKUP
Review comment:
newline
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
Review comment:
perhaps this should be 'with' rather than 'add' as there can only be one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6947fa2) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50479 +87
- Misses 9736 9776 +40
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713416946
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
"we can only perform validation if Java reflection preserved arg names" is what concerns me. Is this the default mode? If so, maybe that's good enough.
Good point about schemas requiring names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f58547a) into [master](https://codecov.io/gh/apache/beam/commit/823ab5a1bee062b917cb8811633579614622e1e1?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (823ab5a) will **decrease** coverage by `0.00%`.
> The diff coverage is `88.23%`.
> :exclamation: Current head f58547a differs from pull request most recent head 3808ea1. Consider uploading reports for the commit 3808ea1 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.74% 83.74% -0.01%
==========================================
Files 443 443
Lines 60077 60128 +51
==========================================
+ Hits 50312 50354 +42
- Misses 9765 9774 +9
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.37% <0.00%> (-4.66%)` | :arrow_down: |
| [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.65% <0.00%> (-1.87%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (ø)` | |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.95% <0.00%> (+0.15%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [823ab5a...3808ea1](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713522450
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
May be we should limit https://github.com/apache/beam/pull/15546 to positional arguments to prevent confusion ?
I think supporting keyword arguments in "JavaClassLookupPayloadBuilder" for default Java compilation will be useful since any higher level wrappers can easily reorder arguments before calling "JavaClassLookupPayloadBuilder" (to respect its spec).
Also, even when parameter names are available, "JavaClassLookupPayloadBuilder" will still require keyword arguments to be specified in the correct order since Schema (and the Java invocation) requires it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-917223476
Run Python PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (7f50e89) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.02%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.78% -0.03%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50483 +91
- Misses 9736 9772 +36
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...7f50e89](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ihji commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
ihji commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r711410111
##########
File path: sdks/python/apache_beam/transforms/external_test.py
##########
@@ -406,6 +410,88 @@ class DataclassTransform(beam.ExternalTransform):
return get_payload(DataclassTransform(**values))
+class JavaClassLookupPayloadBuilderTest(unittest.TestCase):
Review comment:
Do we also need a test for setting `with_constructor_method` twice? or setting `with_constructor_method` after `with_constructor`?
##########
File path: sdks/python/apache_beam/transforms/external_test.py
##########
@@ -406,6 +410,88 @@ class DataclassTransform(beam.ExternalTransform):
return get_payload(DataclassTransform(**values))
+class JavaClassLookupPayloadBuilderTest(unittest.TestCase):
+ def _verify_row(self, schema, row_payload, expected_values):
+ row = RowCoder(schema).decode(row_payload)
+
+ for attr_name, expected_value in expected_values.items():
+ self.assertTrue(hasattr(row, attr_name))
+ value = getattr(row, attr_name)
+ self.assertEqual(expected_value, value)
+
+ def test_build_payload_with_constructor(self):
+ payload_builder = JavaClassLookupPayloadBuilder('dummy_class_name')
+
+ payload_builder.with_constructor('abc', 123, str_field='def', int_field=456)
+ payload_bytes = payload_builder.payload()
+ payload_from_bytes = proto_utils.parse_Bytes(
+ payload_bytes, JavaClassLookupPayload)
+ self.assertTrue(isinstance(payload_from_bytes, JavaClassLookupPayload))
Review comment:
Also check `constructor_method` field is empty?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0fc66aa) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 443
Lines 60128 60181 +53
==========================================
+ Hits 50392 50416 +24
- Misses 9736 9765 +29
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.70% <0.00%> (-1.68%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.14% <0.00%> (-0.63%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=) | `96.25% <0.00%> (-0.54%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...0fc66aa](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r712173592
##########
File path: sdks/python/apache_beam/transforms/external_test.py
##########
@@ -406,6 +410,88 @@ class DataclassTransform(beam.ExternalTransform):
return get_payload(DataclassTransform(**values))
+class JavaClassLookupPayloadBuilderTest(unittest.TestCase):
Review comment:
Done.
##########
File path: sdks/python/apache_beam/transforms/external_test.py
##########
@@ -406,6 +410,88 @@ class DataclassTransform(beam.ExternalTransform):
return get_payload(DataclassTransform(**values))
+class JavaClassLookupPayloadBuilderTest(unittest.TestCase):
+ def _verify_row(self, schema, row_payload, expected_values):
+ row = RowCoder(schema).decode(row_payload)
+
+ for attr_name, expected_value in expected_values.items():
+ self.assertTrue(hasattr(row, attr_name))
+ value = getattr(row, attr_name)
+ self.assertEqual(expected_value, value)
+
+ def test_build_payload_with_constructor(self):
+ payload_builder = JavaClassLookupPayloadBuilder('dummy_class_name')
+
+ payload_builder.with_constructor('abc', 123, str_field='def', int_field=456)
+ payload_bytes = payload_builder.payload()
+ payload_from_bytes = proto_utils.parse_Bytes(
+ payload_bytes, JavaClassLookupPayload)
+ self.assertTrue(isinstance(payload_from_bytes, JavaClassLookupPayload))
Review comment:
Done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r714372641
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Sent https://github.com/apache/beam/pull/15557
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r713409178
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
> if the Java constructor/method does not have an argument named foo then builderMethod(foo=x) will fail, right?
Yes (unless arg name starts with the predefined 'ignore' prefix). Also we can only perform validation if Java reflection preserved arg names otherwise we just validate using types.
> And if it does have an argument named foo and bar then builderMethod(foo=x, bar=y) will produce the same result as builderMethod(bar=y, foo=x) (or one of them rejected as the name order doesn't match)?
If Java builder method is 'myMethod(int foo, int bar)'
Then 'builderMethod(foo=x, bar=y)' will pass while 'builderMethod(bar=x, foo=y)' will fail. In other words, Python arguments have to be specified in exactly the same order as Java. I documented it in the 'JavaClassLookupPayloadBuilder'
https://github.com/apache/beam/blob/c471fd31e67a8c29315e7c1859482177d1fe7afa/sdks/python/apache_beam/transforms/external.py#L270
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r709679908
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
Note that Python 3.6 and above preserve the order of **kwargs: https://www.python.org/dev/peps/pep-0468/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (663c221) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `89.39%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50480 +88
- Misses 9736 9775 +39
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.38% <89.06%> (+2.98%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `90.24% <0.00%> (-4.88%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [22 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...663c221](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6947fa2) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `90.90%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 444 +1
Lines 60128 60255 +127
==========================================
+ Hits 50392 50479 +87
- Misses 9736 9776 +40
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `77.70% <90.62%> (+3.30%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...ache\_beam/runners/interactive/recording\_manager.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9yZWNvcmRpbmdfbWFuYWdlci5weQ==) | `96.55% <0.00%> (-2.43%)` | :arrow_down: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.50% <0.00%> (-1.89%)` | :arrow_down: |
| [...ython/apache\_beam/runners/interactive/sql/utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9zcWwvdXRpbHMucHk=) | `89.58% <0.00%> (-1.49%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| ... and [23 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...6947fa2](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15484:
URL: https://github.com/apache/beam/pull/15484#discussion_r714085688
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -144,6 +147,101 @@ def _get_named_tuple_instance(self):
return self._tuple_instance
+class JavaClassLookupPayloadBuilder(PayloadBuilder):
+ """
+ Builds a payload for directly instantiating a Java transform using a
+ constructor and builder methods.
+ """
+ def __init__(self, class_name):
+ """
+ :param class_name: fully qualified name of the transform class.
+ """
+ if not class_name:
+ raise ValueError('Class name must not be empty')
+
+ self._class_name = class_name
+ self._constructor_method = None
+ self._constructor_params = None
+ self._builder_methods_and_params = OrderedDict()
+
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ for key, value in kwargs.items():
+ if not key:
+ raise ValueError('Parameter name cannot be empty')
+ if value is None:
+ raise ValueError(
+ 'Received value None for key %s. None values are currently not '
+ 'supported' % key)
+ named_fields.append(
+ (key, convert_to_typing_type(instance_to_type(value))))
+
+ schema_proto = named_fields_to_schema(named_fields)
+ row = named_tuple_from_schema(schema_proto)(**kwargs)
+ schema = named_tuple_to_schema(type(row))
+
+ payload = RowCoder(schema).encode(row)
+ return (schema_proto, payload)
+
+ def build(self):
+ constructor_params = self._constructor_params or {}
+ constructor_schema, constructor_payload = (
+ self._get_schema_proto_and_payload(**constructor_params))
+ payload = JavaClassLookupPayload(
+ class_name=self._class_name,
+ constructor_schema=constructor_schema,
+ constructor_payload=constructor_payload)
+ if self._constructor_method:
+ payload.constructor_method = self._constructor_method
+
+ for builder_method_name, params in self._builder_methods_and_params.items():
+ builder_method_schema, builder_method_payload = (
+ self._get_schema_proto_and_payload(**params))
+ builder_method = BuilderMethod(
+ name=builder_method_name,
+ schema=builder_method_schema,
+ payload=builder_method_payload)
+ builder_method.name = builder_method_name
+ payload.builder_methods.append(builder_method)
+ return payload
+
+ def add_constructor(self, **kwargs):
+ """
+ Specifies the Java constructor to use.
+
+ :param kwargs: parameter names and values of the constructor.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_params = kwargs
+
+ def add_constructor_method(self, method_name, **kwargs):
+ """
+ Specifies the Java constructor method to use.
+
+ :param method_name: name of the constructor method.
+ :param kwargs: parameter names and values of the constructor method.
+ """
+ if self._constructor_method or self._constructor_params:
+ raise ValueError(
+ 'Constructor or constructor method can only be specified once')
+
+ self._constructor_method = method_name
+ self._constructor_params = kwargs
+
+ def add_builder_method(self, method_name, **kwargs):
Review comment:
We should accept keyword arguments, for both, only when the parameter names are available. Maybe it makes sense to take this to the list?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15484:
URL: https://github.com/apache/beam/pull/15484#issuecomment-915594678
# [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15484](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (0fc66aa) into [master](https://codecov.io/gh/apache/beam/commit/25c67cdbae3ed2b9158d1f18ff4d522422dd3886?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (25c67cd) will **decrease** coverage by `0.03%`.
> The diff coverage is `88.23%`.
[](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15484 +/- ##
==========================================
- Coverage 83.80% 83.77% -0.04%
==========================================
Files 443 443
Lines 60128 60181 +53
==========================================
+ Hits 50392 50416 +24
- Misses 9736 9765 +29
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `76.58% <87.75%> (+2.18%)` | :arrow_up: |
| [sdks/python/apache\_beam/portability/common\_urns.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvY29tbW9uX3VybnMucHk=) | `100.00% <100.00%> (ø)` | |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `87.70% <0.00%> (-1.68%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/source\_test\_utils.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vc291cmNlX3Rlc3RfdXRpbHMucHk=) | `88.47% <0.00%> (-1.39%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `91.47% <0.00%> (-0.78%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.14% <0.00%> (-0.63%)` | :arrow_down: |
| [sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=) | `96.25% <0.00%> (-0.54%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/apache/beam/pull/15484/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [25c67cd...0fc66aa](https://codecov.io/gh/apache/beam/pull/15484?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj merged pull request #15484: [BEAM-12769] Python support for directly using Java transforms using constructor and builder methods
Posted by GitBox <gi...@apache.org>.
chamikaramj merged pull request #15484:
URL: https://github.com/apache/beam/pull/15484
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org