You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2021/11/01 17:23:49 UTC
[GitHub] [beam] robertwb opened a new pull request #15857: Allow wildcards for java class lookup transform providers.
robertwb opened a new pull request #15857:
URL: https://github.com/apache/beam/pull/15857
This is especially useful for cases where the expansion service is implicitly
started by the user for ad-hoc invocation of cross-language transforms (in which
case there is no security benefit to an explicit allow list).
Also automatically starts up an expansion service for Python's
JavaExternalTransform and adds the ability to manually specify a classpath.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`).
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier).
`ValidatesRunner` compliance status (on master branch)
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/badge/icon?subject=Portable+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Samza/lastCompletedBuild/badge/icon?subject=Portable">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/badge/icon">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/badge/icon?subject=Structured+Streaming">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon?subject=ValCont">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_PVR_Flink_Cron/lastCompletedBuild/badge/icon?subject=Portable">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Dataflow/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Samza/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
</tr>
</tbody>
</table>
Examples testing status on various runners
--------------------------------------------------------
<table>
<thead>
<tr>
<th>Lang</th>
<th>ULR</th>
<th>Dataflow</th>
<th>Flink</th>
<th>Samza</th>
<th>Spark</th>
<th>Twister2</th>
</tr>
</thead>
<tbody>
<tr>
<td>Go</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Java</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/lastCompletedBuild/badge/icon?subject=V1">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Java11_Cron/lastCompletedBuild/badge/icon?subject=V1+Java11">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2">
</a><br>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>Python</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
<tr>
<td>XLang</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
Post-Commit SDK/Transform Integration Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>Go</th>
<th>Java</th>
<th>Python</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/badge/icon?subject=3.6">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/badge/icon?subject=3.7">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/badge/icon?subject=3.8">
</a>
</td>
</tr>
</tbody>
</table>
Pre-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
<table>
<thead>
<tr>
<th>---</th>
<th>Java</th>
<th>Python</th>
<th>Go</th>
<th>Website</th>
<th>Whitespace</th>
<th>Typescript</th>
</tr>
</thead>
<tbody>
<tr>
<td>Non-portable</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Java_Cron/lastCompletedBuild/badge/icon">
</a><br>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/lastCompletedBuild/badge/icon?subject=Tests">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonLint_Cron/lastCompletedBuild/badge/icon?subject=Lint">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocker_Cron/badge/icon?subject=Docker">
</a><br>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_PythonDocs_Cron/badge/icon?subject=Docs">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Go_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Website_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Whitespace_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Typescript_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
</tr>
<tr>
<td>Portable</td>
<td>---</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>
<a href="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/">
<img alt="Build Status" src="https://ci-beam.apache.org/job/beam_PreCommit_GoPortable_Cron/lastCompletedBuild/badge/icon">
</a>
</td>
<td>---</td>
<td>---</td>
<td>---</td>
</tr>
</tbody>
</table>
See [.test-infra/jenkins/README](https://github.com/apache/beam/blob/master/.test-infra/jenkins/README.md) for trigger phrase, status and link of all Jenkins jobs.
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-971859548
Python is now happy (we now make sure we unbock in processes_test.py) and the java issues look unrelated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r745148209
##########
File path: sdks/python/apache_beam/transforms/external_test.py
##########
@@ -547,6 +550,27 @@ def test_implicit_builder_with_constructor_method(self):
self._verify_row(build_method.schema, build_method.payload, {})
+class JavaJarExpansionServiceTest(unittest.TestCase):
+ def test_classpath(self):
Review comment:
Yeah, I agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-970732012
Looks like someone is monkey-patching mock into there and it's not getting un-mocked... It passed locally. I'll try to fix this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r742457520
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
Review comment:
Done.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
Review comment:
Yes, that's the example above.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Looks like the file stager does handle directories. Updated the code to handle globs.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Ideally the file stager could handle those as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r749781170
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,7 +161,11 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
+ if classpath or True:
Review comment:
Thanks for catching this. (I was just trying to run all existing tests through this code.)
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -283,6 +290,39 @@ def beam_services(cls, replacements):
finally:
cls._BEAM_SERVICES.replacements = old
+ @classmethod
+ def make_classpath_jar(cls, main_jar, extra_jars, cache_dir=None):
Review comment:
Done.
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -283,6 +290,39 @@ def beam_services(cls, replacements):
finally:
cls._BEAM_SERVICES.replacements = old
+ @classmethod
+ def make_classpath_jar(cls, main_jar, extra_jars, cache_dir=None):
+ if cache_dir is None:
+ cache_dir = cls.JAR_CACHE
+ composite_jar_dir = os.path.join(cache_dir, 'composite-jars')
+ os.makedirs(composite_jar_dir, exist_ok=True)
+ classpath = []
+ # Class-Path references from a jar must be relative.
+ for pattern in [main_jar] + list(extra_jars):
+ for path in glob.glob(pattern) or [pattern]:
+ rel_path = hashlib.sha256(
+ path.encode('utf-8')).hexdigest() + os.path.splitext(path)[1]
+ classpath.append(rel_path)
+ if not os.path.lexists(os.path.join(composite_jar_dir, rel_path)):
+ os.symlink(path, os.path.join(composite_jar_dir, rel_path))
+ composite_jar = os.path.join(
+ composite_jar_dir,
+ hashlib.sha256(' '.join(sorted(classpath)).encode('ascii')).hexdigest()
+ + '.jar')
+ if not os.path.exists(composite_jar):
Review comment:
Fair point. The code here is best-effort, but for anything complicated you'll have to roll your own expansion service. (It already went deeper into jars than I wanted, but otherwise isn't very useful...)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-956429335
R: @lukecwik , @chamikaramj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...f3b9159](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r746901926
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -283,6 +290,39 @@ def beam_services(cls, replacements):
finally:
cls._BEAM_SERVICES.replacements = old
+ @classmethod
+ def make_classpath_jar(cls, main_jar, extra_jars, cache_dir=None):
Review comment:
Please add a unit test for this as well.
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -283,6 +290,39 @@ def beam_services(cls, replacements):
finally:
cls._BEAM_SERVICES.replacements = old
+ @classmethod
+ def make_classpath_jar(cls, main_jar, extra_jars, cache_dir=None):
+ if cache_dir is None:
+ cache_dir = cls.JAR_CACHE
+ composite_jar_dir = os.path.join(cache_dir, 'composite-jars')
+ os.makedirs(composite_jar_dir, exist_ok=True)
+ classpath = []
+ # Class-Path references from a jar must be relative.
+ for pattern in [main_jar] + list(extra_jars):
+ for path in glob.glob(pattern) or [pattern]:
+ rel_path = hashlib.sha256(
+ path.encode('utf-8')).hexdigest() + os.path.splitext(path)[1]
+ classpath.append(rel_path)
+ if not os.path.lexists(os.path.join(composite_jar_dir, rel_path)):
+ os.symlink(path, os.path.join(composite_jar_dir, rel_path))
+ composite_jar = os.path.join(
+ composite_jar_dir,
+ hashlib.sha256(' '.join(sorted(classpath)).encode('ascii')).hexdigest()
+ + '.jar')
+ if not os.path.exists(composite_jar):
Review comment:
It's unclear to me if this will work for all Jars. @lukecwik might be able to confirm. (non-blocking comment)
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -283,6 +290,39 @@ def beam_services(cls, replacements):
finally:
cls._BEAM_SERVICES.replacements = old
+ @classmethod
+ def make_classpath_jar(cls, main_jar, extra_jars, cache_dir=None):
+ if cache_dir is None:
+ cache_dir = cls.JAR_CACHE
+ composite_jar_dir = os.path.join(cache_dir, 'composite-jars')
+ os.makedirs(composite_jar_dir, exist_ok=True)
+ classpath = []
+ # Class-Path references from a jar must be relative.
Review comment:
Could you expand the comment here to explain what's done here.
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,7 +161,11 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
+ if classpath or True:
Review comment:
This is always True ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...f3b9159](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-960347757
Also rebased on top of https://github.com/apache/beam/pull/15891
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **increase** coverage by `0.00%`.
> The diff coverage is `65.00%`.
> :exclamation: Current head f3b9159 differs from pull request most recent head 8375984. Consider uploading reports for the commit 8375984 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
=======================================
Coverage 83.52% 83.52%
=======================================
Files 445 445
Lines 61385 61385
=======================================
+ Hits 51270 51274 +4
+ Misses 10115 10111 -4
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <75.00%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `94.87% <0.00%> (-0.04%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.73% <0.00%> (+0.32%)` | :arrow_up: |
| [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `89.20% <0.00%> (+1.70%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...8375984](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15752d9) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **increase** coverage by `0.08%`.
> The diff coverage is `83.67%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.53% 83.61% +0.08%
==========================================
Files 445 445
Lines 61385 61414 +29
==========================================
+ Hits 51276 51354 +78
+ Misses 10109 10060 -49
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `58.73% <90.62%> (+5.59%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...eam/portability/api/beam\_expansion\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `57.89% <0.00%> (-4.02%)` | :arrow_down: |
| [...eam/portability/api/beam\_provision\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcHJvdmlzaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `73.68% <0.00%> (-2.51%)` | :arrow_down: |
| [...e\_beam/portability/api/beam\_runner\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjJfZ3JwYy5weQ==) | `78.94% <0.00%> (-2.01%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | `96.78% <0.00%> (-0.57%)` | :arrow_down: |
| ... and [32 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...15752d9](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15752d9) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **increase** coverage by `0.08%`.
> The diff coverage is `83.67%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.53% 83.61% +0.08%
==========================================
Files 445 445
Lines 61385 61414 +29
==========================================
+ Hits 51276 51354 +78
+ Misses 10109 10060 -49
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `58.73% <90.62%> (+5.59%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...eam/portability/api/beam\_expansion\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `57.89% <0.00%> (-4.02%)` | :arrow_down: |
| [...eam/portability/api/beam\_provision\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcHJvdmlzaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `73.68% <0.00%> (-2.51%)` | :arrow_down: |
| [...e\_beam/portability/api/beam\_runner\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjJfZ3JwYy5weQ==) | `78.94% <0.00%> (-2.01%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | `96.78% <0.00%> (-0.57%)` | :arrow_down: |
| ... and [32 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...15752d9](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...f3b9159](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
> :exclamation: Current head f3b9159 differs from pull request most recent head 5655c57. Consider uploading reports for the commit 5655c57 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...5655c57](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r745211708
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
+ if classpath:
+ cp_args = ['-classpath', os.pathsep.join(classpath)]
+ else:
+ cp_args = []
super().__init__(
- stub_class, ['java', '-jar', path_to_jar] + list(java_arguments))
+ stub_class,
+ ['java'] + cp_args + ['-jar', path_to_jar] + list(java_arguments))
Review comment:
OK, I added support for this. I was tempted to just drop the classpath thing, but then the default JavaExternalTransform would be much less useful, and I think this'll be needed for the JDBC transforms as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-960347757
Also rebased on top of https://github.com/apache/beam/pull/15891
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-967746526
Hi Robert, any updates on this ?
It's great if we can get this into Beam 2.35.0 (cut 11/17)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r741540490
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
Probably we should document.
* How to create an allowlist file to allow everything.
* How to create an allowlist file to just allow a given package.
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
Review comment:
Could we add a unit test to cover this ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
+ expansion_service = expansion_service or BeamJarExpansionService(
Review comment:
So the default version will just offer the classes packaged with the <expansion-service> package ? I'm not sure whether this will be useful (it's whatever in that package and Beam core). Should we make providing classpath (or additional jars) required to make the default usage useful ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
Review comment:
JavaExternalTransform('fully.qualified.ClassName')(\<params\>) works as well, right ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
Review comment:
Please document what classpath should be (seems like a list based on the usage below).
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionServiceOptions.java
##########
@@ -53,6 +52,9 @@ public AllowList create(PipelineOptions options) {
String allowListFile =
options.as(ExpansionServiceOptions.class).getJavaClassLookupAllowlistFile();
if (allowListFile != null) {
+ if (allowListFile.equals("*")) {
Review comment:
Please document the updated usage in the corresponding property of ExpansionServiceOptions.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Seems like we assume that the provided classpath only contains jars ? Classpath also supports directories and wild cards.
https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8375984) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **decrease** coverage by `0.02%`.
> The diff coverage is `37.50%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
- Coverage 83.52% 83.49% -0.03%
==========================================
Files 445 445
Lines 61385 61427 +42
==========================================
+ Hits 51270 51288 +18
- Misses 10115 10139 +24
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `47.80% <19.35%> (-5.34%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (-0.25%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.25% <0.00%> (-0.17%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.72% <0.00%> (+0.34%)` | :arrow_up: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `86.36% <0.00%> (+0.50%)` | :arrow_up: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `76.58% <0.00%> (+0.97%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...8375984](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r743979210
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,12 +284,27 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName'
+ )(contructorArg, ... ).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
+
+ :param class_name: fully qualified name of the java class
+ :param expansion_service: (Optional) an expansion service to use. If none is
+ provided, a default expansion service will be started.
+ :param classpath: (Optional) A list paths to additional jars to place on the
+ expansion service classpath.
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
+ assert not (expansion_service and classpath), (expansion_service, classpath)
Review comment:
Did you expect to include an error message instead of "(expansion_service, classpath)" ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -302,7 +318,11 @@ def __getattr__(self, name):
# Don't try to emulate special methods.
if name.startswith('__') and name.endswith('__'):
return super().__getattr__(name)
+ else:
+ return self[name]
+ def __getitem__(self, name):
Review comment:
What does moving this to a dictionary fix ? (for my knowledge)
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,22 +680,29 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
- if extra_args is None:
Review comment:
Probably need to rebase again (based on the diff).
##########
File path: sdks/python/apache_beam/transforms/external_test.py
##########
@@ -547,6 +550,27 @@ def test_implicit_builder_with_constructor_method(self):
self._verify_row(build_method.schema, build_method.payload, {})
+class JavaJarExpansionServiceTest(unittest.TestCase):
+ def test_classpath(self):
Review comment:
BTW I think we should add more comprehensive unit testing for "JavaJarExpansionService" and "BeamJarExpansionService" (can be a separate PR).
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,22 +680,29 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
- if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
self._path_to_jar = path_to_jar
self._extra_args = extra_args
+ self._classpath = classpath
self._service_count = 0
+ def _default_args(self):
+ to_stage = ','.join([self._path_to_jar] + sum(
Review comment:
Fail here if self._path_to_jar is not set ?
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
+ if classpath:
+ cp_args = ['-classpath', os.pathsep.join(classpath)]
+ else:
+ cp_args = []
super().__init__(
- stub_class, ['java', '-jar', path_to_jar] + list(java_arguments))
+ stub_class,
+ ['java'] + cp_args + ['-jar', path_to_jar] + list(java_arguments))
Review comment:
Unfortunately I don't think "java -jar" command can take a CLASSPATH as an argument. So this probably won't work.
https://docs.oracle.com/en/java/javase/17/docs/specs/man/jar.html
Im fine with partially rolling this back to use a single jar when we automatically startup the expansion service.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r745147146
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -302,7 +318,11 @@ def __getattr__(self, name):
# Don't try to emulate special methods.
if name.startswith('__') and name.endswith('__'):
return super().__getattr__(name)
+ else:
+ return self[name]
+ def __getitem__(self, name):
Review comment:
It just makes it so we can use names that would otherwise overlap with existing attributes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb merged pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb merged pull request #15857:
URL: https://github.com/apache/beam/pull/15857
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r741540490
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
Probably we should document.
* How to create an allowlist file to allow everything.
* How to create an allowlist file to just allow a given package.
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
Review comment:
Could we add a unit test to cover this ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
+ expansion_service = expansion_service or BeamJarExpansionService(
Review comment:
So the default version will just offer the classes packaged with the <expansion-service> package ? I'm not sure whether this will be useful (it's whatever in that package and Beam core). Should we make providing classpath (or additional jars) required to make the default usage useful ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
Review comment:
JavaExternalTransform('fully.qualified.ClassName')(\<params\>) works as well, right ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
Review comment:
Please document what classpath should be (seems like a list based on the usage below).
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionServiceOptions.java
##########
@@ -53,6 +52,9 @@ public AllowList create(PipelineOptions options) {
String allowListFile =
options.as(ExpansionServiceOptions.class).getJavaClassLookupAllowlistFile();
if (allowListFile != null) {
+ if (allowListFile.equals("*")) {
Review comment:
Please document the updated usage in the corresponding property of ExpansionServiceOptions.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Seems like we assume that the provided classpath only contains jars ? Classpath also supports directories and wild cards.
https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r744004687
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
+ if classpath:
+ cp_args = ['-classpath', os.pathsep.join(classpath)]
+ else:
+ cp_args = []
super().__init__(
- stub_class, ['java', '-jar', path_to_jar] + list(java_arguments))
+ stub_class,
+ ['java'] + cp_args + ['-jar', path_to_jar] + list(java_arguments))
Review comment:
This is the correct link: https://docs.oracle.com/javase/7/docs/technotes/tools/windows/java.html
"When you use this option, the JAR file is the source of all user classes, and other user class path settings are ignored."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8375984) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **decrease** coverage by `0.02%`.
> The diff coverage is `37.50%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
- Coverage 83.52% 83.49% -0.03%
==========================================
Files 445 445
Lines 61385 61427 +42
==========================================
+ Hits 51270 51288 +18
- Misses 10115 10139 +24
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `47.80% <19.35%> (-5.34%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (-0.25%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.25% <0.00%> (-0.17%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.72% <0.00%> (+0.34%)` | :arrow_up: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `86.36% <0.00%> (+0.50%)` | :arrow_up: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `76.58% <0.00%> (+0.97%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...8375984](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8375984) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **decrease** coverage by `0.03%`.
> The diff coverage is `37.50%`.
> :exclamation: Current head 8375984 differs from pull request most recent head 15752d9. Consider uploading reports for the commit 15752d9 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
- Coverage 83.53% 83.49% -0.04%
==========================================
Files 445 445
Lines 61385 61427 +42
==========================================
+ Hits 51276 51288 +12
- Misses 10109 10139 +30
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `47.80% <19.35%> (-5.34%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (-0.50%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.25% <0.00%> (-0.17%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...15752d9](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15752d9) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **increase** coverage by `0.08%`.
> The diff coverage is `83.67%`.
> :exclamation: Current head 15752d9 differs from pull request most recent head 8f81a2c. Consider uploading reports for the commit 8f81a2c to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.53% 83.61% +0.08%
==========================================
Files 445 445
Lines 61385 61414 +29
==========================================
+ Hits 51276 51354 +78
+ Misses 10109 10060 -49
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `58.73% <90.62%> (+5.59%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...eam/portability/api/beam\_expansion\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `57.89% <0.00%> (-4.02%)` | :arrow_down: |
| [...eam/portability/api/beam\_provision\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcHJvdmlzaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `73.68% <0.00%> (-2.51%)` | :arrow_down: |
| [...e\_beam/portability/api/beam\_runner\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjJfZ3JwYy5weQ==) | `78.94% <0.00%> (-2.01%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | `96.78% <0.00%> (-0.57%)` | :arrow_down: |
| ... and [32 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...8f81a2c](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-956429335
R: @lukecwik , @chamikaramj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...f3b9159](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...f3b9159](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f3b9159) into [master](https://codecov.io/gh/apache/beam/commit/a5a0bd26ded0117240b2f6a967eb9f4c65209e6c?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (a5a0bd2) will **increase** coverage by `0.02%`.
> The diff coverage is `61.90%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.50% 83.52% +0.02%
==========================================
Files 445 445
Lines 61299 61385 +86
==========================================
+ Hits 51189 51274 +85
- Misses 10110 10111 +1
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `52.24% <25.00%> (-0.90%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.59% <70.58%> (+0.96%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: |
| [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `96.72% <0.00%> (-1.64%)` | :arrow_down: |
| [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.00% <0.00%> (-1.00%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.85% <0.00%> (-0.51%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.37% <0.00%> (-0.35%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.86% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/coders/row\_coder.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vY29kZXJzL3Jvd19jb2Rlci5weQ==) | `95.07% <0.00%> (ø)` | |
| ... and [6 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [a5a0bd2...f3b9159](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r745149530
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
+ if classpath:
+ cp_args = ['-classpath', os.pathsep.join(classpath)]
+ else:
+ cp_args = []
super().__init__(
- stub_class, ['java', '-jar', path_to_jar] + list(java_arguments))
+ stub_class,
+ ['java'] + cp_args + ['-jar', path_to_jar] + list(java_arguments))
Review comment:
Oh, that's really unfortunate. This really limits the ability to use classes in an ad-hoc way... I'll have to think about this some more.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15752d9) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **increase** coverage by `0.08%`.
> The diff coverage is `83.67%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.53% 83.61% +0.08%
==========================================
Files 445 445
Lines 61385 61414 +29
==========================================
+ Hits 51276 51354 +78
+ Misses 10109 10060 -49
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `58.73% <90.62%> (+5.59%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...eam/portability/api/beam\_expansion\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `57.89% <0.00%> (-4.02%)` | :arrow_down: |
| [...eam/portability/api/beam\_provision\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcHJvdmlzaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `73.68% <0.00%> (-2.51%)` | :arrow_down: |
| [...e\_beam/portability/api/beam\_runner\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjJfZ3JwYy5weQ==) | `78.94% <0.00%> (-2.01%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | `96.78% <0.00%> (-0.57%)` | :arrow_down: |
| ... and [32 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...15752d9](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15752d9) into [master](https://codecov.io/gh/apache/beam/commit/5c4f99b87de037c91079b0d1be98057feaaf762d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5c4f99b) will **increase** coverage by `0.00%`.
> The diff coverage is `83.67%`.
> :exclamation: Current head 15752d9 differs from pull request most recent head 4d8266e. Consider uploading reports for the commit 4d8266e to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
=======================================
Coverage 83.61% 83.61%
=======================================
Files 445 445
Lines 61363 61414 +51
=======================================
+ Hits 51310 51354 +44
- Misses 10053 10060 +7
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `58.73% <90.62%> (+5.59%)` | :arrow_up: |
| [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5weQ==) | `94.02% <0.00%> (-2.24%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.39% <0.00%> (-0.13%)` | :arrow_down: |
| [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.74% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/transforms/core.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9jb3JlLnB5) | `92.64% <0.00%> (+<0.01%)` | :arrow_up: |
| [sdks/python/apache\_beam/typehints/typehints.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHlwZWhpbnRzL3R5cGVoaW50cy5weQ==) | `93.43% <0.00%> (+0.09%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [6b14474...4d8266e](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lukecwik commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
lukecwik commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-973302175
The change in the shadow closure added the JUL logging adapter creating an infinite loop when logging during spark and flink tests.
Rolling back.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r742418804
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
I could find documentation of this file anywhere, but I think the code is self explanatory enough (for both non-wildcard and wild-card containing lists).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r742418804
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
I could find documentation of this file anywhere, but I think the code is self explanatory enough (for both non-wildcard and wild-card containing lists).
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
Review comment:
Done.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
Review comment:
Yes, that's the example above.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Looks like the file stager does handle directories. Updated the code to handle globs.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Ideally the file stager could handle those as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8375984) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **decrease** coverage by `0.03%`.
> The diff coverage is `37.50%`.
> :exclamation: Current head 8375984 differs from pull request most recent head 711b913. Consider uploading reports for the commit 711b913 to get more accurate results
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
- Coverage 83.53% 83.49% -0.04%
==========================================
Files 445 445
Lines 61385 61427 +42
==========================================
+ Hits 51276 51288 +12
- Misses 10109 10139 +30
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `47.80% <19.35%> (-5.34%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (-0.50%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.25% <0.00%> (-0.17%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...711b913](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (15752d9) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **increase** coverage by `0.08%`.
> The diff coverage is `83.67%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
+ Coverage 83.53% 83.61% +0.08%
==========================================
Files 445 445
Lines 61385 61414 +29
==========================================
+ Hits 51276 51354 +78
+ Misses 10109 10060 -49
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `58.73% <90.62%> (+5.59%)` | :arrow_up: |
| [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `87.80% <0.00%> (-7.32%)` | :arrow_down: |
| [...eam/portability/api/beam\_expansion\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fZXhwYW5zaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `57.89% <0.00%> (-4.02%)` | :arrow_down: |
| [...eam/portability/api/beam\_provision\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcHJvdmlzaW9uX2FwaV9wYjJfZ3JwYy5weQ==) | `73.68% <0.00%> (-2.51%)` | :arrow_down: |
| [...e\_beam/portability/api/beam\_runner\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fcnVubmVyX2FwaV9wYjJfZ3JwYy5weQ==) | `78.94% <0.00%> (-2.01%)` | :arrow_down: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.60% <0.00%> (-0.98%)` | :arrow_down: |
| [...ache\_beam/portability/api/beam\_job\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fam9iX2FwaV9wYjJfZ3JwYy5weQ==) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [...beam/portability/api/beam\_artifact\_api\_pb2\_grpc.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcG9ydGFiaWxpdHkvYXBpL2JlYW1fYXJ0aWZhY3RfYXBpX3BiMl9ncnBjLnB5) | `56.04% <0.00%> (-0.95%)` | :arrow_down: |
| [sdks/python/apache\_beam/io/textio.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vdGV4dGlvLnB5) | `96.78% <0.00%> (-0.57%)` | :arrow_down: |
| ... and [32 more](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...15752d9](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-970720304
Seems like Python failure is related: https://ci-beam.apache.org/job/beam_PreCommit_Python_Commit/20540/testReport/junit/apache_beam.utils.subprocess_server_test/JavaJarServerTest/test_classpath_jar/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-973129206
Thanks. LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
chamikaramj commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r741540490
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
Probably we should document.
* How to create an allowlist file to allow everything.
* How to create an allowlist file to just allow a given package.
##########
File path: sdks/python/apache_beam/utils/subprocess_server.py
##########
@@ -158,9 +158,14 @@ class JavaJarServer(SubprocessServer):
'local', (threading.local, ),
dict(__init__=lambda self: setattr(self, 'replacements', {})))()
- def __init__(self, stub_class, path_to_jar, java_arguments):
+ def __init__(self, stub_class, path_to_jar, java_arguments, classpath=None):
Review comment:
Could we add a unit test to cover this ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
+ expansion_service = expansion_service or BeamJarExpansionService(
Review comment:
So the default version will just offer the classes packaged with the <expansion-service> package ? I'm not sure whether this will be useful (it's whatever in that package and Beam core). Should we make providing classpath (or additional jars) required to make the default usage useful ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
Review comment:
JavaExternalTransform('fully.qualified.ClassName')(\<params\>) works as well, right ?
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -283,9 +283,21 @@ def _has_constructor(self):
class JavaExternalTransform(ptransform.PTransform):
"""A proxy for Java-implemented external transforms.
- One builds these transforms just as one would in Java.
+ One builds these transforms just as one would in Java, e.g.::
+
+ transform = JavaExternalTransform('fully.qualified.ClassName')(
+ contrustorArg, ...).builderMethod(...)
+
+ or::
+
+ JavaExternalTransform('fully.qualified.ClassName').staticConstructor(
+ ...).builderMethod1(...).builderMethod2(...)
"""
- def __init__(self, class_name, expansion_service=None):
+ def __init__(self, class_name, expansion_service=None, classpath=None):
Review comment:
Please document what classpath should be (seems like a list based on the usage below).
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/ExpansionServiceOptions.java
##########
@@ -53,6 +52,9 @@ public AllowList create(PipelineOptions options) {
String allowListFile =
options.as(ExpansionServiceOptions.class).getJavaClassLookupAllowlistFile();
if (allowListFile != null) {
+ if (allowListFile.equals("*")) {
Review comment:
Please document the updated usage in the corresponding property of ExpansionServiceOptions.
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,11 +667,13 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ to_stage = ','.join([path_to_jar] + list(classpath or []))
Review comment:
Seems like we assume that the provided classpath only contains jars ? Classpath also supports directories and wild cards.
https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r742418804
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
I could find documentation of this file anywhere, but I think the code is self explanatory enough (for both non-wildcard and wild-card containing lists).
##########
File path: sdks/java/expansion-service/src/main/java/org/apache/beam/sdk/expansion/service/JavaClassLookupTransformProvider.java
##########
@@ -508,12 +523,32 @@ static AllowList create(
@AutoValue
public abstract static class AllowedClass {
+ public static final List<String> WILDCARD = Collections.singletonList("*");
+
public abstract String getClassName();
public abstract List<String> getAllowedBuilderMethods();
public abstract List<String> getAllowedConstructorMethods();
+ public boolean isAllowedClass(String className) {
+ String pattern = getClassName();
Review comment:
I could find documentation of this file anywhere, but I think the code is self explanatory enough (for both non-wildcard and wild-card containing lists).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] robertwb commented on a change in pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
robertwb commented on a change in pull request #15857:
URL: https://github.com/apache/beam/pull/15857#discussion_r745209468
##########
File path: sdks/python/apache_beam/transforms/external.py
##########
@@ -655,22 +680,29 @@ class JavaJarExpansionService(object):
argument which will spawn a subprocess using this jar to expand the
transform.
"""
- def __init__(self, path_to_jar, extra_args=None):
- if extra_args is None:
- extra_args = ['{{PORT}}', f'--filesToStage={path_to_jar}']
+ def __init__(self, path_to_jar, extra_args=None, classpath=None):
self._path_to_jar = path_to_jar
self._extra_args = extra_args
+ self._classpath = classpath
self._service_count = 0
+ def _default_args(self):
+ to_stage = ','.join([self._path_to_jar] + sum(
Review comment:
It will always be set. This is a private method, only factored out for testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] edited a comment on pull request #15857: Allow wildcards for java class lookup transform providers.
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #15857:
URL: https://github.com/apache/beam/pull/15857#issuecomment-961468620
# [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#15857](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8375984) into [master](https://codecov.io/gh/apache/beam/commit/2f2e4fac9c9814ecba2e34d3700414fc4bc9c85d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (2f2e4fa) will **decrease** coverage by `0.02%`.
> The diff coverage is `37.50%`.
[](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #15857 +/- ##
==========================================
- Coverage 83.52% 83.49% -0.03%
==========================================
Files 445 445
Lines 61385 61427 +42
==========================================
+ Hits 51270 51288 +18
- Misses 10115 10139 +24
```
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/utils/subprocess\_server.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvc3VicHJvY2Vzc19zZXJ2ZXIucHk=) | `47.80% <19.35%> (-5.34%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `79.36% <70.58%> (+0.73%)` | :arrow_up: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.26% <0.00%> (-0.25%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.25% <0.00%> (-0.17%)` | :arrow_down: |
| [...eam/runners/interactive/interactive\_environment.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9lbnZpcm9ubWVudC5weQ==) | `91.72% <0.00%> (+0.34%)` | :arrow_up: |
| [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `86.36% <0.00%> (+0.50%)` | :arrow_up: |
| [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15857/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `76.58% <0.00%> (+0.97%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [2f2e4fa...8375984](https://codecov.io/gh/apache/beam/pull/15857?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org