You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "ahmedabu98 (via GitHub)" <gi...@apache.org> on 2023/04/04 13:02:03 UTC
[GitHub] [beam] ahmedabu98 opened a new pull request, #26100: Use external config schema to construct Python SchemaTransform payload
ahmedabu98 opened a new pull request, #26100:
URL: https://github.com/apache/beam/pull/26100
Development on external SchemaTransforms may lead to changes in how their configurations look (ie. adding and removing fields, changing the order of fields). In addition to that, currently most Java SchemaTransformProviders inherit from [TypedSchemaTransformProvider](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/transforms/TypedSchemaTransformProvider.java), which infers its configuration schema by using a configuration class and [AutoValueSchema](https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/schemas/AutoValueSchema.java). While this approach is very convenient, the ordering of fields in the inferred schema is unfortunately not consistent. All of this is to say that the configuration schema of external transforms is prone to changes.
When we use an external SchemaTransform in Python, we build a payload that includes the configuration fields. These are the same fields used to set up the external SchemaTransform. Currently, we only use the input kwargs to construct the payload, so we are blind to what the external configuration schema actually is. The changes in this PR make it so that we first fetch the external configuration schema then construct the payload in accordance to that schema.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1158954600
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ fields_to_values = OrderedDict()
+ external_config_schema_fields = \
+ self._schematransform_config.configuration_schema._fields
+ kwargs_fields = tuple(self._kwargs.keys())
+
+ if self._strict_schema and external_config_schema_fields != kwargs_fields:
+ raise ValueError(
+ "Parameters in kwargs: %s do not match the external "
+ "SchemaTransform's configuration fields: %s" %
+ (kwargs_fields, external_config_schema_fields))
+
+ # The discover API allows us to obtain an ordered configuration schema
Review Comment:
Sounds good, I like keeping the less costly option as the default. So the default can continue using the existing `_get_schema_proto_and_payload()`. However, I think we can't use this method for the `rearrange_based_on_discovery` option because it builds the payload based off kwargs only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1158926921
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
Review Comment:
right, `strict_schema` flag should catch that too. will add that check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1498449621
Run Portable_Python PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1495933641
R: @chamikaramj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1495958090
## [Codecov](https://codecov.io/gh/apache/beam/pull/26100?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#26100](https://codecov.io/gh/apache/beam/pull/26100?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (348e722) into [master](https://codecov.io/gh/apache/beam/commit/619dcfab65bdbd6e512f609f206f0dc7349c593b?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (619dcfa) will **decrease** coverage by `0.06%`.
> The diff coverage is `75.86%`.
```diff
@@ Coverage Diff @@
## master #26100 +/- ##
==========================================
- Coverage 71.41% 71.36% -0.06%
==========================================
Files 782 783 +1
Lines 102856 102962 +106
==========================================
+ Hits 73457 73476 +19
- Misses 27922 28009 +87
Partials 1477 1477
```
| Flag | Coverage Δ | |
|---|---|---|
| python | `79.85% <75.86%> (-0.10%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/26100?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [sdks/python/apache\_beam/transforms/external.py](https://codecov.io/gh/apache/beam/pull/26100?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9leHRlcm5hbC5weQ==) | `80.26% <75.86%> (-0.26%)` | :arrow_down: |
... and [8 files with indirect coverage changes](https://codecov.io/gh/apache/beam/pull/26100/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1495966208
Run Python_Xlang_Gcp_Dataflow PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1158928440
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
Review Comment:
yeah good idea, will add that check
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1158957283
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ fields_to_values = OrderedDict()
+ external_config_schema_fields = \
+ self._schematransform_config.configuration_schema._fields
+ kwargs_fields = tuple(self._kwargs.keys())
+
+ if self._strict_schema and external_config_schema_fields != kwargs_fields:
+ raise ValueError(
+ "Parameters in kwargs: %s do not match the external "
+ "SchemaTransform's configuration fields: %s" %
+ (kwargs_fields, external_config_schema_fields))
+
+ # The discover API allows us to obtain an ordered configuration schema
Review Comment:
We can rearrange kwargs before the method call and use the same method, can't we ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1163109261
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -327,17 +327,61 @@ class SchemaAwareExternalTransform(ptransform.PTransform):
:param expansion_service: an expansion service to use. This should already be
available and the Schema-aware transforms to be used must already be
deployed.
+ :param rearrange_based_on_discovery: if this flag is set, the input kwargs
+ will be rearranged to match the order of fields in the external
+ SchemaTransform configuration. A discovery call will be made to fetch
+ the configuration.
:param classpath: (Optional) A list paths to additional jars to place on the
expansion service classpath.
:kwargs: field name to value mapping for configuring the schema transform.
keys map to the field names of the schema of the SchemaTransform
(in-order).
"""
- def __init__(self, identifier, expansion_service, classpath=None, **kwargs):
+ def __init__(
+ self,
+ identifier,
+ expansion_service,
+ rearrange_based_on_discovery=False,
+ classpath=None,
+ **kwargs):
self._expansion_service = expansion_service
- self._payload_builder = SchemaTransformPayloadBuilder(identifier, **kwargs)
+ self._kwargs = kwargs
self._classpath = classpath
+ _kwargs = kwargs
+ if rearrange_based_on_discovery:
+ _kwargs = self._rearrange_kwargs(identifier)
+
+ self._payload_builder = SchemaTransformPayloadBuilder(identifier, **_kwargs)
+
+ def _rearrange_kwargs(self, identifier):
+ # discover and fetch the external SchemaTransform configuration then
+ # use it to build an appropriate payload
+ schematransform_config = SchemaAwareExternalTransform.discover_config(
+ self._expansion_service, identifier)
+
+ external_config_fields = schematransform_config.configuration_schema._fields
+ ordered_kwargs = {}
Review Comment:
I think this has to be an OrderedMap. Otherwise the order is not guaranteed by Python.
https://docs.python.org/3/library/collections.html#collections.OrderedDict
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1163109261
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -327,17 +327,61 @@ class SchemaAwareExternalTransform(ptransform.PTransform):
:param expansion_service: an expansion service to use. This should already be
available and the Schema-aware transforms to be used must already be
deployed.
+ :param rearrange_based_on_discovery: if this flag is set, the input kwargs
+ will be rearranged to match the order of fields in the external
+ SchemaTransform configuration. A discovery call will be made to fetch
+ the configuration.
:param classpath: (Optional) A list paths to additional jars to place on the
expansion service classpath.
:kwargs: field name to value mapping for configuring the schema transform.
keys map to the field names of the schema of the SchemaTransform
(in-order).
"""
- def __init__(self, identifier, expansion_service, classpath=None, **kwargs):
+ def __init__(
+ self,
+ identifier,
+ expansion_service,
+ rearrange_based_on_discovery=False,
+ classpath=None,
+ **kwargs):
self._expansion_service = expansion_service
- self._payload_builder = SchemaTransformPayloadBuilder(identifier, **kwargs)
+ self._kwargs = kwargs
self._classpath = classpath
+ _kwargs = kwargs
+ if rearrange_based_on_discovery:
+ _kwargs = self._rearrange_kwargs(identifier)
+
+ self._payload_builder = SchemaTransformPayloadBuilder(identifier, **_kwargs)
+
+ def _rearrange_kwargs(self, identifier):
+ # discover and fetch the external SchemaTransform configuration then
+ # use it to build an appropriate payload
+ schematransform_config = SchemaAwareExternalTransform.discover_config(
+ self._expansion_service, identifier)
+
+ external_config_fields = schematransform_config.configuration_schema._fields
+ ordered_kwargs = {}
Review Comment:
I think this has to be an OrderedDict. Otherwise the order is not guaranteed by Python.
https://docs.python.org/3/library/collections.html#collections.OrderedDict
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] johnjcasey commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "johnjcasey (via GitHub)" <gi...@apache.org>.
johnjcasey commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1158868626
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
Review Comment:
do you also want to check that there are no kwargs beyond those in the schema?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1158961118
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ fields_to_values = OrderedDict()
+ external_config_schema_fields = \
+ self._schematransform_config.configuration_schema._fields
+ kwargs_fields = tuple(self._kwargs.keys())
+
+ if self._strict_schema and external_config_schema_fields != kwargs_fields:
+ raise ValueError(
+ "Parameters in kwargs: %s do not match the external "
+ "SchemaTransform's configuration fields: %s" %
+ (kwargs_fields, external_config_schema_fields))
+
+ # The discover API allows us to obtain an ordered configuration schema
Review Comment:
yeah that probably works actually, will try that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #26100:
URL: https://github.com/apache/beam/pull/26100#discussion_r1157573798
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
Review Comment:
I think it should be possible to use SchemaTransforms without the full config or the schema (i.e. just using the schema transform ID and a set of kwargs). Can you adjust the change so that the additional validation is optional ?
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
Review Comment:
Can we move the additional checks before this call and continue to use the existing external._get_schema_proto_and_payload() method ?
##########
sdks/python/apache_beam/transforms/external.py:
##########
@@ -180,14 +180,52 @@ def _get_named_tuple_instance(self):
class SchemaTransformPayloadBuilder(PayloadBuilder):
- def __init__(self, identifier, **kwargs):
- self._identifier = identifier
+ def __init__(self, schematransform_config, strict_schema=False, **kwargs):
+ self._schematransform_config = schematransform_config
+ self._strict_schema = strict_schema
self._kwargs = kwargs
+ def _get_schema_proto_and_payload(self, **kwargs):
+ named_fields = []
+ fields_to_values = OrderedDict()
+ external_config_schema_fields = \
+ self._schematransform_config.configuration_schema._fields
+ kwargs_fields = tuple(self._kwargs.keys())
+
+ if self._strict_schema and external_config_schema_fields != kwargs_fields:
+ raise ValueError(
+ "Parameters in kwargs: %s do not match the external "
+ "SchemaTransform's configuration fields: %s" %
+ (kwargs_fields, external_config_schema_fields))
+
+ # The discover API allows us to obtain an ordered configuration schema
Review Comment:
I think instead of the "strict_schema" option, we should do a "rearrange_based_on_discovery" option. If the option is not provided, we use kwargs as is without the overhead of the additional RPC (this will work for anything other than TypedSchemaTransformProvider). For TypedSchemaTransformProvider, we would set the "rearrange_based_on_discovery" option to true and would rearrange kwargs based on a discovery call before the "_get_schema_proto_and_payload" invocation. WDYT ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1495965811
Run Python_Xlang_Gcp_Direct PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj merged pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj merged PR #26100:
URL: https://github.com/apache/beam/pull/26100
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1495935963
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1499270807
R: @chamikaramj
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1499229886
`beam_PreCommit_Portable_Python_Commit` failing with:
```
Project 'py${lowestSupported}' not found in project ':sdks:python:test-suites:portable'
```
not sure what is causing this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] ahmedabu98 commented on pull request #26100: Use external config schema to construct Python SchemaTransform payload
Posted by "ahmedabu98 (via GitHub)" <gi...@apache.org>.
ahmedabu98 commented on PR #26100:
URL: https://github.com/apache/beam/pull/26100#issuecomment-1498448015
Run Python_Xlang_Gcp_Direct PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org