You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "charlespnh (via GitHub)" <gi...@apache.org> on 2025/07/28 16:48:55 UTC
[PR] YAML MLTransform import more transforms [beam]
charlespnh opened a new pull request, #35716:
URL: https://github.com/apache/beam/pull/35716
Fixes https://github.com/apache/beam/issues/35693
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] YAML MLTransform import more transforms [beam]
Posted by "charlespnh (via GitHub)" <gi...@apache.org>.
charlespnh closed pull request #35716: YAML MLTransform import more transforms
URL: https://github.com/apache/beam/pull/35716
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] YAML MLTransform import more transforms [beam]
Posted by "charlespnh (via GitHub)" <gi...@apache.org>.
charlespnh commented on PR #35716:
URL: https://github.com/apache/beam/pull/35716#issuecomment-3133683145
Closing this sample PR. There's already work in progress fixing this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] YAML MLTransform import more transforms [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #35716:
URL: https://github.com/apache/beam/pull/35716#discussion_r2237259988
##########
sdks/python/apache_beam/ml/transforms/embeddings/open_ai.py:
##########
@@ -71,6 +75,12 @@ def __init__(
user: Optional[str] = None,
max_batch_size: Optional[int] = None,
):
+ if not openai:
+ raise ImportError(
+ "OpenAI Python SDK is required to use "
+ "OpenAITextEmbeddings. "
+ "Please install it with using `pip install openai`.")
Review Comment:
The phrase 'with using' is slightly ungrammatical. 'using' is sufficient and more natural here.
```suggestion
"Please install it using `pip install openai`.")
```
##########
sdks/python/apache_beam/ml/transforms/embeddings/huggingface.py:
##########
@@ -71,7 +71,7 @@ def __init__(
if not SentenceTransformer:
raise ImportError(
"sentence-transformers is required to use "
- "SentenceTransformerEmbeddings."
+ "SentenceTransformerEmbeddings. "
"Please install it with using `pip install sentence-transformers`.")
Review Comment:
The phrase 'with using' is slightly ungrammatical. 'using' is sufficient and more natural here.
```suggestion
"Please install it using `pip install sentence-transformers`.")
```
##########
sdks/python/apache_beam/yaml/yaml_ml.py:
##########
@@ -37,6 +41,17 @@
except ImportError:
tft = None # type: ignore
+# Iterate over all submodules in the 'embeddings' package path
+for module_info in pkgutil.iter_modules(embeddings.__path__,
+ f"{embeddings.__name__}."):
+ try:
+ submodule = importlib.import_module(module_info.name)
+ for name in getattr(submodule, '__all__', []):
+ _transform_constructors[name] = getattr(submodule, name)
+ except ImportError:
+ # Skip any modules that fail to import
+ continue
Review Comment:
Silently ignoring `ImportError` can make debugging difficult if a transform is unexpectedly missing. For example, a typo in an import statement within one of the embedding modules would be silenced. It would be better to log these errors to provide visibility into why a particular transform might not be available.
```suggestion
except ImportError as e:
# Skip any modules that fail to import, but log a warning.
logging.warning('Could not import %s, skipping: %s', module_info.name, e)
continue
```
##########
sdks/python/apache_beam/ml/transforms/embeddings/vertex_ai.py:
##########
@@ -89,6 +93,13 @@ def __init__(
location: Optional[str] = None,
credentials: Optional[Credentials] = None,
**kwargs):
+ if not vertexai:
+ raise ImportError(
+ "Vertex AI Python SDK is required to use "
+ "VertexAITextEmbeddings and VertexAIImageEmbeddings. "
+ "Please install it with using "
Review Comment:
The phrase 'with using' is slightly ungrammatical. 'using' is sufficient and more natural here.
```suggestion
"Please install it using "
```
##########
sdks/python/apache_beam/ml/transforms/embeddings/tensorflow_hub.py:
##########
@@ -41,6 +50,13 @@ class _TensorflowHubModelHandler(TFModelHandlerTensor):
Note: Intended for internal use only. No backwards compatibility guarantees.
"""
def __init__(self, preprocessing_url: Optional[str], *args, **kwargs):
+ if not (tf and hub and text):
+ raise ImportError(
+ "tensorflow, tensorflow_hub and tensorflow_text are required to use "
+ "TensorflowHubTextEmbeddings and TensorflowHubImageEmbeddings. "
+ "Please install them with using "
Review Comment:
The phrase 'with using' is slightly ungrammatical. 'using' is sufficient and more natural here.
```suggestion
"Please install them using "
```
##########
sdks/python/apache_beam/yaml/yaml_ml.py:
##########
@@ -20,10 +20,14 @@
from typing import Any
from typing import Optional
+import pkgutil
+import importlib
Review Comment:
To support logging import errors when dynamically loading modules, please add the `logging` import. This will help in debugging cases where a transform is not available as expected.
```suggestion
import importlib
import logging
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org