You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "riteshghorse (via GitHub)" <gi...@apache.org> on 2023/07/07 19:47:10 UTC
[GitHub] [beam] riteshghorse opened a new pull request, #27399: [WIP] Hugging Face pipeline support
riteshghorse opened a new pull request, #27399:
URL: https://github.com/apache/beam/pull/27399
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #27399: [WIP] Hugging Face pipeline support
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1626010696
## [Codecov](https://app.codecov.io/gh/apache/beam/pull/27399?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#27399](https://app.codecov.io/gh/apache/beam/pull/27399?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (8dd0ff2) into [master](https://app.codecov.io/gh/apache/beam/commit/a4281d8f35cf26bc62444f06b48d0019085e8e64?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (a4281d8) will **decrease** coverage by `0.31%`.
> The diff coverage is `3.78%`.
```diff
@@ Coverage Diff @@
## master #27399 +/- ##
==========================================
- Coverage 71.49% 71.19% -0.31%
==========================================
Files 858 864 +6
Lines 104715 105424 +709
==========================================
+ Hits 74865 75054 +189
- Misses 28302 28822 +520
Partials 1548 1548
```
| Flag | Coverage Δ | |
|---|---|---|
| python | `80.25% <3.78%> (-0.56%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://app.codecov.io/gh/apache/beam/pull/27399?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [.../apache\_beam/ml/inference/huggingface\_inference.py](https://app.codecov.io/gh/apache/beam/pull/27399?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vbWwvaW5mZXJlbmNlL2h1Z2dpbmdmYWNlX2luZmVyZW5jZS5weQ==) | `0.00% <0.00%> (ø)` | |
| [...xamples/inference/huggingface\_language\_modeling.py](https://app.codecov.io/gh/apache/beam/pull/27399?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZXhhbXBsZXMvaW5mZXJlbmNlL2h1Z2dpbmdmYWNlX2xhbmd1YWdlX21vZGVsaW5nLnB5) | `13.23% <13.23%> (ø)` | |
... and [23 files with indirect coverage changes](https://app.codecov.io/gh/apache/beam/pull/27399/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1652070012
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1648739755
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @liferoad for label python.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1280973248
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -109,6 +149,13 @@ def is_gpu_available_tensorflow(device):
return True
+def _validate_constructor_args_hf_pipeline(task, model):
+ if not task and not model:
+ raise RuntimeError(
+ 'Please provide both task and model to HuggingFacePipelineModelHandler.'
+ 'If the model already defines the task, no need to specify the task.')
Review Comment:
If we have a task, do we always need a model? Should this error just be `Please provide either a task or a model to HuggingFacePipelineModelHandler` instead of asking the user to provide both?
##########
sdks/python/apache_beam/ml/inference/huggingface_inference_it_test.py:
##########
@@ -74,6 +75,37 @@ def test_hf_language_modeling(self):
predicted_predicted_text = predictions_dict[text]
self.assertEqual(actual_predicted_text, predicted_predicted_text)
+ def test_hf_pipeline(self):
+ test_pipeline = TestPipeline(is_integration_test=True)
+ # Path to text file containing some questions and context
+ input_file = 'gs://apache-beam-ml/datasets/custom/questions.txt'
+ output_file_dir = 'gs://apache-beam-ml/hf/testing/predictions'
Review Comment:
Will this break if the underlying model changes? Could we pin to a single revision?
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +533,130 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
"""
- return "BeamML_HuggingFaceModelHandler_Tensor"
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
Review Comment:
Nit: Could we change model to pipeline in this function and others? I think that will help for readability, and some might try to use this as an example of what to do.
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +533,130 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
"""
- return "BeamML_HuggingFaceModelHandler_Tensor"
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
+ predicitons = model(batch, **inference_args)
+ return predicitons
+
+
+class HuggingFacePipelineModelHandler(ModelHandler[str,
+ PredictionResult,
+ Pipeline]):
+ def __init__(
+ self,
+ task: Union[str, PipelineTask] = "",
+ model=None,
+ *,
+ inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
+ load_model_args: Optional[Dict[str, Any]] = None,
Review Comment:
Should this be `load_pipeline_args` per https://docs.google.com/document/d/107T71QHMxvu67PyG-mkHwRhm7W2-07jihnqUMz-yuEM/edit
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -71,6 +72,39 @@
Iterable[PredictionResult]]
+class PipelineTask(str, Enum):
Review Comment:
Could you add a short docstring explaining the purpose of this Enum?
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +533,130 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
"""
- return "BeamML_HuggingFaceModelHandler_Tensor"
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
+ predicitons = model(batch, **inference_args)
+ return predicitons
+
+
+class HuggingFacePipelineModelHandler(ModelHandler[str,
+ PredictionResult,
+ Pipeline]):
+ def __init__(
+ self,
+ task: Union[str, PipelineTask] = "",
+ model=None,
+ *,
+ inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
+ load_model_args: Optional[Dict[str, Any]] = None,
+ inference_args: Optional[Dict[str, Any]] = None,
+ min_batch_size: Optional[int] = None,
+ max_batch_size: Optional[int] = None,
+ large_model: bool = False,
+ **kwargs):
+ """
+ Implementation of the ModelHandler interface for Hugging Face Pipelines.
+
+ **Note:** To specify which device to use (CPU/GPU),
+ use the load_model_args with key-value as you would do in the usual
+ Hugging Face pipeline. Ex: load_model_args={'device':0})
+
+ Example Usage model::
+ pcoll | RunInference(HuggingFacePipelineModelHandler(
+ task="fill-mask"))
+
+ Args:
+ task (str or enum.Enum): task supported by HuggingFace Pipelines.
+ Accepts a string task or an enum.Enum from PipelineTask.
+ model : path to pretrained model on Hugging Face Models Hub to use custom
+ model for the chosen task. If the model already defines the task then
+ no need to specify the task parameter.
+ inference_fn: the inference function to use during RunInference.
+ Default is _default_pipeline_inference_fn.
+ load_model_args (Dict[str, Any]): keyword arguments to provide load
+ options while loading models from Hugging Face Hub. Defaults to None.
+ inference_args (Dict[str, Any]): Non-batchable arguments
+ required as inputs to the model's inference function.
+ Defaults to None.
+ min_batch_size: the minimum batch size to use when batching inputs.
+ max_batch_size: the maximum batch size to use when batching inputs.
+ large_model: set to true if your model is large enough to run into
+ memory pressure if you load multiple copies. Given a model that
+ consumes N memory and a machine with W cores and M memory, you should
+ set this to True if N*W > M.
+ kwargs: 'env_vars' can be used to set environment variables
+ before loading the model.
+
+ **Supported Versions:** HuggingFacePipelineModelHandler supports
+ transformers>=4.18.0.
+ """
+ self._task = task
+ self._model = model
+ self._inference_fn = inference_fn
+ self._load_model_args = load_model_args if load_model_args else {}
+ self._inference_args = inference_args if inference_args else {}
+ self._batching_kwargs = {}
+ self._framework = "torch"
+ self._env_vars = kwargs.get('env_vars', {})
+ if min_batch_size is not None:
+ self._batching_kwargs['min_batch_size'] = min_batch_size
+ if max_batch_size is not None:
+ self._batching_kwargs['max_batch_size'] = max_batch_size
+ self._large_model = large_model
+ _validate_constructor_args_hf_pipeline(self._task, self._model)
+
+ def load_model(self):
+ return pipeline(task=self._task, model=self._model, **self._load_model_args)
+
+ def run_inference(
+ self,
+ batch: Sequence[str],
+ model: Pipeline,
+ inference_args: Optional[Dict[str, Any]] = None
+ ) -> Iterable[PredictionResult]:
+ """
+ Runs inferences on a batch of examples passed as a string resource.
+ These can either be string sentences, or string path to images or
+ audio files.
+
+ Args:
+ batch: A sequence of strings resources.
+ model: A Hugging Face Pipeline.
+ inference_args: Non-batchable arguments required as inputs to the model's
+ inference function.
+ Returns:
+ An Iterable of type PredictionResult.
+ """
+ inference_args = {} if not inference_args else inference_args
+ predictions = self._inference_fn(batch, model, inference_args)
+ return _convert_to_result(batch, predictions)
+
+ def update_model_path(self, model_path: Optional[str] = None):
+ self._model = model_path if model_path else self._model
Review Comment:
I'm trying to figure out what model updates mean in the context of hugging face pipelines. I imagine not too many people would do it, but I think leaving it makes sense as is. If they do updates, its unlikely to be a task update. Could you add a brief docstring to this function explaining what model updates mean.
##########
sdks/python/apache_beam/ml/inference/huggingface_inference_it_test.py:
##########
@@ -74,6 +75,37 @@ def test_hf_language_modeling(self):
predicted_predicted_text = predictions_dict[text]
self.assertEqual(actual_predicted_text, predicted_predicted_text)
+ def test_hf_pipeline(self):
+ test_pipeline = TestPipeline(is_integration_test=True)
+ # Path to text file containing some questions and context
+ input_file = 'gs://apache-beam-ml/datasets/custom/questions.txt'
+ output_file_dir = 'gs://apache-beam-ml/hf/testing/predictions'
Review Comment:
Ooh, actually same question about the existing tests
##########
sdks/python/apache_beam/examples/inference/huggingface_question_answering.py:
##########
@@ -0,0 +1,139 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+""""A pipeline that uses RunInference to perform Question Answering using the
+model from Hugging Face Models Hub.
+
+This pipeline takes questions and context from a custom text file separated by
+a semicolon. These are converted to SquadExamples by using the utility provided
+by transformers.QuestionAnsweringPipeline and passed to the model handler.
+We just provide the model name here because the model repository specifies the
+task that it will do. The pipeline then writes the prediction to an output
+file in which users can then compare against the original context.
+"""
+
+import argparse
+import logging
+from typing import Iterable
+from typing import Tuple
+
+import apache_beam as beam
+from apache_beam.ml.inference.base import KeyedModelHandler
+from apache_beam.ml.inference.base import PredictionResult
+from apache_beam.ml.inference.base import RunInference
+from apache_beam.ml.inference.huggingface_inference import HuggingFacePipelineModelHandler
+from apache_beam.ml.inference.huggingface_inference import PipelineTask
+from apache_beam.options.pipeline_options import PipelineOptions
+from apache_beam.options.pipeline_options import SetupOptions
+from apache_beam.runners.runner import PipelineResult
+from transformers import QuestionAnsweringPipeline
+
+
+class PostProcessor(beam.DoFn):
+ """Processes the PredictionResult to get the predicted answer.
+
+ Hugging Face Pipeline for Question Answering returns a dictionary
+ with score, start and end index of answer and the answer.
+ """
+ def process(self, result: Tuple[str, PredictionResult]) -> Iterable[str]:
+ text, prediction = result
+ predicted_answer = prediction.inference['answer']
+ yield text + ';' + predicted_answer
+
+
+def preprocess(text):
+ if len(text.strip()) > 0:
+ question, context = text.split(';')
+ yield (question, context)
+
+
+def create_squad_example(text):
+ question, context = text
+ yield question, QuestionAnsweringPipeline.create_sample(question, context)
Review Comment:
Could you add a docstring to this function explaining what it is doing? Probably good to do that for the other preprocessing function too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1660336580
assign to next reviewer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1660541929
R: @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1655903796
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1660545656
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1652199125
hugging face integration tests passed on 3.8 PostCommit
<img width="1267" alt="image" src="https://github.com/apache/beam/assets/25881114/3d516923-eb3a-434b-b8dd-d7e52101f2c1">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1650118940
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1281049162
##########
sdks/python/apache_beam/ml/inference/huggingface_inference_it_test.py:
##########
@@ -74,6 +75,37 @@ def test_hf_language_modeling(self):
predicted_predicted_text = predictions_dict[text]
self.assertEqual(actual_predicted_text, predicted_predicted_text)
+ def test_hf_pipeline(self):
+ test_pipeline = TestPipeline(is_integration_test=True)
+ # Path to text file containing some questions and context
+ input_file = 'gs://apache-beam-ml/datasets/custom/questions.txt'
+ output_file_dir = 'gs://apache-beam-ml/hf/testing/predictions'
Review Comment:
Sounds good. yeah internal changes would corrupt our model. Pinning it to `4.30.0` for `huggingface_tests_requirements.txt`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1273970878
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +499,129 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
+ """
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
+ predicitons = model(batch, **inference_args)
+ return predicitons
+
+
+class HuggingFacePipelineModelHandler(ModelHandler[str,
+ PredictionResult,
+ Pipeline]):
+ def __init__(
+ self,
+ task: str = "",
+ model=None,
+ *,
+ inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
+ load_model_args: Optional[Dict[str, Any]] = None,
+ inference_args: Optional[Dict[str, Any]] = None,
+ min_batch_size: Optional[int] = None,
+ max_batch_size: Optional[int] = None,
+ large_model: bool = False,
+ **kwargs):
+ """
+ Implementation of the ModelHandler interface for Hugging Face Pipelines.
+
+ **Note:** To specify which device to use (CPU/GPU),
+ use the load_model_args with key-value as you would do in the usual
+ Hugging Face pipeline. Ex: load_model_args={'device':0})
+
+ Example Usage model::
+ pcoll | RunInference(HuggingFacePipelineModelHandler(
+ task="fill-mask"))
+
+ Args:
+ task (str): task supported by HuggingFace Pipelines.
Review Comment:
sounds good
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] liferoad commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1273960409
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +499,129 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
+ """
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
+ predicitons = model(batch, **inference_args)
+ return predicitons
+
+
+class HuggingFacePipelineModelHandler(ModelHandler[str,
+ PredictionResult,
+ Pipeline]):
+ def __init__(
+ self,
+ task: str = "",
+ model=None,
+ *,
+ inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
+ load_model_args: Optional[Dict[str, Any]] = None,
+ inference_args: Optional[Dict[str, Any]] = None,
+ min_batch_size: Optional[int] = None,
+ max_batch_size: Optional[int] = None,
+ large_model: bool = False,
+ **kwargs):
+ """
+ Implementation of the ModelHandler interface for Hugging Face Pipelines.
+
+ **Note:** To specify which device to use (CPU/GPU),
+ use the load_model_args with key-value as you would do in the usual
+ Hugging Face pipeline. Ex: load_model_args={'device':0})
+
+ Example Usage model::
+ pcoll | RunInference(HuggingFacePipelineModelHandler(
+ task="fill-mask"))
+
+ Args:
+ task (str): task supported by HuggingFace Pipelines.
Review Comment:
Better provide the link for users to check what tasks are: https://huggingface.co/transformers/v4.12.5/_modules/transformers/pipelines.html#pipeline has the current list. We could support both enum and str.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] liferoad commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1273867463
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +499,129 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
+ """
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
+ predicitons = model(batch, **inference_args)
+ return predicitons
+
+
+class HuggingFacePipelineModelHandler(ModelHandler[str,
+ PredictionResult,
+ Pipeline]):
+ def __init__(
+ self,
+ task: str = "",
+ model=None,
+ *,
+ inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
+ load_model_args: Optional[Dict[str, Any]] = None,
+ inference_args: Optional[Dict[str, Any]] = None,
+ min_batch_size: Optional[int] = None,
+ max_batch_size: Optional[int] = None,
+ large_model: bool = False,
+ **kwargs):
+ """
+ Implementation of the ModelHandler interface for Hugging Face Pipelines.
+
+ **Note:** To specify which device to use (CPU/GPU),
+ use the load_model_args with key-value as you would do in the usual
+ Hugging Face pipeline. Ex: load_model_args={'device':0})
+
+ Example Usage model::
+ pcoll | RunInference(HuggingFacePipelineModelHandler(
+ task="fill-mask"))
+
+ Args:
+ task (str): task supported by HuggingFace Pipelines.
Review Comment:
str or enum?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1650119278
Run Python 3.11 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1281062630
##########
sdks/python/apache_beam/ml/inference/huggingface_inference_it_test.py:
##########
@@ -74,6 +75,37 @@ def test_hf_language_modeling(self):
predicted_predicted_text = predictions_dict[text]
self.assertEqual(actual_predicted_text, predicted_predicted_text)
+ def test_hf_pipeline(self):
+ test_pipeline = TestPipeline(is_integration_test=True)
+ # Path to text file containing some questions and context
+ input_file = 'gs://apache-beam-ml/datasets/custom/questions.txt'
+ output_file_dir = 'gs://apache-beam-ml/hf/testing/predictions'
Review Comment:
I actually mean the model itself that the pipeline is built on top of, e.g. if they update from v2 to v3 - https://huggingface.co/deepset/roberta-base-squad2/commit/deedc3e42208524e0df3d9149d1f26aa6934f05f
I guess that's _that_ likely to happen, but we could just pin to v2 or even a commit sha with the revision parameter - https://huggingface.co/docs/transformers/main_classes/pipelines#transformers.pipeline.revision
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1653729392
Run Python 3.8 PostCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1655821534
PTAL, ready for review. Hugging Face integration tests passed. Some other tests unrelated to this PR are failing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] liferoad commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1663095349
Thanks, feel free to resolve my comments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on PR #27399:
URL: https://github.com/apache/beam/pull/27399#issuecomment-1660318680
assign next reviewer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on a diff in pull request #27399: [Python] Hugging Face pipeline support
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on code in PR #27399:
URL: https://github.com/apache/beam/pull/27399#discussion_r1273932757
##########
sdks/python/apache_beam/ml/inference/huggingface_inference.py:
##########
@@ -483,6 +499,129 @@ def share_model_across_processes(self) -> bool:
def get_metrics_namespace(self) -> str:
"""
Returns:
- A namespace for metrics collected by the RunInference transform.
+ A namespace for metrics collected by the RunInference transform.
+ """
+ return 'BeamML_HuggingFaceModelHandler_Tensor'
+
+
+def _convert_to_result(
+ batch: Iterable,
+ predictions: Union[Iterable, Dict[Any, Iterable]],
+ model_id: Optional[str] = None,
+) -> Iterable[PredictionResult]:
+ return [
+ PredictionResult(x, y, model_id) for x, y in zip(batch, [predictions])
+ ]
+
+
+def _default_pipeline_inference_fn(
+ batch, model, inference_args) -> Iterable[PredictionResult]:
+ predicitons = model(batch, **inference_args)
+ return predicitons
+
+
+class HuggingFacePipelineModelHandler(ModelHandler[str,
+ PredictionResult,
+ Pipeline]):
+ def __init__(
+ self,
+ task: str = "",
+ model=None,
+ *,
+ inference_fn: PipelineInferenceFn = _default_pipeline_inference_fn,
+ load_model_args: Optional[Dict[str, Any]] = None,
+ inference_args: Optional[Dict[str, Any]] = None,
+ min_batch_size: Optional[int] = None,
+ max_batch_size: Optional[int] = None,
+ large_model: bool = False,
+ **kwargs):
+ """
+ Implementation of the ModelHandler interface for Hugging Face Pipelines.
+
+ **Note:** To specify which device to use (CPU/GPU),
+ use the load_model_args with key-value as you would do in the usual
+ Hugging Face pipeline. Ex: load_model_args={'device':0})
+
+ Example Usage model::
+ pcoll | RunInference(HuggingFacePipelineModelHandler(
+ task="fill-mask"))
+
+ Args:
+ task (str): task supported by HuggingFace Pipelines.
Review Comment:
It is a str. There are a lot of tasks to be defined under enum for us https://huggingface.co/docs/transformers/main/main_classes/pipelines
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org