You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "AnandInguva (via GitHub)" <gi...@apache.org> on 2023/03/23 15:31:40 UTC
[GitHub] [beam] AnandInguva opened a new pull request, #25947: Add documentation for the auto model updates
AnandInguva opened a new pull request, #25947:
URL: https://github.com/apache/beam/pull/25947
**Please** add a meaningful description for your change here
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1146708146
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
Review Comment:
`It is used to load the model for inference.` - could you make it explicit that this should be in the same format that the given framework expects for a saved model file (e.g. state_dict for PyTorch)?
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+**Note**: The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
Review Comment:
Also, maybe you could link to the Watch Transform that you created itself as a pattern to follow?
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+**Note**: The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
+
+**Note**: If the main PCollection emits inputs and side input has yet to receive inputs, the main PCollection will get buffered until there is
+ an update to the side input. This could happen with Global windowed side inputs with data driven triggers such as `AfterCount`, `AfterProcessingTime`. So until there is an update to the side input, emit the default/initial model id that is used to pass the respective `ModelHandler` as side input..
Review Comment:
```suggestion
an update to the side input. This could happen with Global windowed side inputs with data driven triggers such as `AfterCount`, `AfterProcessingTime`. So until there is an update to the side input, emit the default/initial model id that is used to pass the respective `ModelHandler` as side input.
```
nit
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+**Note**: The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
Review Comment:
This is probably hard for the average developer to understand - could we lead with the WatchTransform pattern (ideally with a small code snippet) and then include the generalized ModelMetadata approach as something like "if just using a file watch transform doesn't work for you"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1151906761
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
Review Comment:
```suggestion
title: "Auto Update ML models using WatchFilePattern"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on pull request #25947: Add documentation for the auto model updates
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1482898316
> once that is done, I can modify the current page with a link to that page
SGTM, I'd like to block this PR on that though since that's the main pattern we want people following
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1486977431
PTAL @rszper @damccorm. I addressed the comments. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25947: Add documentation for the auto model updates
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1481753058
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1481751857
R: @damccorm @rszper
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1153688583
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` to run inference on images using TensorFlow models. It uses a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to update the model.
+
+Using side inputs, you can update your model (which is passed in a `ModelHandler` configuration object) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection` of ModelMetadata compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
Review Comment:
```suggestion
`model_metadata_pcoll` expects a `PCollection` of ModelMetadata compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton). Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1490802842
@damccorm what do you think about the content here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1146891464
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
Review Comment:
Is slowly updating a type of side input pattern? If so, I think it should be hyphenated:
Slowly-updating side input pattern
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+**Note**: The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
+
+**Note**: If the main PCollection emits inputs and side input has yet to receive inputs, the main PCollection will get buffered until there is
Review Comment:
```suggestion
**Note**: If the main PCollection emits inputs and a side input has yet to receive inputs, the main PCollection is buffered until there is
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+**Note**: The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
+
+**Note**: If the main PCollection emits inputs and side input has yet to receive inputs, the main PCollection will get buffered until there is
Review Comment:
We generally don't want two notes in a row. Does this content need to be in a note? Can it just be text in the file? It both sections must be notes, could they be combined into one note?
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
+to update the models used by the ModelHandler in the RunInference PTransform without the need of stopping the pipeline for the model updates.
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+**Note**: The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
+
+**Note**: If the main PCollection emits inputs and side input has yet to receive inputs, the main PCollection will get buffered until there is
+ an update to the side input. This could happen with Global windowed side inputs with data driven triggers such as `AfterCount`, `AfterProcessingTime`. So until there is an update to the side input, emit the default/initial model id that is used to pass the respective `ModelHandler` as side input..
Review Comment:
```suggestion
an update to the side input. This could happen with Global windowed side inputs with data driven triggers such as `AfterCount`, `AfterProcessingTime`. Until the side input is updated, emit the default or initial model ID that is used to pass the respective `ModelHandler` as a side input.
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
+The RunInference PTransform will accept a side input of [ModelMetadata](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata), which is a `NamedTuple` containing the `model_id` and `model_name`,
Review Comment:
```suggestion
To update models used by the model handler in the RunInference PTransform without stopping the pipeline, use a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input, which is a `NamedTuple` containing the `model_id` and `model_name`.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1152254375
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
Review Comment:
```suggestion
# Use WatchFilePattern to auto-update ML models in RunInference
```
I think this is a clearer title for the non-beam user. The "how" (side inputs) is sufficiently explained below, so the title can just focus on the behavior the user is trying to achieve.
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
Review Comment:
```suggestion
The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` to run inference on images using TensorFlow models. It uses a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to update the model.
```
Suggestion for clarity - the current form makes it sound like the `ModelMetadata` (or maybe the side input) is the thing running inference
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
Review Comment:
```suggestion
Using side inputs, you can update your model (which is passed in a `ModelHandler` configuration object) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
```
This mirrors how we talk about `ModelHandlers` elsewhere in our docs
##########
website/www/site/layouts/partials/section-menu/en/documentation.html:
##########
@@ -226,6 +226,7 @@
<li><a href="/documentation/ml/tensorrt-runinference">Build a custom model handler with TensorRT</a></li>
<li><a href="/documentation/ml/large-language-modeling">Use LLM inference</a></li>
<li><a href="/documentation/ml/multi-language-inference/">Build a multi-language inference pipeline</a></li>
+ <li><a href="/documentation/ml/side-input-updates/">Use side inputs to automatically update models</a></li>
Review Comment:
Same thing, I think focusing on the desired behavior vs the mechanism will help newer users
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
Review Comment:
@rszper this following section has a lot of code blocks which collectively make it kinda hard to read IMO (http://apache-beam-website-pull-requests.storage.googleapis.com/25947/documentation/ml/side-input-updates/index.html), but I don't know of a great way around it - any ideas on how we could make this section a little cleaner looking/easier to read? The content itself is correct
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [`AsSingleton`](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+After the pipeline starts processing data and when you see some outputs emitted from the RunInference `PTransform`, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as a side input.
+
+**Note**: Side input update frequency is non-deterministic and can have longer intervals between updates.
+
+```python
+import apache_beam as beam
+from apache_beam.ml.inference.utils import WatchFilePattern
+from apache_beam.ml.inference.base import RunInference
+with beam.Pipeline() as pipeline:
+
+ file_pattern = 'gs://<your-bucket>/*.h5'
+ pubsub_topic = '<topic_emitting_image_names>'
+
+ side_input_pcoll = (
+ pipeline
+ | "FilePatternUpdates" >> WatchFilePattern(file_pattern=file_pattern))
+
+ images_pcoll = (
+ pipeline
+ | "ReadFromPubSub" >> beam.io.ReadFromPubSub(topic=pubsub_topic)
+ | "DecodeBytes" >> beam.Map(lambda x: x.decode('utf-8'))
+ | "PreProcessImage" >> beam.Map(read_image)
+ )
+
+ inference_pcoll = (
+ images_pcoll
+ | "RunInference" >> RunInference(
+ model_handler=tf_model_handler,
+ model_metadata_pcoll=side_input_pcoll))
+
+```
+
+
+## Post-process the `PredictionResult` object
+
+When the inference is complete, RunInference outputs a `PredictionResult` object that contains `example`, `inference`, and `model_id`. Here, the `model_id` is used to identify which model is used for running the inference.
Review Comment:
```suggestion
When the inference is complete, RunInference outputs a `PredictionResult` object that contains `example`, `inference`, and `model_id` fields. The `model_id` is used to identify which model is used for running the inference.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [`AsSingleton`](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+After the pipeline starts processing data and when you see some outputs emitted from the RunInference `PTransform`, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as a side input.
+
+**Note**: Side input update frequency is non-deterministic and can have longer intervals between updates.
+
+```python
+import apache_beam as beam
+from apache_beam.ml.inference.utils import WatchFilePattern
+from apache_beam.ml.inference.base import RunInference
+with beam.Pipeline() as pipeline:
+
+ file_pattern = 'gs://<your-bucket>/*.h5'
+ pubsub_topic = '<topic_emitting_image_names>'
+
+ side_input_pcoll = (
+ pipeline
+ | "FilePatternUpdates" >> WatchFilePattern(file_pattern=file_pattern))
+
+ images_pcoll = (
+ pipeline
+ | "ReadFromPubSub" >> beam.io.ReadFromPubSub(topic=pubsub_topic)
+ | "DecodeBytes" >> beam.Map(lambda x: x.decode('utf-8'))
+ | "PreProcessImage" >> beam.Map(read_image)
+ )
+
+ inference_pcoll = (
+ images_pcoll
+ | "RunInference" >> RunInference(
+ model_handler=tf_model_handler,
+ model_metadata_pcoll=side_input_pcoll))
+
+```
+
+
+## Post-process the `PredictionResult` object
+
+When the inference is complete, RunInference outputs a `PredictionResult` object that contains `example`, `inference`, and `model_id`. Here, the `model_id` is used to identify which model is used for running the inference.
+
+```python
+from apache_beam.ml.inference.base import PredictionResult
+
+class PostProcessor(beam.DoFn):
+ """
+ Process the PredictionResult to get the predicted label and model id used for inference.
+ """
+ def process(self, element: PredictionResult) -> typing.Iterable[str]:
+ predicted_class = numpy.argmax(element.inference[0], axis=-1)
+ labels_path = tf.keras.utils.get_file(
+ 'ImageNetLabels.txt',
+ 'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt'
+ )
+ imagenet_labels = numpy.array(open(labels_path).read().splitlines())
+ predicted_class_name = imagenet_labels[predicted_class]
+ return predicted_class_name.title(), element.model_id
+
+post_processor_pcoll = (inference_pcoll | "PostProcessor" >> PostProcessor())
+```
+
+## Run the pipeline
+```python
+result = pipeline.run().wait_until_finish()
+```
+**Note**: The `model_name` of the `ModelMetaData` object will be attached as prefix to the [metrics](https://beam.apache.org/documentation/ml/runinference-metrics/) calculated by the RunInference `PTransform`.
+
+## Final remarks
+Use this example as a pattern when using side inputs with the RunInference `PTransform` to auto-update the models without stopping the pipeline. You can see a similar example for PyTorch on [GitHub](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_image_classification_with_side_inputs.py).
Review Comment:
```suggestion
You can use this example as a pattern when using side inputs with the RunInference `PTransform` to auto-update the models without stopping the pipeline. You can see a similar example for PyTorch on [GitHub](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_image_classification_with_side_inputs.py).
```
Nit: consistency of style with the next sentence.
##########
website/www/site/layouts/partials/section-menu/en/documentation.html:
##########
@@ -226,6 +226,7 @@
<li><a href="/documentation/ml/tensorrt-runinference">Build a custom model handler with TensorRT</a></li>
<li><a href="/documentation/ml/large-language-modeling">Use LLM inference</a></li>
<li><a href="/documentation/ml/multi-language-inference/">Build a multi-language inference pipeline</a></li>
+ <li><a href="/documentation/ml/side-input-updates/">Use side inputs to automatically update models</a></li>
Review Comment:
```suggestion
<li><a href="/documentation/ml/side-input-updates/">Update your model during pipeline execution</a></li>
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [`AsSingleton`](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
Review Comment:
```suggestion
`model_metadata_pcoll` expects a `PCollection` of ModelMetadata compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
```
I think this reads/renders more cleanly (code block links don't render well, see this line in http://apache-beam-website-pull-requests.storage.googleapis.com/25947/documentation/ml/side-input-updates/index.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1153690833
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` to run inference on images using TensorFlow models. It uses a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to update the model.
+
+Using side inputs, you can update your model (which is passed in a `ModelHandler` configuration object) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection` of ModelMetadata compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
Review Comment:
I removed the word `view`. We call (or may be I call it) the PCollection passed to the side input as `view` because in RunInference, it is viewed as a Singleton.
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` to run inference on images using TensorFlow models. It uses a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to update the model.
+
+Using side inputs, you can update your model (which is passed in a `ModelHandler` configuration object) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1482134568
@damccorm I am thinking of adding a webpage similar to this https://github.com/apache/beam/blob/ddae966f3346fbe247486324cbf8a8a532895316/website/www/site/content/en/documentation/ml/multi-language-inference.md?plain=1#L18. The use case will be using TFModelHandler and WatchFilePattern, I am working on it now. once that is done, I can modify the current page with a link to that page
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1151125750
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
Review Comment:
Maybe change this to:
Auto-update ML models
I think this primarily appears in the window/tab in the browser, so shorter might be better. We usually have this match the H1 title.
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
Review Comment:
```suggestion
or by configuring a custom side input `PCollection` that defines the logic for the model update.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
Review Comment:
```suggestion
This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
Review Comment:
```suggestion
the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
Review Comment:
```suggestion
## Pre-process images for inference
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
Review Comment:
```suggestion
2. Read and pre-process the images using the `read_image` function.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
Review Comment:
```suggestion
3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
Review Comment:
```suggestion
1. Get the image names from the Pub/Sub topic.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Since the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+Once the pipeline starts processing data and when you see some outputs emitted from the `RunInference` PTransform, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as side input.
+
+**Note**: Side input update frequency is non-deterministic and can have longer interval between updates.
Review Comment:
```suggestion
**Note**: Side input update frequency is non-deterministic and can have longer intervals between updates.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Since the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+Once the pipeline starts processing data and when you see some outputs emitted from the `RunInference` PTransform, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as side input.
+
+**Note**: Side input update frequency is non-deterministic and can have longer interval between updates.
+
+```python
+import apache_beam as beam
+from apache_beam.ml.inference.utils import WatchFilePattern
+from apache_beam.ml.inference.base import RunInference
+with beam.Pipeline() as pipeline:
+
+ file_pattern = 'gs://<your-bucket>/*.h5'
+ pubsub_topic = '<topic_emitting_image_names>'
+
+ side_input_pcoll = (
+ pipeline
+ | "FilePatternUpdates" >> WatchFilePattern(file_pattern=file_pattern))
+
+ images_pcoll = (
+ pipeline
+ | "ReadFromPubSub" >> beam.io.ReadFromPubSub(topic=pubsub_topic)
+ | "DecodeBytes" >> beam.Map(lambda x: x.decode('utf-8'))
+ | "PreProcessImage" >> beam.Map(read_image)
+ )
+
+ inference_pcoll = (
+ images_pcoll
+ | "RunInference" >> RunInference(
+ model_handler=tf_model_handler,
+ model_metadata_pcoll=side_input_pcoll))
+
+```
+
+
+### Post-process `PredictionResult` object
+
+Once the inference is done, RunInference outputs `PredictionResult` object that contains `example`, `inference` and, `model_id`. Here, the `model_id` is used to identify which model is used for running the inference.
Review Comment:
```suggestion
When the inference is complete, RunInference outputs a `PredictionResult` object that contains `example`, `inference`, and `model_id`. Here, the `model_id` is used to identify which model is used for running the inference.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
Review Comment:
```suggestion
For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
Review Comment:
```suggestion
## Set up the source
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` loads a model using weights and a model class initially, and during the update, If you pass in weights from a different model class when you update the model via side inputs, it won’t load properly because it’s expecting the weights from the original model class.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
Review Comment:
```suggestion
* `model_name`: Human-readable name for the model. You can use this name to identify the model in the metrics generated by the RunInference transform.
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` loads a model using weights and a model class initially, and during the update, If you pass in weights from a different model class when you update the model via side inputs, it won’t load properly because it’s expecting the weights from the original model class.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+Use cases:
+ * Use `WatchFilePattern` as side input to the RunInference PTransform for the auto updates of the ML model. More details can be found [here](/website/www/site/content/en/documentation/ml/side-input-updates.md).
+
+The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
Review Comment:
```suggestion
The side input `PCollection` must follow the [`AsSingleton`](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view to avoid errors.
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
Review Comment:
```suggestion
* `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective `ModelHandlers` can load the models without errors.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
Review Comment:
```suggestion
## Models for image segmentation
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
Review Comment:
```suggestion
based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
Review Comment:
```suggestion
The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
Review Comment:
```suggestion
For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
```
##########
website/www/site/layouts/partials/section-menu/en/documentation.html:
##########
@@ -226,6 +226,7 @@
<li><a href="/documentation/ml/tensorrt-runinference">Build a custom model handler with TensorRT</a></li>
<li><a href="/documentation/ml/large-language-modeling">Use LLM inference</a></li>
<li><a href="/documentation/ml/multi-language-inference/">Build a multi-language inference pipeline</a></li>
+ <li><a href="/documentation/ml/side-input-updates/">Auto Model Updates in RunInference Transforms using SideInputs</a></li>
Review Comment:
```suggestion
<li><a href="/documentation/ml/side-input-updates/">Use side inputs to automatically update models</a></li>
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` loads a model using weights and a model class initially, and during the update, If you pass in weights from a different model class when you update the model via side inputs, it won’t load properly because it’s expecting the weights from the original model class.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+Use cases:
+ * Use `WatchFilePattern` as side input to the RunInference PTransform for the auto updates of the ML model. More details can be found [here](/website/www/site/content/en/documentation/ml/side-input-updates.md).
+
+The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
+
+**Note**: If the main PCollection emits inputs and a side input has yet to receive inputs, the main PCollection is buffered until there is
Review Comment:
```suggestion
**Note**: If the main `PCollection` emits inputs and a side input has yet to receive inputs, the main `PCollection` is buffered until there is
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` loads a model using weights and a model class initially, and during the update, If you pass in weights from a different model class when you update the model via side inputs, it won’t load properly because it’s expecting the weights from the original model class.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+Use cases:
+ * Use `WatchFilePattern` as side input to the RunInference PTransform for the auto updates of the ML model. More details can be found [here](/website/www/site/content/en/documentation/ml/side-input-updates.md).
+
+The side input PCollection must follow [AsSingleton](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view or the pipeline will result in error.
+
+**Note**: If the main PCollection emits inputs and a side input has yet to receive inputs, the main PCollection is buffered until there is
+ an update to the side input. This could happen with Global windowed side inputs with data driven triggers such as `AfterCount`, `AfterProcessingTime`. Until the side input is updated, emit the default or initial model ID that is used to pass the respective `ModelHandler` as a side input.
Review Comment:
```suggestion
an update to the side input. This could happen with global windowed side inputs with data driven triggers, such as `AfterCount`, `AfterProcessingTime`. Until the side input is updated, emit the default or initial model ID that is used to pass the respective `ModelHandler` as a side input.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
Review Comment:
```suggestion
The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Since the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+Once the pipeline starts processing data and when you see some outputs emitted from the `RunInference` PTransform, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as side input.
Review Comment:
```suggestion
After the pipeline starts processing data and when you see some outputs emitted from the RunInference `PTransform`, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as a side input.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Since the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
Review Comment:
```suggestion
`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [`AsSingleton`](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Since the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+Once the pipeline starts processing data and when you see some outputs emitted from the `RunInference` PTransform, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as side input.
+
+**Note**: Side input update frequency is non-deterministic and can have longer interval between updates.
+
+```python
+import apache_beam as beam
+from apache_beam.ml.inference.utils import WatchFilePattern
+from apache_beam.ml.inference.base import RunInference
+with beam.Pipeline() as pipeline:
+
+ file_pattern = 'gs://<your-bucket>/*.h5'
+ pubsub_topic = '<topic_emitting_image_names>'
+
+ side_input_pcoll = (
+ pipeline
+ | "FilePatternUpdates" >> WatchFilePattern(file_pattern=file_pattern))
+
+ images_pcoll = (
+ pipeline
+ | "ReadFromPubSub" >> beam.io.ReadFromPubSub(topic=pubsub_topic)
+ | "DecodeBytes" >> beam.Map(lambda x: x.decode('utf-8'))
+ | "PreProcessImage" >> beam.Map(read_image)
+ )
+
+ inference_pcoll = (
+ images_pcoll
+ | "RunInference" >> RunInference(
+ model_handler=tf_model_handler,
+ model_metadata_pcoll=side_input_pcoll))
+
+```
+
+
+### Post-process `PredictionResult` object
+
+Once the inference is done, RunInference outputs `PredictionResult` object that contains `example`, `inference` and, `model_id`. Here, the `model_id` is used to identify which model is used for running the inference.
+
+```python
+from apache_beam.ml.inference.base import PredictionResult
+
+class PostProcessor(beam.DoFn):
+ """
+ Process the PredictionResult to get the predicted label and model id used for inference.
+ """
+ def process(self, element: PredictionResult) -> typing.Iterable[str]:
+ predicted_class = numpy.argmax(element.inference[0], axis=-1)
+ labels_path = tf.keras.utils.get_file(
+ 'ImageNetLabels.txt',
+ 'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt'
+ )
+ imagenet_labels = numpy.array(open(labels_path).read().splitlines())
+ predicted_class_name = imagenet_labels[predicted_class]
+ return predicted_class_name.title(), element.model_id
+
+post_processor_pcoll = (inference_pcoll | "PostProcessor" >> PostProcessor())
+```
+
+## Run the pipeline
+```python
+result = pipeline.run().wait_until_finish()
+```
+**Note**: `model_name` of the `ModelMetaData` object will be attached as prefix to the [metrics](https://beam.apache.org/documentation/ml/runinference-metrics/) calculated by the RunInference PTransform.
Review Comment:
```suggestion
**Note**: The `model_name` of the `ModelMetaData` object will be attached as prefix to the [metrics](https://beam.apache.org/documentation/ml/runinference-metrics/) calculated by the RunInference `PTransform`.
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
Review Comment:
```suggestion
To perform automatic updates of the models used with the RunInference `PTransform` without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input `PCollection` to the RunInference input parameter `model_metadata_pcoll`.
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
Review Comment:
```suggestion
## Slowly-updating side input pattern to auto-update models used in RunInference
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern as side input to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to run inferences on images using TensorFlow models.
+
+Using side inputs, you can update your model (which is passed in the `ModelHandler`) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input PCollection that defines the logic for the model update.
+
+**More about `side inputs` can be found at https://beam.apache.org/documentation/programming-guide/#side-inputs.**
+
+This example uses [WatchFilePattern](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to auto update the ML model without stopping the Beam pipeline.
+
+### Setting up source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [model_handler](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection[ModelMetadata]` compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Since the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
+
+
+Once the pipeline starts processing data and when you see some outputs emitted from the `RunInference` PTransform, upload a `.h5` `TensorFlow` model that matches the `file_pattern` to the Google Cloud Storage bucket. RunInference will update the `model_uri` of `TFModelHandlerTensor` using `WatchFilePattern` as side input.
+
+**Note**: Side input update frequency is non-deterministic and can have longer interval between updates.
+
+```python
+import apache_beam as beam
+from apache_beam.ml.inference.utils import WatchFilePattern
+from apache_beam.ml.inference.base import RunInference
+with beam.Pipeline() as pipeline:
+
+ file_pattern = 'gs://<your-bucket>/*.h5'
+ pubsub_topic = '<topic_emitting_image_names>'
+
+ side_input_pcoll = (
+ pipeline
+ | "FilePatternUpdates" >> WatchFilePattern(file_pattern=file_pattern))
+
+ images_pcoll = (
+ pipeline
+ | "ReadFromPubSub" >> beam.io.ReadFromPubSub(topic=pubsub_topic)
+ | "DecodeBytes" >> beam.Map(lambda x: x.decode('utf-8'))
+ | "PreProcessImage" >> beam.Map(read_image)
+ )
+
+ inference_pcoll = (
+ images_pcoll
+ | "RunInference" >> RunInference(
+ model_handler=tf_model_handler,
+ model_metadata_pcoll=side_input_pcoll))
+
+```
+
+
+### Post-process `PredictionResult` object
Review Comment:
```suggestion
## Post-process the `PredictionResult` object
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` loads a model using weights and a model class initially, and during the update, If you pass in weights from a different model class when you update the model via side inputs, it won’t load properly because it’s expecting the weights from the original model class.
Review Comment:
```suggestion
**For example**, `PyTorchModelHandler` initially loads a model using weights and a model class. If you pass in weights from a different model class when you update the model using side inputs, the model doesn't load properly, because it expects the weights from the original model class.
```
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto update models used in RunInference
+To perform auto updates of the models used in `RunInference` PTransform without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input PCollection to the input parameter `model_metadata_pcoll` of RunInference.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective ModelHandlers can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` loads a model using weights and a model class initially, and during the update, If you pass in weights from a different model class when you update the model via side inputs, it won’t load properly because it’s expecting the weights from the original model class.
+ * `model_name`: Human-readable name for the model. This can be used to identify the model in the metrics generated by the RunInference transform.
+
+Use cases:
+ * Use `WatchFilePattern` as side input to the RunInference PTransform for the auto updates of the ML model. More details can be found [here](/website/www/site/content/en/documentation/ml/side-input-updates.md).
Review Comment:
```suggestion
* Use `WatchFilePattern` as side input to the RunInference `PTransform` to automatically update the ML model. For more information, see [Use `WatchFilePattern` as side input to auto-update ML models in RunInference](https://beam.apache.org/documentation/ml/side-input-updates).
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1153591669
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` to run inference on images using TensorFlow models. It uses a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to update the model.
+
+Using side inputs, you can update your model (which is passed in a `ModelHandler` configuration object) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
+
+## Models for image segmentation
+
+For the purpose of this example, use TensorFlow models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format.
+
+
+## Pre-process images for inference
+The Pub/Sub topic emits an image path. We need to read and preprocess the image to use it for RunInference. The `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Pipeline steps**:
+1. Get the image names from the Pub/Sub topic.
+2. Read and pre-process the images using the `read_image` function.
+3. Pass the images to the RunInference `PTransform`. RunInference takes `model_handler` and `model_metadata_pcoll` as input parameters.
+
+For the [`model_handler`](https://github.com/apache/beam/blob/07f52a478174f8733c7efedb7189955142faa5fa/sdks/python/apache_beam/ml/inference/base.py#L308), we use [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+# initialize TFModelHandlerTensor with a .h5 model saved in a directory accessible by the pipeline.
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` to the RunInference `PTransform`. This side input is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+We will use `WatchFilePattern` as side input to watch a glob pattern matching `.h5` files.
+
+`model_metadata_pcoll` expects a `PCollection` of ModelMetadata compatible with [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) view. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
Review Comment:
```suggestion
`model_metadata_pcoll` expects a `PCollection` of ModelMetadata compatible with the [AsSingleton](https://beam.apache.org/releases/pydoc/2.4.0/apache_beam.pvalue.html#apache_beam.pvalue.AsSingleton) class. Because the pipeline uses `WatchFilePattern` as side input, it will take care of windowing and wrapping the output into `ModelMetadata`.
```
I'm not sure if this update is accurate, so please verify. I just find "compatible with AsSingleton view" confusing, so I'm trying to make what we mean here clearer.
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto-update models used in RunInference
+To perform automatic updates of the models used with the RunInference `PTransform` without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input `PCollection` to the RunInference input parameter `model_metadata_pcoll`.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective `ModelHandlers` can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` initially loads a model using weights and a model class. If you pass in weights from a different model class when you update the model using side inputs, the model doesn't load properly, because it expects the weights from the original model class.
+ * `model_name`: Human-readable name for the model. You can use this name to identify the model in the metrics generated by the RunInference transform.
+
+Use cases:
+ * Use `WatchFilePattern` as side input to the RunInference `PTransform` to automatically update the ML model. For more information, see [Use `WatchFilePattern` as side input to auto-update ML models in RunInference](https://beam.apache.org/documentation/ml/side-input-updates).
+
+The side input `PCollection` must follow the [`AsSingleton`](https://beam.apache.org/releases/pydoc/current/apache_beam.pvalue.html?highlight=assingleton#apache_beam.pvalue.AsSingleton) view to avoid errors.
+
+**Note**: If the main `PCollection` emits inputs and a side input has yet to receive inputs, the main `PCollection` is buffered until there is
+ an update to the side input. This could happen with global windowed side inputs with data driven triggers, such as `AfterCount`, `AfterProcessingTime`. Until the side input is updated, emit the default or initial model ID that is used to pass the respective `ModelHandler` as a side input.
Review Comment:
```suggestion
an update to the side input. This could happen with global windowed side inputs with data driven triggers, such as `AfterCount` and `AfterProcessingTime`. Until the side input is updated, emit the default or initial model ID that is used to pass the respective `ModelHandler` as a side input.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Update ML models using WatchFilePattern"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use WatchFilePattern to auto-update ML models in RunInference
+
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` to run inference on images using TensorFlow models. It uses a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) `PCollection` that emits `ModelMetadata` to update the model.
+
+Using side inputs, you can update your model (which is passed in a `ModelHandler` configuration object) in real-time, even while the Beam pipeline is still running. This can be done either by leveraging one of Beam's provided patterns, such as the `WatchFilePattern`,
+or by configuring a custom side input `PCollection` that defines the logic for the model update.
+
+For more information about side inputs, see the [Side inputs](https://beam.apache.org/documentation/programming-guide/#side-inputs) section in the Apache Beam Programming Guide.
+
+This example uses [`WatchFilePattern`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.utils.html#apache_beam.ml.inference.utils.WatchFilePattern) as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [`ModelMetadata`](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to automatically update the ML model without stopping the Beam pipeline.
+
+## Set up the source
+
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
Review Comment:
I don't think this needs to be a bullet point. These two sentences could just be one paragraph.
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,23 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly-updating side input pattern to auto-update models used in RunInference
+To perform automatic updates of the models used with the RunInference `PTransform` without stopping the Beam pipeline, pass a [`ModelMetadata`](https://beam.apache.org/releases/pydoc/current/apache_beam.ml.inference.base.html#apache_beam.ml.inference.base.ModelMetadata) side input `PCollection` to the RunInference input parameter `model_metadata_pcoll`.
+
+`ModelMetdata` is a `NamedTuple` containing:
+ * `model_id`: Unique identifier for the model. This can be a file path or a URL where the model can be accessed. It is used to load the model for inference. The URL or file path must be in the compatible format so that the respective `ModelHandlers` can load the models without errors.
+
+ **For example**, `PyTorchModelHandler` initially loads a model using weights and a model class. If you pass in weights from a different model class when you update the model using side inputs, the model doesn't load properly, because it expects the weights from the original model class.
Review Comment:
```suggestion
For example, `PyTorchModelHandler` initially loads a model using weights and a model class. If you pass in weights from a different model class when you update the model using side inputs, the model doesn't load properly, because it expects the weights from the original model class.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1148026943
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,138 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
Review Comment:
@riteshghorse Would you mind taking a look since this is a new feature and fresh set of eyes could reveal confusing content?
##########
website/www/site/content/en/documentation/sdks/python-machine-learning.md:
##########
@@ -243,6 +243,17 @@ For more information, see the [`PredictionResult` documentation](https://github.
For detailed instructions explaining how to build and run a Python pipeline that uses ML models, see the
[Example RunInference API pipelines](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference) on GitHub.
+## Slowly updating side input pattern to update models used by ModelHandler
Review Comment:
Changed it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1149770971
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,22 +15,22 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+# Use slowly-updating side input patterns to auto-update models
-The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a side input `PCollection` that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
-In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
-`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
+This example uses `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to dynamically update the model without stopping the Beam pipeline.
-**Note**: Slowly-updating side input pattern is non-deterministic.
+**Note**: Slowly-updating side input patterns are non-deterministic.
### Setting up source
-We will use PubSub topic as a source to read the image names.
- * PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
-### Models for image segmentation
+## Models for image segmentation
For the purpose of this example, use models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format. Initially, pass a model to the Tensorflow ModelHandler for predictions until there is an update via side input.
After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the `WatchFilePattern`.
Review Comment:
GCS bucket should be Google Cloud Storage bucket.
Also, the second sentence has a grammatical error. I think it should be:
The bucket path will be used as a glob pattern and is passed to the `WatchFilePattern`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1148027333
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
Review Comment:
```suggestion
The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a side input `PCollection` that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
Review Comment:
```suggestion
the RunInference `PTransform` to dynamically update the model without stopping the Beam pipeline.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
Review Comment:
```suggestion
# Use slowly-updating side input patterns to auto-update models
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -128,5 +121,15 @@ class PostProcessor(beam.DoFn):
predicted_class_name = imagenet_labels[predicted_class]
return predicted_class_name.title(), element.model_id
-(inference_pcoll | "PostProcessor" >> PostProcessor())
+post_processor_pcoll = (inference_pcoll | "PostProcessor" >> PostProcessor())
```
+
+### Run the pipeline
+```python
+result = pipeline.run().wait_until_finish()
+```
+Once the pipeline is run with initial settings, upload a model matching the `file_pattern` to GCS bucket. After some time, you will see that your pipeline starts to use the updated model instead of the initial model.
+**Note**: `model_name` of the `ModelMetaData` object will be attached as prefix to the [metrics](https://beam.apache.org/documentation/ml/runinference-metrics/) calculated by the RunInference PTransform
Review Comment:
```suggestion
**Note**: The `model_name` of the `ModelMetaData` object is attached as prefix to the [metrics](https://beam.apache.org/documentation/ml/runinference-metrics/) calculated by the RunInference `PTransform`.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -128,5 +121,15 @@ class PostProcessor(beam.DoFn):
predicted_class_name = imagenet_labels[predicted_class]
return predicted_class_name.title(), element.model_id
-(inference_pcoll | "PostProcessor" >> PostProcessor())
+post_processor_pcoll = (inference_pcoll | "PostProcessor" >> PostProcessor())
```
+
+### Run the pipeline
+```python
+result = pipeline.run().wait_until_finish()
+```
+Once the pipeline is run with initial settings, upload a model matching the `file_pattern` to GCS bucket. After some time, you will see that your pipeline starts to use the updated model instead of the initial model.
+**Note**: `model_name` of the `ModelMetaData` object will be attached as prefix to the [metrics](https://beam.apache.org/documentation/ml/runinference-metrics/) calculated by the RunInference PTransform
+
+## Final remarks
+Use this example as a pattern on how to use side inputs with RunInference PTransform to auto update the models without the need to stop the pipeline. A similar example for PyTorch can be found on [GitHub](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_image_classification_with_side_inputs.py).
Review Comment:
```suggestion
Use this example as a pattern when using side inputs with the RunInference `PTransform` to auto-update the models without stopping the pipeline. You can see a similar example for PyTorch on [GitHub](https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/inference/pytorch_image_classification_with_side_inputs.py).
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
Review Comment:
```suggestion
based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
We will use PubSub topic as a source to read the image names.
Review Comment:
```suggestion
To read the image names, use a Pub/Sub topic as the source.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
Review Comment:
```suggestion
This example uses `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
Review Comment:
```suggestion
**Note**: Slowly-updating side input patterns are non-deterministic.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
Review Comment:
```suggestion
## Set up the source
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
We will use PubSub topic as a source to read the image names.
* PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
-## Models for image segmentation
+### Models for image segmentation
-We will use `resnet_v2_101` for initial predictions. After a while, we will upload a `resnet_v2_152` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the WatchFilePattern.
-
+We will use `resnet_v2_101` for initial predictions. After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the WatchFilePattern.
Review Comment:
```suggestion
For initial predictions, use `resnet_v2_101`. Upload a model that matches the `file_pattern` to the Google Cloud Storage bucket. The bucket path is used as a glob pattern and is passed to `WatchFilePattern`.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
We will use PubSub topic as a source to read the image names.
* PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
Review Comment:
```suggestion
* The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
We will use PubSub topic as a source to read the image names.
* PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
-## Models for image segmentation
+### Models for image segmentation
Review Comment:
```suggestion
## Models for image segmentation
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
We will use PubSub topic as a source to read the image names.
* PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
-## Models for image segmentation
+### Models for image segmentation
-We will use `resnet_v2_101` for initial predictions. After a while, we will upload a `resnet_v2_152` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the WatchFilePattern.
-
+We will use `resnet_v2_101` for initial predictions. After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the WatchFilePattern.
+Once there is an update, the RunInference PTransform will update the `model_uri` to use the latest model/file.
Review Comment:
```suggestion
After the update, the RunInference `PTransform` updates the `model_uri` to use the latest model and file.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
-You can find the code used in this example in the [Beam repository] (link).
-
-## Setting up source.
+### Setting up source.
We will use PubSub topic as a source to read the image names.
* PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
-## Models for image segmentation
+### Models for image segmentation
-We will use `resnet_v2_101` for initial predictions. After a while, we will upload a `resnet_v2_152` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the WatchFilePattern.
-
+We will use `resnet_v2_101` for initial predictions. After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the WatchFilePattern.
+Once there is an update, the RunInference PTransform will update the `model_uri` to use the latest model/file.
### ModelHandler used for Predictions.
Review Comment:
```suggestion
## ModelHandler for predictions
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -128,5 +121,15 @@ class PostProcessor(beam.DoFn):
predicted_class_name = imagenet_labels[predicted_class]
return predicted_class_name.title(), element.model_id
-(inference_pcoll | "PostProcessor" >> PostProcessor())
+post_processor_pcoll = (inference_pcoll | "PostProcessor" >> PostProcessor())
```
+
+### Run the pipeline
+```python
+result = pipeline.run().wait_until_finish()
+```
+Once the pipeline is run with initial settings, upload a model matching the `file_pattern` to GCS bucket. After some time, you will see that your pipeline starts to use the updated model instead of the initial model.
Review Comment:
```suggestion
After you run the pipeline with the initial settings, upload a model matching the `file_pattern` to the Google Cloud Storage bucket. Your pipeline will use the updated model instead of the initial model.
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -128,5 +121,15 @@ class PostProcessor(beam.DoFn):
predicted_class_name = imagenet_labels[predicted_class]
return predicted_class_name.title(), element.model_id
-(inference_pcoll | "PostProcessor" >> PostProcessor())
+post_processor_pcoll = (inference_pcoll | "PostProcessor" >> PostProcessor())
```
+
+### Run the pipeline
Review Comment:
```suggestion
## Run the pipeline
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm merged pull request #25947: Add documentation for the auto model updates
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #25947:
URL: https://github.com/apache/beam/pull/25947
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1148040679
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,138 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
Review Comment:
I'd add a couple sentences along the lines of "Side inputs can be used to do live updates of your model while your pipeline is still running. You can either use one of Beam's built in side inputs or configure a custom one to define how and when you'd like to update."
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,138 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+
+In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
+
+**Note**: Slowly-updating side input pattern is non-deterministic.
+
+### Setting up source
+
+We will use PubSub topic as a source to read the image names.
+ * PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format. Initially, pass a model to the Tensorflow ModelHandler for predictions until there is an update via side input.
+After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the `WatchFilePattern`.
Review Comment:
I'd phrase these as "First we will do X. Then we will do Y which will have Z effect." This currently reads as if you're instructing the user to do this now, but you have more detailed instructions below.
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,138 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+
+In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
+
+**Note**: Slowly-updating side input pattern is non-deterministic.
+
+### Setting up source
+
+We will use PubSub topic as a source to read the image names.
+ * PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format. Initially, pass a model to the Tensorflow ModelHandler for predictions until there is an update via side input.
+After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the `WatchFilePattern`.
+Once there is an update, the RunInference PTransform will update the `model_uri` to use the latest model for inferences.
+
+### ModelHandler used for inference
+
+For the ModelHandler, we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll`.
+ 1. For the `model_handler`, `TFModelHandlerTensor` is used.
+ 2. The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+ 1. The `WatchFilePattern` is used as side input, which is used to watch a glob pattern matching `.h5` files. We use [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) standard to load the models.
Review Comment:
I would rewrite these as sentences after the steps rather than substeps. So something like:
```
1. Get the image names from the PubSub topic.
2. Read and pre-process the images using `read_image` function.
3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll`.
We will use `TFModelHandlerTensor` for our `model_handler`, ...
```
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,138 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+
+In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
+
+**Note**: Slowly-updating side input pattern is non-deterministic.
+
+### Setting up source
+
+We will use PubSub topic as a source to read the image names.
+ * PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
+
+### Models for image segmentation
+
+For the purpose of this example, use models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format. Initially, pass a model to the Tensorflow ModelHandler for predictions until there is an update via side input.
+After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the `WatchFilePattern`.
+Once there is an update, the RunInference PTransform will update the `model_uri` to use the latest model for inferences.
+
+### ModelHandler used for inference
+
+For the ModelHandler, we will be using [TFModelHandlerTensor](https://github.com/apache/beam/blob/186973b110d82838fb8e5ba27f0225a67c336591/sdks/python/apache_beam/ml/inference/tensorflow_inference.py#L184).
+```python
+from apache_beam.ml.inference.tensorflow_inference import TFModelHandlerTensor
+tf_model_handler = TFModelHandlerTensor(model_uri='gs://<your-bucket>/<model_path.h5>')
+```
+
+### Pre-processing image for inference
+The PubSub topic emits an image path. We need to read and preprocess the image to use it for RunInference. `read_image` function is used to read the image for inference.
+
+```python
+import io
+from PIL import Image
+from apache_beam.io.filesystems import FileSystems
+import numpy
+import tensorflow as tf
+
+def read_image(image_file_name):
+ with FileSystems().open(image_file_name, 'r') as file:
+ data = Image.open(io.BytesIO(file.read())).convert('RGB')
+ img = data.resize((224, 224))
+ img = numpy.array(img) / 255.0
+ img_tensor = tf.cast(tf.convert_to_tensor(img[...]), dtype=tf.float32)
+ return img_tensor
+```
+
+Now, let's jump into the pipeline code.
+
+**Steps**:
+1. Get the image names from the PubSub topic.
+2. Read and pre-process the images using `read_image` function.
+3. Pass the images to the `RunInference` PTransform. RunInference takes `model_handler` and `model_metadata_pcoll`.
+ 1. For the `model_handler`, `TFModelHandlerTensor` is used.
+ 2. The `model_metadata_pcoll` is a [side input](https://beam.apache.org/documentation/programming-guide/#side-inputs) PCollection to the RunInference PTransform. This is used to update the models in the `model_handler` without needing to stop the beam pipeline.
+ 1. The `WatchFilePattern` is used as side input, which is used to watch a glob pattern matching `.h5` files. We use [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) standard to load the models.
Review Comment:
Also, generally be careful about jumping back and forth between first person active verbs and passive verbs (e.g. `We use` vs `This is used`) - you should pick one (ideally active verbs) and stick with it.
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,138 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
Review Comment:
It also might be worth defining side inputs actually.
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,28 +15,25 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Update Models in RunInference Transform
+# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
-The pipeline in this example uses RunInference PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest `ModelMetadata`, which is used in
-`RunInference` PTransform for the dynamic model updates without the need for stopping
-the beam pipeline.
+based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
**Note**: Slowly-updating side input pattern is non-deterministic.
Review Comment:
I'd also add more info here on the possible behavior gaps you can observe (aka updates may not happen immediately)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on PR #25947:
URL: https://github.com/apache/beam/pull/25947#issuecomment-1490571745
@rszper can you take a look and provide LGTM if this looks good? thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1149792086
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -15,22 +15,22 @@ See the License for the specific language governing permissions and
limitations under the License.
-->
-# Use Slowly-Updating Side Input Pattern to Auto Update Models in RunInference Transform
+# Use slowly-updating side input patterns to auto-update models
-The pipeline in this example uses [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) PTransform with a `side input` PCollection that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
+The pipeline in this example uses a [RunInference](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/) `PTransform` with a side input `PCollection` that emits `ModelMetadata` to run inferences on images using open source Tensorflow models trained on `imagenet`.
-In this example, we will use `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
-based on timestamps and emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
-`RunInference` PTransform for the dynamic auto model updates without the need for stopping the beam pipeline.
+This example uses `WatchFilePattern` as a side input. `WatchFilePattern` is used to watch for the file updates matching the `file_pattern`
+based on timestamps. It emits the latest [ModelMetadata](https://beam.apache.org/documentation/transforms/python/elementwise/runinference/), which is used in
+the RunInference `PTransform` to dynamically update the model without stopping the Beam pipeline.
-**Note**: Slowly-updating side input pattern is non-deterministic.
+**Note**: Slowly-updating side input patterns are non-deterministic.
### Setting up source
-We will use PubSub topic as a source to read the image names.
- * PubSub topic emits a `UTF-8` encoded model path that will be used read and preprocess images for running the inference.
+To read the image names, use a Pub/Sub topic as the source.
+ * The Pub/Sub topic emits a `UTF-8` encoded model path that is used to read and preprocess images to run the inference.
-### Models for image segmentation
+## Models for image segmentation
For the purpose of this example, use models saved in [HDF5](https://www.tensorflow.org/tutorials/keras/save_and_load#hdf5_format) format. Initially, pass a model to the Tensorflow ModelHandler for predictions until there is an update via side input.
After a while, upload a model that matches the `file_pattern` to the GCS bucket. The bucket path will be used a glob pattern and is passed to the `WatchFilePattern`.
Review Comment:
Thanks. I changed it already. Working on a final draft now. I will ping you for another review soon. Thanks!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1151150113
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
Review Comment:
I think the H1 is fine as-is. It's just too long to use in this field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on a diff in pull request #25947: Add documentation for the auto model updates
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on code in PR #25947:
URL: https://github.com/apache/beam/pull/25947#discussion_r1151149187
##########
website/www/site/content/en/documentation/ml/side-input-updates.md:
##########
@@ -0,0 +1,144 @@
+---
+title: "Auto Model Updates in RunInference Transforms using SideInputs"
Review Comment:
should I change the H1 title as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org