You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2023/01/19 15:18:23 UTC
[GitHub] [beam] andres-vv opened a new pull request, #25083: Multi language/runinference example
andres-vv opened a new pull request, #25083:
URL: https://github.com/apache/beam/pull/25083
**Please** add a meaningful description for your change here
This PR adds an example that demonstrates a cross language Java pipeline that uses RunInference via a PythonExternalTransform.
An open source model (bert-base-uncased) is used to predict the last word of sentences.
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/get-started-contributing/#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1129762189
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("transformers==4.26.0");
+ local_packages.add("torch==1.13.1");
+ local_packages.add(options.getLocalPackage());
+
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform")
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ .withExtraPackages(local_packages)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+ p.run().waitUntilFinish();
+```
+
+The `PythonExternalTransform` is used to inject the cross-language Python transform into the Java pipeline. `PythonExternalTransform` takes a string parameter which is the fully qualified name of the Python transform.
+
+The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section. Finally the `withExtraPackages` method is used to specify the additional python packages that are needed for the Python transform. In this example the `local_packages` list is used, which contains the path to the compiled Python package.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
+1. Local is fine and should be a better example since most users would not want to push pipeline specific packages to PyPi.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1085811255
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,89 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Port number of the expansion service.")
+ @Required
+ String getPort();
+
+ void setPort(String value);
+ }
+
+ public static void main(String[] args) {
+
+ MultiLanguageOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
+ .as(MultiLanguageOptions.class);
+
+ Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ /* For 2.44.0 and on
+ List<String> local_packages=new ArrayList<String>();
Review Comment:
Yes, that should work for 2.44.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1085485321
##########
website/www/site/layouts/partials/section-menu/en/documentation.html:
##########
@@ -1,13 +1,13 @@
{{/*
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License. See accompanying LICENSE file.
Review Comment:
Could you please undo these indentation changes (they look accidental)?
##########
website/www/site/content/en/documentation/ml/overview.md:
##########
@@ -90,4 +90,5 @@ You can find examples of end-to-end AI/ML pipelines for several use cases:
* [Multi model pipelines in Beam](/documentation/ml/multi-model-pipelines): Explains how multi-model pipelines work and gives an overview of what you need to know to build one using the RunInference API.
* [Online Clustering in Beam](/documentation/ml/online-clustering): Demonstrates how to set up a real-time clustering pipeline that can read text from Pub/Sub, convert the text into an embedding using a transformer-based language model with the RunInference API, and cluster the text using BIRCH with stateful processing.
* [Anomaly Detection in Beam](/documentation/ml/anomaly-detection): Demonstrates how to set up an anomaly detection pipeline that reads text from Pub/Sub in real time and then detects anomalies using a trained HDBSCAN clustering model with the RunInference API.
-* [Large Language Model Inference in Beam](/documentation/ml/large-language-modeling): Demonstrates a pipeline that uses RunInference to perform translation with the T5 language model which contains 11 billion parameters.
\ No newline at end of file
+* [Large Language Model Inference in Beam](/documentation/ml/large-language-modeling): Demonstrates a pipeline that uses RunInference to perform translation with the T5 language model which contains 11 billion parameters.
+* [Cross-language RunInference in Beam](/documentation/ml/multi-language-inference): Demonstrates a cross-language java pipeline that uses custom python transform to do preprocessing, postprocessing, and inference.
Review Comment:
```suggestion
* [Cross-language RunInference in Beam](/documentation/ml/multi-language-inference): Demonstrates a cross-language java pipeline that uses a custom python transform to do preprocessing, postprocessing, and inference.
```
##########
website/www/site/layouts/partials/section-menu/en/documentation.html:
##########
@@ -1,13 +1,13 @@
{{/*
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License. See accompanying LICENSE file.
Review Comment:
Actually, could you just remove the non-content changes to this page? Looks like you may have had an autoformatter change things
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
damccorm commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1398513181
Also, for context for anyone coming across this PR, this is a continuation of https://github.com/apache/beam/pull/24656
The website changes are rendered here - http://apache-beam-website-pull-requests.storage.googleapis.com/25083/documentation/ml/multi-language-inference/index.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1085658344
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md:
##########
@@ -0,0 +1,60 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+## Setting up the Expansion service
+*Note: skip this step for Beam 2.44 and later.*
+
+Because we can not add local packages in Beam 2.43 we must create our own expansion service.
+Start up the expansion service with this command:
+
+```bash
+export PORT = <port to host expansion service>
+export IMAGE = <custom docker image>
+
+python -m multi_language_custom_transform.start_expansion_service \
+ --port=$PORT \
+ --fully_qualified_name_glob="*" \
+ --environment_config=$IMAGE \
+ --environment_type=DOCKER
+```
+## Running the Java pipeline
+In another terminal, run the following command to start the Java pipeline:
+
+```bash
+export JOB_SERVER_PORT= <port to host expansion service>
+export JAVA_HOME= <path to java HOME>
+export GCP_PROJECT= <your gcp project>
+export GCP_BUCKET= <your gcp bucker>
+export GCP_REGION= <region of bucket>
+
+cd multi-language-beam
Review Comment:
There should be instructions to download the Maven arche-type (or setup the Maven project) before this. For example,
https://github.com/apache/beam/tree/master/examples/multi-language#instructions-for-running-the-java-pipeline-on-released-beam-beam-2430-and-later
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,89 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Port number of the expansion service.")
+ @Required
+ String getPort();
+
+ void setPort(String value);
+ }
+
+ public static void main(String[] args) {
+
+ MultiLanguageOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
+ .as(MultiLanguageOptions.class);
+
+ Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ /* For 2.44.0 and on
+ List<String> local_packages=new ArrayList<String>();
Review Comment:
2.44.0 is out now. But I'm not sure if you even need this since Python modules mentioned here are released with Beam. `withExtraPackages` is only needed when installing custom packages that are not available in Beam **and** not using a custom expansion service.
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,89 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Port number of the expansion service.")
+ @Required
+ String getPort();
+
+ void setPort(String value);
+ }
+
+ public static void main(String[] args) {
+
+ MultiLanguageOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
+ .as(MultiLanguageOptions.class);
+
+ Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ /* For 2.44.0 and on
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("multi_language_custom_transform");
Review Comment:
FYI, `withExtraPackages` takes Python packages (PyPi or local tarball) that can be staged by Beam: https://beam.apache.org/documentation/sdks/python-pipeline-dependencies/#local-or-nonpypi
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,89 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Port number of the expansion service.")
+ @Required
+ String getPort();
+
+ void setPort(String value);
+ }
+
+ public static void main(String[] args) {
+
+ MultiLanguageOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
+ .as(MultiLanguageOptions.class);
+
+ Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ /* For 2.44.0 and on
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("multi_language_custom_transform");
+ */
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform", "localhost:" + options.getPort())
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ // .withExtraPackages(multi_language_custom_transform)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+
+ p.run().waitUntilFinish();
Review Comment:
+1 but this might not even be needed in the future if Python modules here are released with Beam.
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/multi_language_custom_transform/start_expansion_service.py:
##########
@@ -0,0 +1,25 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+import logging
+import sys
+
+from apache_beam.runners.portability import expansion_service_main
+
+if __name__ == '__main__':
+ # Start the expansion service.
+ logging.getLogger().setLevel(logging.INFO)
+ expansion_service_main.main(sys.argv)
Review Comment:
Can we just run "expansion_service_main" directly and remove this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1140124433
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md:
##########
@@ -0,0 +1,69 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+For a detailed explanation of this inference example, visit the [documentation](https://beam.apache.org/documentation/ml/multi-language-inference/).
+## Set up Python virtual environment
+Make sure to set up a virtual environment for Python with all the required dependencies.
+More details on how to do this can be found [here](https://beam.apache.org/get-started/quickstart-py/#set-up-your-environment).
+## Running the Java pipeline
+Make sure you have Maven installed and added to PATH. Also make sure that JAVA_HOME
+points to the correct Java version.
+
+First we need to download the Maven arche-type for Beam. Run the following command:
Review Comment:
```suggestion
First we need to download the Maven archetype for Beam. Run the following command:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1082651068
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,89 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Port number of the expansion service.")
+ @Required
+ String getPort();
+
+ void setPort(String value);
+ }
+
+ public static void main(String[] args) {
+
+ MultiLanguageOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
+ .as(MultiLanguageOptions.class);
+
+ Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ /* For 2.44.0 and on
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("multi_language_custom_transform");
+ */
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform", "localhost:" + options.getPort())
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ // .withExtraPackages(multi_language_custom_transform)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+
+ p.run().waitUntilFinish();
Review Comment:
Now that 2.44's release candidate has been accepted (and it should be shortly published), can we default to the 2.44.0 implementation?
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,161 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
Review Comment:
Could we add a sentence here about running inference with the built in x-lang RunInference transform? Something along the lines of:
`To run inference from Java, you can use the [cross-language RunInference transform](https://github.com/apache/beam/blob/master/sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/transforms/RunInference.java). The [Java Sklearn Mnist Classification example](https://github.com/apache/beam/tree/master/examples/multi-language) demonstrates how you can use this transform if you don't have pre/post-processing that must be done in Python. In this example, in addition to running inference, we also want to perform preprocessing and postprocessing on the data in Python. ....`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1397696321
R: @chamikaramj @damccorm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] andres-vv commented on a diff in pull request #25083: Multi language/runinference example
Posted by "andres-vv (via GitHub)" <gi...@apache.org>.
andres-vv commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1129427540
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("transformers==4.26.0");
+ local_packages.add("torch==1.13.1");
+ local_packages.add(options.getLocalPackage());
+
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform")
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ .withExtraPackages(local_packages)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+ p.run().waitUntilFinish();
+```
+
+The `PythonExternalTransform` is used to inject the cross-language Python transform into the Java pipeline. `PythonExternalTransform` takes a string parameter which is the fully qualified name of the Python transform.
+
+The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section. Finally the `withExtraPackages` method is used to specify the additional python packages that are needed for the Python transform. In this example the `local_packages` list is used, which contains the path to the compiled Python package.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
--localPackage is the path of the local package that is compiled as a tarball. This package contains the custom python transform.
What option is prefered here?
- Publish the package on Pypi? (only purpose it will serve is for this example, since it is custom code)
- Try to store the tarball of the package on GCS and see if the Java Pipeline can access it
- Store the tarball locally
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm merged pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm merged PR #25083:
URL: https://github.com/apache/beam/pull/25083
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1472927871
Please fix failing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1472915011
Run PythonLint PreCommit
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1140134813
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md:
##########
@@ -0,0 +1,69 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+For a detailed explanation of this inference example, visit the [documentation](https://beam.apache.org/documentation/ml/multi-language-inference/).
+## Set up Python virtual environment
+Make sure to set up a virtual environment for Python with all the required dependencies.
+More details on how to do this can be found [here](https://beam.apache.org/get-started/quickstart-py/#set-up-your-environment).
+## Running the Java pipeline
+Make sure you have Maven installed and added to PATH. Also make sure that JAVA_HOME
+points to the correct Java version.
+
+First we need to download the Maven arche-type for Beam. Run the following command:
Review Comment:
I'm going to just commit this since its the last thing blocking merge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] codecov[bot] commented on pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1397171365
# [Codecov](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#25083](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (be47229) into [master](https://codecov.io/gh/apache/beam/commit/1c4e4241c2a4571c7d6c9f293133b114a9e00844?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (1c4e424) will **decrease** coverage by `0.01%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #25083 +/- ##
==========================================
- Coverage 73.12% 73.12% -0.01%
==========================================
Files 735 735
Lines 98153 98153
==========================================
- Hits 71776 71774 -2
- Misses 25013 25015 +2
Partials 1364 1364
```
| Flag | Coverage Δ | |
|---|---|---|
| python | `82.67% <ø> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...python/apache\_beam/runners/worker/worker\_status.py](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvd29ya2VyX3N0YXR1cy5weQ==) | `74.66% <0.00%> (-0.67%)` | :arrow_down: |
| [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `89.08% <0.00%> (-0.17%)` | :arrow_down: |
| [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `96.24% <0.00%> (-0.16%)` | :arrow_down: |
| [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.54% <0.00%> (ø)` | |
| [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/25083?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.71% <0.00%> (+0.12%)` | :arrow_up: |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] andres-vv commented on a diff in pull request #25083: Multi language/runinference example
Posted by "andres-vv (via GitHub)" <gi...@apache.org>.
andres-vv commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1085801971
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,89 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Port number of the expansion service.")
+ @Required
+ String getPort();
+
+ void setPort(String value);
+ }
+
+ public static void main(String[] args) {
+
+ MultiLanguageOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
+ .as(MultiLanguageOptions.class);
+
+ Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ /* For 2.44.0 and on
+ List<String> local_packages=new ArrayList<String>();
Review Comment:
@chamikaramj the python transform I use here is a custom transform that I created.
Would this work in 2.44.0?
- wrap this custom transform in a local package
- add this package with `withExtraPackages`
- remove the step where I start a custom expansion service
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1397760357
Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] andres-vv commented on a diff in pull request #25083: Multi language/runinference example
Posted by "andres-vv (via GitHub)" <gi...@apache.org>.
andres-vv commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1126384992
##########
website/www/site/content/en/documentation/ml/overview.md:
##########
@@ -105,3 +105,4 @@ You can find examples of end-to-end AI/ML pipelines for several use cases:
* [Anomaly Detection in Beam](/documentation/ml/anomaly-detection): Demonstrates how to set up an anomaly detection pipeline that reads text from Pub/Sub in real time and then detects anomalies using a trained HDBSCAN clustering model with the RunInference API.
* [Large Language Model Inference in Beam](/documentation/ml/large-language-modeling): Demonstrates a pipeline that uses RunInference to perform translation with the T5 language model which contains 11 billion parameters.
* [Per Entity Training in Beam](/documentation/ml/per-entity-training): Demonstrates a pipeline that trains a Decision Tree Classifier per education level for predicting if the salary of a person is >= 50k.
+* [Cross-language RunInference in Beam](/documentation/ml/multi-language-inference): Demonstrates a cross-language java pipeline that uses a custom python transform to do preprocessing, postprocessing, and inference.
Review Comment:
Okay, thanks for the heads-up!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1458989275
> Can this be published anywhere (pypi?) and/or available on gcs? Does it have to be locally available on the machine
These dependencies just get added to a requirements file that get processed/staged by Python. So any method supported by Python SDK would work for extra packages (I know that PyPi and local packages work, not sure about GCS).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1126634466
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,91 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Name of local package.")
Review Comment:
Could we be more descriptive? What is the local package being referred to here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
damccorm commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1398580169
Oh, could you please add a short snippet to the overview examples as well? https://github.com/apache/beam/blob/master/website/www/site/content/en/documentation/ml/overview.md#examples
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1416243216
Sounds good. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1140135103
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md:
##########
@@ -0,0 +1,69 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+For a detailed explanation of this inference example, visit the [documentation](https://beam.apache.org/documentation/ml/multi-language-inference/).
+## Set up Python virtual environment
+Make sure to set up a virtual environment for Python with all the required dependencies.
+More details on how to do this can be found [here](https://beam.apache.org/get-started/quickstart-py/#set-up-your-environment).
+## Running the Java pipeline
+Make sure you have Maven installed and added to PATH. Also make sure that JAVA_HOME
+points to the correct Java version.
+
+First we need to download the Maven arche-type for Beam. Run the following command:
Review Comment:
I'll merge once checks pass
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] andres-vv commented on pull request #25083: Multi language/runinference example
Posted by "andres-vv (via GitHub)" <gi...@apache.org>.
andres-vv commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1415528752
>
> Hi, any updates on this ?
No not yet, I haven't had any time to implement the 2.44.0 changes. I plan on picking this up somewhere next week.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1128479519
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
Review Comment:
Can we include this as part of the example?
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("transformers==4.26.0");
+ local_packages.add("torch==1.13.1");
+ local_packages.add(options.getLocalPackage());
+
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform")
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ .withExtraPackages(local_packages)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+ p.run().waitUntilFinish();
+```
+
+The `PythonExternalTransform` is used to inject the cross-language Python transform into the Java pipeline. `PythonExternalTransform` takes a string parameter which is the fully qualified name of the Python transform.
+
+The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section. Finally the `withExtraPackages` method is used to specify the additional python packages that are needed for the Python transform. In this example the `local_packages` list is used, which contains the path to the compiled Python package.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
Can this be published anywhere (pypi?) and/or available on gcs? Does it have to be locally available on the machine
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("transformers==4.26.0");
+ local_packages.add("torch==1.13.1");
+ local_packages.add(options.getLocalPackage());
+
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform")
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ .withExtraPackages(local_packages)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+ p.run().waitUntilFinish();
+```
+
+The `PythonExternalTransform` is used to inject the cross-language Python transform into the Java pipeline. `PythonExternalTransform` takes a string parameter which is the fully qualified name of the Python transform.
+
+The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section. Finally the `withExtraPackages` method is used to specify the additional python packages that are needed for the Python transform. In this example the `local_packages` list is used, which contains the path to the compiled Python package.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
I'd assume any package that pip can find is fine?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on a diff in pull request #25083: Multi language/runinference example
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1129517591
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("transformers==4.26.0");
+ local_packages.add("torch==1.13.1");
+ local_packages.add(options.getLocalPackage());
+
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform")
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ .withExtraPackages(local_packages)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+ p.run().waitUntilFinish();
+```
+
+The `PythonExternalTransform` is used to inject the cross-language Python transform into the Java pipeline. `PythonExternalTransform` takes a string parameter which is the fully qualified name of the Python transform.
+
+The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section. Finally the `withExtraPackages` method is used to specify the additional python packages that are needed for the Python transform. In this example the `local_packages` list is used, which contains the path to the compiled Python package.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
Local is fine, but we should just clarify/document that this can be any package that can be resolved in a setup.py file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on a diff in pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1125979962
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,91 @@
+package org;
Review Comment:
Please change the package to "org.apache.beam.examples...." (any sub-package with that prefix)
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
Review Comment:
How about "Using RunInference from Java SDK" ?
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
Review Comment:
Probably we should not copy code to here since it could go out of date. Just link to the Java class in the repo ?
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Compile Python code into package
+
+The Python code needs to be compiled into a package as a tarball. This package can then be used in the Java pipeline. The following example shows how to compile the Python code into a package:
+
+```bash
+ python setup.py sdist
+ ```
+
+In order to run this, a setup.py file must be made/
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows a snippet of the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ List<String> local_packages=new ArrayList<String>();
+ local_packages.add("transformers==4.26.0");
+ local_packages.add("torch==1.13.1");
+ local_packages.add(options.getLocalPackage());
+
+ List<String> packages=new ArrayList<String>();
+ input.apply("Predict", PythonExternalTransform.<PCollection<String>, PCollection<String>>from(
+ "multi_language_custom_transform.composite_transform.InferenceTransform")
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath())
+ .withExtraPackages(local_packages)
+ )
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()));
+ p.run().waitUntilFinish();
+```
+
+The `PythonExternalTransform` is used to inject the cross-language Python transform into the Java pipeline. `PythonExternalTransform` takes a string parameter which is the fully qualified name of the Python transform.
+
+The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section. Finally the `withExtraPackages` method is used to specify the additional python packages that are needed for the Python transform. In this example the `local_packages` list is used, which contains the path to the compiled Python package.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
We should clarify that we mean "Path to local package" here.
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md:
##########
@@ -0,0 +1,67 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+## Set up Python virtual environment
+Make sure to set up a virtual environment for Python with all the required dependencies.
+More details on how to do this can be found [here](https://beam.apache.org/get-started/quickstart-py/#set-up-your-environment).
+## Running the Java pipeline
+Make sure you have Maven installed and added to PATH. Also make sure that JAVA_HOME
+points to the correct Java version.
+
+First we need to download the Maven arche-type for Beam. Run the following command:
+
+```bash
+export BEAM_VERSION=<Beam version>
+
+mvn archetype:generate \
+ -DarchetypeGroupId=org.apache.beam \
+ -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
+ -DarchetypeVersion=$BEAM_VERSION \
+ -DgroupId=org.example \
+ -DartifactId=multi-language-beam \
+ -Dversion="0.1" \
+ -Dpackage=org.apache.beam.examples \
+ -DinteractiveMode=false
+```
+This will set up all the required dependencies for the Java pipeline. Next the pipeline needs to be
+implemented. The logic of this pipeline is written in the `MultiLangRunInference.java` file. After that,
+run the following command to start the Java pipeline:
+
+```bash
+export GCP_PROJECT=<your gcp project>
+export GCP_BUCKET=<your gcp bucker>
+export GCP_REGION=<region of bucket>
+export MODEL_NAME=bert-base-uncased
+export LOCAL_PACKAGE=<path to tarball>
+
+cd last_word_prediction
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
+ -Pdataflow-runner
+```
+Make sure to run this in the `last_word_prediction` directory. This will start the Java pipeline.
Review Comment:
Let's link to this (in the Beam repo).
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,155 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
Review Comment:
How about just "Python transform" ?
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/last_word_prediction/src/main/java/org/MultiLangRunInference.java:
##########
@@ -0,0 +1,91 @@
+package org;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import java.util.ArrayList;
+import java.util.List;
+
+import org.apache.beam.sdk.Pipeline;
+import org.apache.beam.sdk.extensions.python.PythonExternalTransform;
+import org.apache.beam.sdk.io.TextIO;
+import org.apache.beam.sdk.options.Description;
+import org.apache.beam.sdk.options.PipelineOptions;
+import org.apache.beam.sdk.options.PipelineOptionsFactory;
+import org.apache.beam.sdk.options.Validation.Required;
+import org.apache.beam.sdk.values.PCollection;
+
+public class MultiLangRunInference {
+ public interface MultiLanguageOptions extends PipelineOptions {
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getInputFile();
+
+ void setInputFile(String value);
+
+ @Description("Path to a stored model.")
+ @Required
+ String getModelPath();
+
+ void setModelPath(String value);
+
+ @Description("Path to an input file that contains labels and pixels to feed into the model")
+ @Required
+ String getOutputFile();
+
+ void setOutputFile(String value);
+
+ @Description("Name of the model on HuggingFace.")
+ @Required
+ String getModelName();
+
+ void setModelName(String value);
+
+ @Description("Name of local package.")
Review Comment:
Path to local package ?
##########
sdks/python/apache_beam/examples/inference/multi_language_inference/README.md:
##########
@@ -0,0 +1,67 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+## Set up Python virtual environment
+Make sure to set up a virtual environment for Python with all the required dependencies.
+More details on how to do this can be found [here](https://beam.apache.org/get-started/quickstart-py/#set-up-your-environment).
+## Running the Java pipeline
+Make sure you have Maven installed and added to PATH. Also make sure that JAVA_HOME
+points to the correct Java version.
+
+First we need to download the Maven arche-type for Beam. Run the following command:
+
+```bash
+export BEAM_VERSION=<Beam version>
+
+mvn archetype:generate \
+ -DarchetypeGroupId=org.apache.beam \
+ -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples \
+ -DarchetypeVersion=$BEAM_VERSION \
+ -DgroupId=org.example \
+ -DartifactId=multi-language-beam \
+ -Dversion="0.1" \
+ -Dpackage=org.apache.beam.examples \
+ -DinteractiveMode=false
+```
+This will set up all the required dependencies for the Java pipeline. Next the pipeline needs to be
+implemented. The logic of this pipeline is written in the `MultiLangRunInference.java` file. After that,
+run the following command to start the Java pipeline:
+
+```bash
+export GCP_PROJECT=<your gcp project>
+export GCP_BUCKET=<your gcp bucker>
+export GCP_REGION=<region of bucket>
+export MODEL_NAME=bert-base-uncased
+export LOCAL_PACKAGE=<path to tarball>
+
+cd last_word_prediction
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner \
+ --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --localPackage=$LOCAL_PACKAGE" \
Review Comment:
We should link to "multi-language-inference.md" for instructions for building the local package before this point.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] github-actions[bot] commented on pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1397177790
Assigning reviewers. If you would like to opt out of this review, comment `assign to next reviewer`:
R: @tvalentyn for label python.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25083: Multi language/runinference example
Posted by GitBox <gi...@apache.org>.
rszper commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1082856121
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,161 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Set up the expansion service
+***Note**: In Apache Beam 2.44.0 and later versions, you can use the `withExtraPackages` method to specify the required local packages directly in the Java pipeline. This means you do not need to set up an expansion service anymore and can skip this step.*
Review Comment:
Remove "anymore" from this sentence.
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,161 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Set up the expansion service
+***Note**: In Apache Beam 2.44.0 and later versions, you can use the `withExtraPackages` method to specify the required local packages directly in the Java pipeline. This means you do not need to set up an expansion service anymore and can skip this step.*
+
+Because this example uses transforms from two different languages, we need an SDK for each language, in this case Python and Java. We also need to set up an expansion service, which is used to inject the cross-language Python transform into the Java pipeline. This should be done in a virtual environment which is setup for Beam. [This](https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment) guide shows how set it up.
Review Comment:
Change the following:
"This should be done in a virtual environment which is setup for Beam. [This](https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment) guide shows how set it up."
to:
"Do this step in a virtual environment that is set up for Beam. To see how to set it up, see [Create and activate a virtual environment](https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment) in the Apache Beam Python SDK Quickstart.:
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,161 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
Review Comment:
It would be great if this intro paragraph had one sentence summarizing what this example does. For example:
This example uses a natural language processing (NLP) model and dataset to create a cross-language pipeline using Apache Beam's RunInference transform.
##########
website/www/site/content/en/documentation/ml/multi-language-inference.md:
##########
@@ -0,0 +1,161 @@
+---
+title: "Cross Language RunInference "
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Cross Language RunInference
+
+The pipeline is written in Java and reads the input data from Google Cloud Storage. With the help of a [PythonExternalTransform](https://beam.apache.org/documentation/programming-guide/#1312-creating-cross-language-python-transforms),
+a composite Python transform is called to do the preprocessing, postprocessing, and inference.
+Lastly, the data is written back to Google Cloud Storage in the Java pipeline.
+
+## NLP model and dataset
+A `bert-base-uncased` natural language processing (NLP) model is used to make inference. This model is open source and available on [HuggingFace](https://huggingface.co/bert-base-uncased). This BERT-model is
+used to predict the last word of a sentence based on the context of the sentence.
+
+We also use an [IMDB movie reviews](https://www.kaggle.com/datasets/lakshmi25npathi/imdb-dataset-of-50k-movie-reviews?select=IMDB+Dataset.csv) dataset, which is an open-source dataset that is available on Kaggle.
+
+The following is a sample of the data after preprocessing:
+
+| **Text** | **Last Word** |
+|--- |:--- |
+|<img width=700/>|<img width=100/>|
+| One of the other reviewers has mentioned that after watching just 1 Oz episode you'll be [MASK] | hooked |
+| A wonderful little [MASK] | production |
+| So im not a big fan of Boll's work but then again not many [MASK] | are |
+| This a fantastic movie of three prisoners who become [MASK] | famous |
+| Some films just simply should not be [MASK] | remade |
+| The Karen Carpenter Story shows a little more about singer Karen Carpenter's complex [MASK] | life |
+
+You can see the full code used in this example on [Github](https://github.com/apache/beam/tree/master/sdks/python/apache_beam/examples/inference/multi_language_inference).
+
+
+## Multi-language Inference pipeline
+
+When using multi-language pipelines, you have access to a much larger pool of transforms. For more information, see [Multi-language pipelines](https://beam.apache.org/documentation/programming-guide/#multi-language-pipelines) in the Apache Beam Programming Guide.
+
+### Cross-Language Python transform
+In addition to running inference, we also need to perform preprocessing and postprocessing on the data. Processing the data makes it possible to interpret the output. In order to do these three tasks, one single composite custom PTransform is written, with a unit DoFn or PTransform for each of the tasks, shown in the following example:
+
+```python
+def expand(self, pcoll):
+ return (
+ pcoll
+ | 'Preprocess' >> beam.ParDo(self.Preprocess(self._tokenizer))
+ | 'Inference' >> RunInference(KeyedModelHandler(self._model_handler))
+ | 'Postprocess' >> beam.ParDo(self.Postprocess(
+ self._tokenizer))
+ )
+```
+
+First, the preprocessing of the data. In this case, the raw textual data is cleaned and tokenized for the BERT-model. All these steps are run in the `Preprocess` DoFn. The `Preprocess` DoFn takes a single element as input and returns a list with both the original text and the tokenized text.
+
+The preprocessed data is then used to make inference. This is done in the [`RunInference`](https://beam.apache.org/documentation/ml/overview/#runinference) PTransform, which is already available in the Apache Beam SDK. The `RunInference` PTransform requires one parameter, a model handler. In this example the `KeyedModelHandler` is used, because the `Preprocess` DoFn also outputs the original sentence. You can change how preprocessing is done based on your requirements. This model handler is defined in the following initialization function of the composite PTransform:
+
+```python
+def __init__(self, model, model_path):
+ self._model = model
+ logging.info(f"Downloading {self._model} model from GCS.")
+ self._model_config = BertConfig.from_pretrained(self._model)
+ self._tokenizer = BertTokenizer.from_pretrained(self._model)
+ self._model_handler = self.PytorchModelHandlerKeyedTensorWrapper(
+ state_dict_path=(model_path),
+ model_class=BertForMaskedLM,
+ model_params={'config': self._model_config},
+ device='cuda:0')
+```
+The `PytorchModelHandlerKeyedTensorWrapper`, a wrapper around the `PytorchModelHandlerKeyedTensor` model handler, is used. The `PytorchModelHandlerKeyedTensor` model handler makes inference on a PyTorch model. Because the tokenized strings generated from `BertTokenizer` might have different lengths and stack() requires tensors to be the same size, the `PytorchModelHandlerKeyedTensorWrapper` limits the batch size to 1. Restricting `max_batch_size` to 1 means the run_inference() call contains one example per batch. The following example shows the definition of the wrapper:
+
+```python
+class PytorchModelHandlerKeyedTensorWrapper(PytorchModelHandlerKeyedTensor):
+
+ def batch_elements_kwargs(self):
+ return {'max_batch_size': 1}
+```
+An alternative aproach is to make all the tensors have the same length. This [example](https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/run_inference_pytorch_tensorflow_sklearn.ipynb) shows how to do that.
+
+
+The `ModelConfig` and `ModelTokenizer` are loaded in the initialization function. The `ModelConfig` is used to define the model architecture, and the `ModelTokenizer` is used to tokenize the input data. The following two parameters are used for these tasks:
+- `model`: The name of the model that is used for inference. In this example it is `bert-base-uncased`.
+- `model_path`: The path to the `state_dict` of the model that is used for inference. In this example it is a path to a Google Cloud Storage bucket, where the `state_dict` is stored.
+
+Both of these parameters are specified in the Java `PipelineOptions`.
+
+Finally, postprocess the model predictions in the `Postprocess` DoFn. The `Postprocess` DoFn returns the original text, the last word of the sentence, and the predicted word.
+
+### Set up the expansion service
+***Note**: In Apache Beam 2.44.0 and later versions, you can use the `withExtraPackages` method to specify the required local packages directly in the Java pipeline. This means you do not need to set up an expansion service anymore and can skip this step.*
+
+Because this example uses transforms from two different languages, we need an SDK for each language, in this case Python and Java. We also need to set up an expansion service, which is used to inject the cross-language Python transform into the Java pipeline. This should be done in a virtual environment which is setup for Beam. [This](https://beam.apache.org/get-started/quickstart-py/#create-and-activate-a-virtual-environment) guide shows how set it up.
+
+To set up the expansion service in the created virtual environment, run the following command in the terminal:
+
+```bash
+export PORT = <port to host expansion service>
+export IMAGE = <custom docker image>
+
+python -m multi_language_custom_transform.start_expansion_service \
+ --port=$PORT \
+ --fully_qualified_name_glob="*" \
+ --environment_config=$IMAGE \
+ --environment_type=DOCKER
+```
+
+This runs the `run_inference_expansion.py` script, which starts the expansion service. Specify two important parameters. First, the `port` parameter specifies the port that the expansion service is hosted on. Second, the `environment_config` parameter specifies the docker image used to run the Python transform. The docker image must contain the Apache Beam Python SDK and all of the other dependencies required to run the transform. For this example, the composite Python PTransform, from the previous section, is wrapped in a local package. Next, we install this local package on the custom docker image.
+
+If no custom transforms are used, you can use the default Apache Beam image. This image is used automatically if you don't specify an `environment_config` parameter. In this scenario, you also don't need to start up an explicit expansion service. The expansion service is started automatically when `PythonExternalTransform` is called.
+
+### Run the Java pipeline
+The Java pipeline is defined in the `MultiLangRunInference` class. In this pipeline, the data is read from Google Cloud Storage, the cross-language Python transform is applied, and the output is written to Google Cloud Storage. The following example shows the pipeline:
+
+```java
+Pipeline p = Pipeline.create(options);
+ PCollection<String> input = p.apply("Read Input", TextIO.read().from(options.getInputFile()));
+
+ input.apply("Predict", PythonExternalTransform.
+ <PCollection<String>PCollection<String>>from(
+ "multi_language_custom_transform.\
+ composite_transform.InferenceTransform", "localhost:" + options.getPort())
+ .withKwarg("model", options.getModelName())
+ .withKwarg("model_path", options.getModelPath()))
+ .apply("Write Output", TextIO.write().to(options.getOutputFile()
+ )
+ );
+ p.run().waitUntilFinish();
+```
+
+Use `PythonExternalTransform` to inject the cross-language Python transform. `PythonExternalTransform` takes two parameters. The first parameter is the fully qualified name of the Python transform. The second parameter is the expansion service address. The `withKwarg` method is used to specify the parameters that are needed for the Python transform. In this example the `model` and `model_path` parameters are specified. These parameters are used in the initialization function of the composite Python PTransform, as shown in the first section.
+
+To run the pipeline, use the following command:
+
+```bash
+mvn compile exec:java -Dexec.mainClass=org.MultiLangRunInference \
+ -Dexec.args="--runner=DataflowRunner --project=$GCP_PROJECT\
+ --region=$GCP_REGION \
+ --gcpTempLocation=gs://$GCP_BUCKET/temp/ \
+ --inputFile=gs://$GCP_BUCKET/input/imdb_reviews.csv \
+ --outputFile=gs://$GCP_BUCKET/output/ouput.txt \
+ --modelPath=gs://$GCP_BUCKET/input/bert-model/bert-base-uncased.pth \
+ --modelName=$MODEL_NAME \
+ --port=$PORT" \
+ -Pdataflow-runner \
+ -e
+```
+The standard Google Cloud and Runner parameters are specified. The `inputFile` and `outputFile` parameters are used to specify the input and output files. The `modelPath` and `modelName` custom parameters are passed to the `PythonExternalTransform`. The port parameter is used to specify the port on which the expansion service is hosted.
Review Comment:
I would convert these sentences to a list. Change:
"The `inputFile` and `outputFile` parameters are used to specify the input and output files. The `modelPath` and `modelName` custom parameters are passed to the `PythonExternalTransform`. The port parameter is used to specify the port on which the expansion service is hosted."
To:
- The `inputFile` and `outputFile` parameters are used to specify the input and output files.
- The `modelPath` and `modelName` custom parameters are passed to the `PythonExternalTransform`.
- The port parameter is used to specify the port on which the expansion service is hosted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] chamikaramj commented on pull request #25083: Multi language/runinference example
Posted by "chamikaramj (via GitHub)" <gi...@apache.org>.
chamikaramj commented on PR #25083:
URL: https://github.com/apache/beam/pull/25083#issuecomment-1414591988
Hi, any updates on this ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] rszper commented on a diff in pull request #25083: Multi language/runinference example
Posted by "rszper (via GitHub)" <gi...@apache.org>.
rszper commented on code in PR #25083:
URL: https://github.com/apache/beam/pull/25083#discussion_r1124976782
##########
website/www/site/content/en/documentation/ml/overview.md:
##########
@@ -105,3 +105,4 @@ You can find examples of end-to-end AI/ML pipelines for several use cases:
* [Anomaly Detection in Beam](/documentation/ml/anomaly-detection): Demonstrates how to set up an anomaly detection pipeline that reads text from Pub/Sub in real time and then detects anomalies using a trained HDBSCAN clustering model with the RunInference API.
* [Large Language Model Inference in Beam](/documentation/ml/large-language-modeling): Demonstrates a pipeline that uses RunInference to perform translation with the T5 language model which contains 11 billion parameters.
* [Per Entity Training in Beam](/documentation/ml/per-entity-training): Demonstrates a pipeline that trains a Decision Tree Classifier per education level for predicting if the salary of a person is >= 50k.
+* [Cross-language RunInference in Beam](/documentation/ml/multi-language-inference): Demonstrates a cross-language java pipeline that uses a custom python transform to do preprocessing, postprocessing, and inference.
Review Comment:
Note that this page is being changed in https://github.com/apache/beam/pull/25700/files. If that PR is merged first, this could be added either to the Data processing or to the Prediction and inference examples, depending on which you think is more appropriate.
##########
website/www/site/layouts/partials/section-menu/en/documentation.html:
##########
@@ -1,367 +1,368 @@
{{/*
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
-*/}}
-
-<li><span class="section-nav-list-main-title">Documentation</span></li>
-<li><a href="/documentation">Using the Documentation</a></li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Concepts</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/basics/">Basics of the Beam model</a></li>
- <li><a href="/documentation/runtime/model/">How Beam executes a pipeline</a></li>
- </ul>
-</li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Beam programming guide</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/">Overview</a></li>
- <li><a href="/documentation/programming-guide/#creating-a-pipeline">Pipelines</a></li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">PCollections</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#pcollections">Creating a PCollection</a></li>
- <li><a href="/documentation/programming-guide/#pcollection-characteristics">PCollection characteristics</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Transforms</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#applying-transforms">Applying transforms</a></li>
- <li>
- <span class="section-nav-list-title">Core Beam transforms</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#pardo">ParDo</a></li>
- <li><a href="/documentation/programming-guide/#groupbykey">GroupByKey</a></li>
- <li><a href="/documentation/programming-guide/#cogroupbykey">CoGroupByKey</a></li>
- <li><a href="/documentation/programming-guide/#combine">Combine</a></li>
- <li><a href="/documentation/programming-guide/#flatten">Flatten</a></li>
- <li><a href="/documentation/programming-guide/#partition">Partition</a></li>
- </ul>
- </li>
-
- <li><a href="/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms">Requirements for user code</a></li>
- <li><a href="/documentation/programming-guide/#side-inputs">Side inputs</a></li>
- <li><a href="/documentation/programming-guide/#additional-outputs">Additional outputs</a></li>
- <li><a href="/documentation/programming-guide/#composite-transforms">Composite transforms</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Pipeline I/O</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#pipeline-io">Using I/O transforms</a></li>
- <li><a href="/documentation/io/connectors/">I/O connectors</a></li>
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">I/O connector guides</span>
- <ul class="section-nav-list">
- <li><a href="/documentation/io/built-in/parquet/">Apache Parquet I/O connector</a></li>
- <li><a href="/documentation/io/built-in/hadoop/">Hadoop Input/Output Format IO</a></li>
- <li><a href="/documentation/io/built-in/hcatalog/">HCatalog IO</a></li>
- <li><a href="/documentation/io/built-in/google-bigquery/">Google BigQuery I/O connector</a></li>
- <li><a href="/documentation/io/built-in/snowflake/">Snowflake I/O connector</a></li>
- <li><a href="/documentation/io/built-in/cdap/">CDAP I/O connector</a></li>
- <li><a href="/documentation/io/built-in/sparkreceiver/">Spark Receiver I/O connector</a></li>
- <li><a href="/documentation/io/built-in/singlestore/">SingleStoreDB I/O connector</a></li>
- </ul>
- </li>
-
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Developing new I/O connectors</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/io/developing-io-overview/">Overview: Developing connectors</a></li>
- <li><a href="/documentation/io/developing-io-java/">Developing connectors (Java)</a></li>
- <li><a href="/documentation/io/developing-io-python/">Developing connectors (Python)</a></li>
- <li><a href="/documentation/io/io-standards/">I/O Standards</a></li>
- </ul>
- </li>
-
- <li><a href="/documentation/io/testing/">Testing I/O transforms</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Schemas</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#what-is-a-schema">What is a schema</a></li>
- <li><a href="/documentation/programming-guide/#schemas-for-pl-types">Schemas for programming language types</a></li>
- <li><a href="/documentation/programming-guide/#schema-definition">Schema definition</a></li>
- <li><a href="/documentation/programming-guide/#logical-types">Logical types</a></li>
- <li><a href="/documentation/programming-guide/#creating-schemas">Creating schemas</a></li>
- <li><a href="/documentation/programming-guide/#using-schemas">Using schemas</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Data encoding and type safety</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#data-encoding-and-type-safety">Data encoding basics</a></li>
- <li><a href="/documentation/programming-guide/#specifying-coders">Specifying coders</a></li>
- <li><a href="/documentation/programming-guide/#default-coders-and-the-coderregistry">Default coders and the CoderRegistry</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Windowing</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#windowing">Windowing basics</a></li>
- <li><a href="/documentation/programming-guide/#provided-windowing-functions">Provided windowing functions</a></li>
- <li><a href="/documentation/programming-guide/#setting-your-pcollections-windowing-function">Setting your PCollection’s windowing function</a></li>
- <li><a href="/documentation/programming-guide/#watermarks-and-late-data">Watermarks and late data</a></li>
- <li><a href="/documentation/programming-guide/#adding-timestamps-to-a-pcollections-elements">Adding timestamps to a PCollection’s elements</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Triggers</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#triggers">Trigger basics</a></li>
- <li><a href="/documentation/programming-guide/#event-time-triggers">Event time triggers and the default trigger</a></li>
- <li><a href="/documentation/programming-guide/#processing-time-triggers">Processing time triggers</a></li>
- <li><a href="/documentation/programming-guide/#data-driven-triggers">Data-driven triggers</a></li>
- <li><a href="/documentation/programming-guide/#setting-a-trigger">Setting a trigger</a></li>
- <li><a href="/documentation/programming-guide/#composite-triggers">Composite triggers</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Metrics</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#metrics">Metrics basics</a></li>
- <li><a href="/documentation/programming-guide/#types-of-metrics">Types of metrics</a></li>
- <li><a href="/documentation/programming-guide/#querying-metrics">Querying metrics</a></li>
- <li><a href="/documentation/programming-guide/#using-metrics">Using metrics in pipeline</a></li>
- <li><a href="/documentation/programming-guide/#export-metrics">Export metrics</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">State and Timers</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#types-of-state">Types of state</a></li>
- <li><a href="/documentation/programming-guide/#deferred-state-reads">Deferred state reads</a></li>
- <li><a href="/documentation/programming-guide/#timers">Timers</a></li>
- <li><a href="/documentation/programming-guide/#garbage-collecting-state">Garbage collecting state</a></li>
- <li><a href="/documentation/programming-guide/#state-timers-examples">State and timers examples</a></li>
- </ul>
- </li>
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Splittable DoFns</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#sdf-basics">Basics</a></li>
- <li><a href="/documentation/programming-guide/#sizing-and-progress">Sizing and progress</a></li>
- <li><a href="/documentation/programming-guide/#user-initiated-checkpoint">User-initiated checkpoint</a></li>
- <li><a href="/documentation/programming-guide/#runner-initiated-split">Runner initiated split</a></li>
- <li><a href="/documentation/programming-guide/#watermark-estimation">Watermark estimation</a></li>
- <li><a href="/documentation/programming-guide/#truncating-during-drain">Truncating during drain</a></li>
- <li><a href="/documentation/programming-guide/#bundle-finalization">Bundle finalization</a></li>
- </ul>
- </li>
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Multi-language Pipelines</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/programming-guide/#create-x-lang-transforms">Creating cross-language transforms</a></li>
- <li><a href="/documentation/programming-guide/#use-x-lang-transforms">Using cross-language transforms</a></li>
- <li><a href="/documentation/programming-guide/#x-lang-transform-runner-support">Runner Support</a></li>
- </ul>
- </li>
- <li><a href="/documentation/programming-guide/#batched-dofns">Batched DoFns</a></li>
- </ul>
-</li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Pipeline development lifecycle</span>
- <ul class="section-nav-list">
- <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a></li>
- <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a></li>
- <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a></li>
- </ul>
-</li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Common pipeline patterns</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/patterns/overview/">Overview</a></li>
- <li><a href="/documentation/patterns/file-processing/">File processing</a></li>
- <li><a href="/documentation/patterns/side-inputs/">Side inputs</a></li>
- <li><a href="/documentation/patterns/pipeline-options/">Pipeline options</a></li>
- <li><a href="/documentation/patterns/custom-io/">Custom I/O</a></li>
- <li><a href="/documentation/patterns/custom-windows/">Custom windows</a></li>
- <li><a href="/documentation/patterns/bigqueryio/">BigQueryIO</a></li>
- <li><a href="/documentation/patterns/ai-platform/">AI Platform</a></li>
- <li><a href="/documentation/patterns/schema/">Schema</a></li>
- <li><a href="/documentation/patterns/bqml/">BigQuery ML</a></li>
- <li><a href="/documentation/patterns/grouping-elements-for-efficient-external-service-calls/">Grouping elements for efficient external service calls</a></li>
- </ul>
-</li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">AI/ML pipelines</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/ml/overview/">Overview</a></li>
- <li><a href="/documentation/ml/data-processing/">Data processing</a></li>
- <li><a href="/documentation/ml/runinference-metrics/">RunInference Metrics</a></li>
- <li><a href="/documentation/ml/tensorrt-runinference">TensorRT Inference</a></li>
- <li><a href="/documentation/ml/model-evaluation/">Model evaluation</a></li>
- <li><a href="/documentation/ml/orchestration/">Workflow Orchestration</a></li>
- <li><a href="/documentation/ml/multi-model-pipelines/">Multi-model pipelines</a></li>
- <li><a href="/documentation/ml/online-clustering/">Online Clustering</a></li>
- <li><a href="/documentation/ml/anomaly-detection/">Anomaly Detection</a></li>
- <li><a href="/documentation/ml/large-language-modeling">Large Language Model Inference in Beam</a></li>
- <li><a href="/documentation/ml/per-entity-training">Per Entity Training in Beam</a></li>
- </ul>
-</li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Runtime systems</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/runtime/environments/">Container environments</a></li>
- <li><a href="/documentation/runtime/resource-hints/">Resource hints</a></li>
- <li><a href="/documentation/runtime/sdk-harness-config/">SDK Harness Configuration</a></li>
- </ul>
-</li>
-<li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Transform catalog</span>
-
- <ul class="section-nav-list">
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Python</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/overview/">Overview</a></li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Element-wise</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/elementwise/filter/">Filter</a></li>
- <li><a href="/documentation/transforms/python/elementwise/flatmap/">FlatMap</a></li>
- <li><a href="/documentation/transforms/python/elementwise/keys/">Keys</a></li>
- <li><a href="/documentation/transforms/python/elementwise/kvswap/">KvSwap</a></li>
- <li><a href="/documentation/transforms/python/elementwise/map/">Map</a></li>
- <li><a href="/documentation/transforms/python/elementwise/pardo/">ParDo</a></li>
- <li><a href="/documentation/transforms/python/elementwise/partition/">Partition</a></li>
- <li><a href="/documentation/transforms/python/elementwise/regex/">Regex</a></li>
- <li><a href="/documentation/transforms/python/elementwise/reify/">Reify</a></li>
- <li><a href="/documentation/transforms/python/elementwise/runinference/">RunInference</a></li>
- <li><a href="/documentation/transforms/python/elementwise/tostring/">ToString</a></li>
- <li><a href="/documentation/transforms/python/elementwise/values/">Values</a></li>
- <li><a href="/documentation/transforms/python/elementwise/withtimestamps/">WithTimestamps</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Aggregation</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/aggregation/cogroupbykey/">CoGroupByKey</a></li>
- <li><a href="/documentation/transforms/python/aggregation/combineglobally/">CombineGlobally</a></li>
- <li><a href="/documentation/transforms/python/aggregation/combineperkey/">CombinePerKey</a></li>
- <li><a href="/documentation/transforms/python/aggregation/combinevalues/">CombineValues</a></li>
- <li><a href="/documentation/transforms/python/aggregation/count/">Count</a></li>
- <li><a href="/documentation/transforms/python/aggregation/distinct/">Distinct</a></li>
- <li><a href="/documentation/transforms/python/aggregation/groupbykey/">GroupByKey</a></li>
- <li><a href="/documentation/transforms/python/aggregation/groupby/">GroupBy</a></li>
- <li><a href="/documentation/transforms/python/aggregation/groupintobatches/">GroupIntoBatches</a></li>
- <li><a href="/documentation/transforms/python/aggregation/latest/">Latest</a></li>
- <li><a href="/documentation/transforms/python/aggregation/max/">Max</a></li>
- <li><a href="/documentation/transforms/python/aggregation/min/">Min</a></li>
- <li><a href="/documentation/transforms/python/aggregation/mean/">Mean</a></li>
- <li><a href="/documentation/transforms/python/aggregation/sample/">Sample</a></li>
- <li><a href="/documentation/transforms/python/aggregation/sum/">Sum</a></li>
- <li><a href="/documentation/transforms/python/aggregation/top/">Top</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Other</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/python/other/create/">Create</a></li>
- <li><a href="/documentation/transforms/python/other/flatten/">Flatten</a></li>
- <li><a href="/documentation/transforms/python/other/reshuffle/">Reshuffle</a></li>
- <li><a href="/documentation/transforms/python/other/windowinto/">WindowInto</a></li>
- </ul>
- </li>
- </ul>
- </li>
-
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Java</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/overview/">Overview</a></li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Element-wise</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/elementwise/filter/">Filter</a></li>
- <li><a href="/documentation/transforms/java/elementwise/flatmapelements/">FlatMapElements</a></li>
- <li><a href="/documentation/transforms/java/elementwise/keys/">Keys</a></li>
- <li><a href="/documentation/transforms/java/elementwise/kvswap/">KvSwap</a></li>
- <li><a href="/documentation/transforms/java/elementwise/mapelements/">MapElements</a></li>
- <li><a href="/documentation/transforms/java/elementwise/pardo/">ParDo</a></li>
- <li><a href="/documentation/transforms/java/elementwise/partition/">Partition</a></li>
- <li><a href="/documentation/transforms/java/elementwise/regex/">Regex</a></li>
- <li><a href="/documentation/transforms/java/elementwise/reify/">Reify</a></li>
- <li><a href="/documentation/transforms/java/elementwise/values/">Values</a></li>
- <li><a href="/documentation/transforms/java/elementwise/tostring/">ToString</a></li>
- <li><a href="/documentation/transforms/java/elementwise/withkeys/">WithKeys</a></li>
- <li><a href="/documentation/transforms/java/elementwise/withtimestamps/">WithTimestamps</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Aggregation</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/aggregation/approximatequantiles/">ApproximateQuantiles</a></li>
- <li><a href="/documentation/transforms/java/aggregation/approximateunique/">ApproximateUnique</a></li>
- <li><a href="/documentation/transforms/java/aggregation/cogroupbykey/">CoGroupByKey</a></li>
- <li><a href="/documentation/transforms/java/aggregation/combine/">Combine</a></li>
- <li><a href="/documentation/transforms/java/aggregation/combinewithcontext/">CombineWithContext</a></li>
- <li><a href="/documentation/transforms/java/aggregation/count/">Count</a></li>
- <li><a href="/documentation/transforms/java/aggregation/distinct/">Distinct</a></li>
- <li><a href="/documentation/transforms/java/aggregation/groupbykey/">GroupByKey</a></li>
- <li><a href="/documentation/transforms/java/aggregation/groupintobatches/">GroupIntoBatches</a></li>
- <li><a href="/documentation/transforms/java/aggregation/hllcount/">HllCount</a></li>
- <li><a href="/documentation/transforms/java/aggregation/latest/">Latest</a></li>
- <li><a href="/documentation/transforms/java/aggregation/max/">Max</a></li>
- <li><a href="/documentation/transforms/java/aggregation/mean/">Mean</a></li>
- <li><a href="/documentation/transforms/java/aggregation/min/">Min</a></li>
- <li><a href="/documentation/transforms/java/aggregation/sample/">Sample</a></li>
- <li><a href="/documentation/transforms/java/aggregation/sum/">Sum</a></li>
- <li><a href="/documentation/transforms/java/aggregation/top/">Top</a></li>
- </ul>
- </li>
- <li class="section-nav-item--collapsible">
- <span class="section-nav-list-title">Other</span>
-
- <ul class="section-nav-list">
- <li><a href="/documentation/transforms/java/other/create/">Create</a></li>
- <li><a href="/documentation/transforms/java/other/flatten/">Flatten</a></li>
- <li><a href="/documentation/transforms/java/other/passert/">PAssert</a></li>
- <li><a href="/documentation/transforms/java/other/view/">View</a></li>
- <li><a href="/documentation/transforms/java/other/window/">Window</a></li>
- </ul>
- </li>
- </ul>
- </li>
- </ul>
-</li>
-<li><a href="/documentation/glossary/">Glossary</a></li>
-<li><a href="https://cwiki.apache.org/confluence/display/BEAM/Apache+Beam">Beam Wiki <img src="/images/external-link-icon.png" width="14" height="14" alt="External link."></a></li>
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+ */}}
+
+ <li><span class="section-nav-list-main-title">Documentation</span></li>
+ <li><a href="/documentation">Using the Documentation</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Concepts</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/basics/">Basics of the Beam model</a></li>
+ <li><a href="/documentation/runtime/model/">How Beam executes a pipeline</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Beam programming guide</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/">Overview</a></li>
+ <li><a href="/documentation/programming-guide/#creating-a-pipeline">Pipelines</a></li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">PCollections</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#pcollections">Creating a PCollection</a></li>
+ <li><a href="/documentation/programming-guide/#pcollection-characteristics">PCollection characteristics</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Transforms</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#applying-transforms">Applying transforms</a></li>
+ <li>
+ <span class="section-nav-list-title">Core Beam transforms</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#pardo">ParDo</a></li>
+ <li><a href="/documentation/programming-guide/#groupbykey">GroupByKey</a></li>
+ <li><a href="/documentation/programming-guide/#cogroupbykey">CoGroupByKey</a></li>
+ <li><a href="/documentation/programming-guide/#combine">Combine</a></li>
+ <li><a href="/documentation/programming-guide/#flatten">Flatten</a></li>
+ <li><a href="/documentation/programming-guide/#partition">Partition</a></li>
+ </ul>
+ </li>
+
+ <li><a href="/documentation/programming-guide/#requirements-for-writing-user-code-for-beam-transforms">Requirements for user code</a></li>
+ <li><a href="/documentation/programming-guide/#side-inputs">Side inputs</a></li>
+ <li><a href="/documentation/programming-guide/#additional-outputs">Additional outputs</a></li>
+ <li><a href="/documentation/programming-guide/#composite-transforms">Composite transforms</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Pipeline I/O</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#pipeline-io">Using I/O transforms</a></li>
+ <li><a href="/documentation/io/connectors/">I/O connectors</a></li>
+
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">I/O connector guides</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/io/built-in/parquet/">Apache Parquet I/O connector</a></li>
+ <li><a href="/documentation/io/built-in/hadoop/">Hadoop Input/Output Format IO</a></li>
+ <li><a href="/documentation/io/built-in/hcatalog/">HCatalog IO</a></li>
+ <li><a href="/documentation/io/built-in/google-bigquery/">Google BigQuery I/O connector</a></li>
+ <li><a href="/documentation/io/built-in/snowflake/">Snowflake I/O connector</a></li>
+ <li><a href="/documentation/io/built-in/cdap/">CDAP I/O connector</a></li>
+ <li><a href="/documentation/io/built-in/sparkreceiver/">Spark Receiver I/O connector</a></li>
+ <li><a href="/documentation/io/built-in/singlestore/">SingleStoreDB I/O connector</a></li>
+ </ul>
+ </li>
+
+
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Developing new I/O connectors</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/io/developing-io-overview/">Overview: Developing connectors</a></li>
+ <li><a href="/documentation/io/developing-io-java/">Developing connectors (Java)</a></li>
+ <li><a href="/documentation/io/developing-io-python/">Developing connectors (Python)</a></li>
+ <li><a href="/documentation/io/io-standards/">I/O Standards</a></li>
+ </ul>
+ </li>
+
+ <li><a href="/documentation/io/testing/">Testing I/O transforms</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Schemas</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#what-is-a-schema">What is a schema</a></li>
+ <li><a href="/documentation/programming-guide/#schemas-for-pl-types">Schemas for programming language types</a></li>
+ <li><a href="/documentation/programming-guide/#schema-definition">Schema definition</a></li>
+ <li><a href="/documentation/programming-guide/#logical-types">Logical types</a></li>
+ <li><a href="/documentation/programming-guide/#creating-schemas">Creating schemas</a></li>
+ <li><a href="/documentation/programming-guide/#using-schemas">Using schemas</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Data encoding and type safety</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#data-encoding-and-type-safety">Data encoding basics</a></li>
+ <li><a href="/documentation/programming-guide/#specifying-coders">Specifying coders</a></li>
+ <li><a href="/documentation/programming-guide/#default-coders-and-the-coderregistry">Default coders and the CoderRegistry</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Windowing</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#windowing">Windowing basics</a></li>
+ <li><a href="/documentation/programming-guide/#provided-windowing-functions">Provided windowing functions</a></li>
+ <li><a href="/documentation/programming-guide/#setting-your-pcollections-windowing-function">Setting your PCollection’s windowing function</a></li>
+ <li><a href="/documentation/programming-guide/#watermarks-and-late-data">Watermarks and late data</a></li>
+ <li><a href="/documentation/programming-guide/#adding-timestamps-to-a-pcollections-elements">Adding timestamps to a PCollection’s elements</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Triggers</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#triggers">Trigger basics</a></li>
+ <li><a href="/documentation/programming-guide/#event-time-triggers">Event time triggers and the default trigger</a></li>
+ <li><a href="/documentation/programming-guide/#processing-time-triggers">Processing time triggers</a></li>
+ <li><a href="/documentation/programming-guide/#data-driven-triggers">Data-driven triggers</a></li>
+ <li><a href="/documentation/programming-guide/#setting-a-trigger">Setting a trigger</a></li>
+ <li><a href="/documentation/programming-guide/#composite-triggers">Composite triggers</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Metrics</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#metrics">Metrics basics</a></li>
+ <li><a href="/documentation/programming-guide/#types-of-metrics">Types of metrics</a></li>
+ <li><a href="/documentation/programming-guide/#querying-metrics">Querying metrics</a></li>
+ <li><a href="/documentation/programming-guide/#using-metrics">Using metrics in pipeline</a></li>
+ <li><a href="/documentation/programming-guide/#export-metrics">Export metrics</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">State and Timers</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#types-of-state">Types of state</a></li>
+ <li><a href="/documentation/programming-guide/#deferred-state-reads">Deferred state reads</a></li>
+ <li><a href="/documentation/programming-guide/#timers">Timers</a></li>
+ <li><a href="/documentation/programming-guide/#garbage-collecting-state">Garbage collecting state</a></li>
+ <li><a href="/documentation/programming-guide/#state-timers-examples">State and timers examples</a></li>
+ </ul>
+ </li>
+
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Splittable DoFns</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#sdf-basics">Basics</a></li>
+ <li><a href="/documentation/programming-guide/#sizing-and-progress">Sizing and progress</a></li>
+ <li><a href="/documentation/programming-guide/#user-initiated-checkpoint">User-initiated checkpoint</a></li>
+ <li><a href="/documentation/programming-guide/#runner-initiated-split">Runner initiated split</a></li>
+ <li><a href="/documentation/programming-guide/#watermark-estimation">Watermark estimation</a></li>
+ <li><a href="/documentation/programming-guide/#truncating-during-drain">Truncating during drain</a></li>
+ <li><a href="/documentation/programming-guide/#bundle-finalization">Bundle finalization</a></li>
+ </ul>
+ </li>
+
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Multi-language Pipelines</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/programming-guide/#create-x-lang-transforms">Creating cross-language transforms</a></li>
+ <li><a href="/documentation/programming-guide/#use-x-lang-transforms">Using cross-language transforms</a></li>
+ <li><a href="/documentation/programming-guide/#x-lang-transform-runner-support">Runner Support</a></li>
+ </ul>
+ </li>
+ <li><a href="/documentation/programming-guide/#batched-dofns">Batched DoFns</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Pipeline development lifecycle</span>
+ <ul class="section-nav-list">
+ <li><a href="/documentation/pipelines/design-your-pipeline/">Design Your Pipeline</a></li>
+ <li><a href="/documentation/pipelines/create-your-pipeline/">Create Your Pipeline</a></li>
+ <li><a href="/documentation/pipelines/test-your-pipeline/">Test Your Pipeline</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">Common pipeline patterns</span>
+
+ <ul class="section-nav-list">
+ <li><a href="/documentation/patterns/overview/">Overview</a></li>
+ <li><a href="/documentation/patterns/file-processing/">File processing</a></li>
+ <li><a href="/documentation/patterns/side-inputs/">Side inputs</a></li>
+ <li><a href="/documentation/patterns/pipeline-options/">Pipeline options</a></li>
+ <li><a href="/documentation/patterns/custom-io/">Custom I/O</a></li>
+ <li><a href="/documentation/patterns/custom-windows/">Custom windows</a></li>
+ <li><a href="/documentation/patterns/bigqueryio/">BigQueryIO</a></li>
+ <li><a href="/documentation/patterns/ai-platform/">AI Platform</a></li>
+ <li><a href="/documentation/patterns/schema/">Schema</a></li>
+ <li><a href="/documentation/patterns/bqml/">BigQuery ML</a></li>
+ <li><a href="/documentation/patterns/grouping-elements-for-efficient-external-service-calls/">Grouping elements for efficient external service calls</a></li>
+ </ul>
+ </li>
+ <li class="section-nav-item--collapsible">
+ <span class="section-nav-list-title">AI/ML pipelines</span>
Review Comment:
Note that this section of this page will change in https://github.com/apache/beam/pull/25700/files. If that PR is merged first, add the new topic to whichever category in the left nav corresponds to the table in the Overview page.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org