You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by an...@apache.org on 2024/01/03 22:21:59 UTC
(beam) branch update_docs created (now 031d99a978e)
This is an automated email from the ASF dual-hosted git repository.
anandinguva pushed a change to branch update_docs
in repository https://gitbox.apache.org/repos/asf/beam.git
at 031d99a978e Update MLTransform docs
This branch includes the following new commits:
new 031d99a978e Update MLTransform docs
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(beam) 01/01: Update MLTransform docs
Posted by an...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
anandinguva pushed a commit to branch update_docs
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 031d99a978e1bfb485b9a971bc1449b966f3557d
Author: AnandInguva <an...@gmail.com>
AuthorDate: Wed Jan 3 17:21:28 2024 -0500
Update MLTransform docs
---
.../content/en/documentation/ml/preprocess-data.md | 46 +++-------------------
1 file changed, 5 insertions(+), 41 deletions(-)
diff --git a/website/www/site/content/en/documentation/ml/preprocess-data.md b/website/www/site/content/en/documentation/ml/preprocess-data.md
index 2b291b9c75a..567188a09fb 100644
--- a/website/www/site/content/en/documentation/ml/preprocess-data.md
+++ b/website/www/site/content/en/documentation/ml/preprocess-data.md
@@ -31,8 +31,6 @@ modules for machine learning tasks.
- With `MLTransform`, you can use the same preprocessing steps for both
training and inference, which ensures consistent results.
-- Use `MLTransform` to transform a single example or a batch of
- examples.
- `MLTransform` can do a full pass on the dataset, which is useful when
you need to transform a single element only after analyzing the entire
dataset. For example, with `MLTransform`, you can complete the following tasks:
@@ -48,10 +46,9 @@ modules for machine learning tasks.
## Support and limitations {#support}
-- Available in the Apache Beam Python SDK versions 2.50.0 and later.
-- Supports Python 3.8 and 3.9.
+- Available in the Apache Beam Python SDK versions 2.53.0 and later.
+- Supports Python 3.8, 3.9 and 3.10.
- Only available for pipelines that use [default windows](/documentation/programming-guide/#single-global-window).
-- Only supports one-to-one transform mapping on a single element.
## Transforms {#transforms}
@@ -79,12 +76,8 @@ each column's data.
## I/O requirements {#io}
-- Input to the `MLTransform` class must be in one of the following formats:
- - A `dict` of `str`
- - Primitive types
- - List of primitive types
- - NumPy arrays
-- `MLTransform` outputs a Beam `Row` object with NumPy arrays.
+- Input to the `MLTransform` class must be a dictionary.
+- `MLTransform` outputs a Beam `Row` object with transformed elements.
- The output `PCollection` is a schema `PCollection`. The output schema
contains the transformed columns.
@@ -196,33 +189,4 @@ Replace the following values:
For more examples, see
[MLTransform for data processing](/documentation/transforms/python/elementwise/mltransform)
-in the [transform catalog](/documentation/transforms/python/overview/).
-
-### ScaleTo01 example {#scaleto01}
-
-This example demonstrates how to use `MLTransform` to normalize your data
-between 0 and 1 by using the minimum and maximum values from your entire
-dataset. `MLTransform` uses the `ScaleTo01` transformation.
-
-Use the following snippet to apply `ScaleTo01` on column `x` of the input
-data.
-
-```
-data_pcoll | MLTransform(write_artifact_location=<LOCATION>).with_transform(ScaleTo01(columns=['x']))
-```
-
-The `ScaleTo01` transformation produces two artifacts: the `min` and the `max`
-of the entire dataset. For more information, see the
-[Artifacts](#artifacts) section on this page.
-
-## Metrics {#metrics}
-
-When you use MLTransform, the following metrics are available.
-
-{{< table >}}
-| Metric | Description |
-| ------- | ---------------|
-| Data throughput | The number of records processed per second. This metric indicates the processing capacity of the pipeline for `beam.MLTransform.` |
-| Memory usage | The number of records processed per second. This metric indicates the processing capacity of the pipeline for `beam.MLTransform`. |
-| Counters | Tracks the number of elements processed. Each `MLTransform` has a counter. |:
-{{< /table >}}
\ No newline at end of file
+in the [transform catalog](/documentation/transforms/python/overview/).
\ No newline at end of file