You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by tv...@apache.org on 2026/10/16 19:17:13 UTC
(beam) branch master updated: Clarify known restrictions for model updates. (#40600)
This is an automated email from the ASF dual-hosted git repository.
tvalentyn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new a17e73a0da4 Clarify known restrictions for model updates. (#40600)
a17e73a0da4 is described below
commit a17e73a0da4710a1fee926ccd2b7abac2ea2ec08
Author: tvalentyn <tv...@users.noreply.github.com>
AuthorDate: Fri Oct 16 12:17:05 2026 -0700
Clarify known restrictions for model updates. (#40600)
---
sdks/python/apache_beam/ml/inference/utils.py | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/sdks/python/apache_beam/ml/inference/utils.py b/sdks/python/apache_beam/ml/inference/utils.py
index dfbb01a2904..8cbe836e22d 100644
--- a/sdks/python/apache_beam/ml/inference/utils.py
+++ b/sdks/python/apache_beam/ml/inference/utils.py
@@ -133,14 +133,17 @@ class WatchFilePattern(beam.PTransform):
1. Any previously used filenames cannot be reused. If a file is added
or updated to a previously used filename, this transform will ignore
that update. To trigger a model update, always upload a file with
- unique name.
- 2. Initially, before the pipeline startup time, WatchFilePattern expects
+ unique name. Do not preserve old modification timestamps when copying
+ or uploading a file (for example, when rolling back to an older model),
+ as files with timestamps older than the pipeline start time are ignored.
+ 2. Do not upload more than one model update per polling ``interval``, as
+ matching multiple new files in the same poll can cause undefined
+ ordering or singleton side-input errors.
+ 3. Initially, before the pipeline startup time, WatchFilePattern expects
at least one file present that matches the file_pattern.
- 3. This transform is supported in streaming mode since
+ 4. This transform is supported in streaming mode since
MatchContinuously produces an unbounded source. Running in batch
mode can lead to undesired results or result in pipeline being stuck.
-
-
"""
self.file_pattern = file_pattern
self.interval = interval