You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/11/04 15:49:12 UTC
[beam] branch dependabot/go_modules/sdks/google.golang.org/api-0.102.0 updated (1ebaacdfb9b -> 3d9f20c2e07)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/go_modules/sdks/google.golang.org/api-0.102.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 1ebaacdfb9b Bump google.golang.org/api from 0.101.0 to 0.102.0 in /sdks
add 8d71aae2c61 [Spark dataset runner] Add direct translation of Combine.GroupedValues (related to #23845)
add b7110268ab5 Merge pull request #23846: [Spark dataset runner] Add direct translation of Combine.GroupedValues
add 85b6b643bff Concept guide on orchestrating Beam preprocessing (#23094)
add 05c89ed2fd6 Initial draft of Batched DoFn user guide (#23909)
add 279b2755744 WIP: Dataframe API ML preprocessing notebook (#22587)
add 15297da74bc emit load job IDs as soon as they come up
add 9c3ce5debec style fix
add 4f64c7e547c Merge pull request #23954: Emit job ids via side output in TriggerFileLoads process to keep beam.Flatten() happy for Spark and Flink runners
add ec6da53ba23 [Python] Added none check while accessing active_process_bundle (#23947)
add 77116c5c814 [Tour Of Beam] saving user code (#23938)
add e2463a4c965 Disable flaky fn_api_runner tests (#23971)
add 7da182a8d1d Make BatchConverter inference errors more helpful (#23965)
add 1563b9721b4 Improve Iterator error message (#23972)
add 30b26171b40 Update watermark during periodic sequence/impulse (#23507)
add 16852518861 TFX image classification example (#23456)
add 7dba78d6110 Immediately truncate full restriction on drain of periodic impulse (#23765)
add 058c56af735 [Task]: PR Bot will push commits only if they are non-empty (#23937)
add 97628802fe4 Bump cloud.google.com/go/datastore from 1.8.0 to 1.9.0 in /sdks (#23916)
add 3d9f20c2e07 Bump google.golang.org/api from 0.101.0 to 0.102.0 in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (1ebaacdfb9b)
\
N -- N -- N refs/heads/dependabot/go_modules/sdks/google.golang.org/api-0.102.0 (3d9f20c2e07)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../workflows/tour_of_beam_backend_integration.yml | 18 +-
build.gradle.kts | 10 +-
.../beam-ml/dataframe_api_preprocessing.ipynb | 3496 +++++++++++++++++++
examples/notebooks/tour-of-beam/dataframes.ipynb | 24 +-
learning/tour-of-beam/backend/README.md | 8 +
learning/tour-of-beam/backend/docker-compose.yml | 13 +
learning/tour-of-beam/backend/function.go | 62 +-
learning/tour-of-beam/backend/go.mod | 2 +
.../tour-of-beam/backend/integration_tests/api.go | 10 +
.../{auth_test.go => auth_emulator.go} | 54 +-
.../backend/integration_tests/auth_test.go | 156 +-
.../backend/integration_tests/client.go | 16 +-
.../backend/integration_tests/function_test.go | 10 +-
.../backend/integration_tests/local.sh | 4 +
learning/tour-of-beam/backend/internal/entity.go | 15 +-
.../backend/internal/service/content.go | 16 +-
.../{storage/iface.go => service/pg_adapter.go} | 33 +-
.../backend/internal/storage/adapter.go | 5 +-
.../backend/internal/storage/datastore.go | 38 +-
.../tour-of-beam/backend/internal/storage/iface.go | 1 +
.../tour-of-beam/backend/internal/storage/mock.go | 4 +
.../backend/internal/storage/schema.go | 1 +
.../tour-of-beam/backend/playground_api/api.pb.go | 3507 ++++++++++++++++++++
.../backend/playground_api/api_grpc.pb.go | 791 +++++
.../storage/iface.go => playground_api/helper.go} | 29 +-
.../tour-of-beam/backend/playground_api/mock.go | 1077 ++++++
.../backend/samples/api/get_user_progress.json | 7 +-
.../translation/TransformTranslator.java | 11 +
.../batch/CombineGroupedValuesTranslatorBatch.java | 75 +
.../translation/batch/PipelineTranslatorBatch.java | 6 +-
.../batch/CombineGroupedValuesTest.java | 70 +
scripts/ci/pr-bot/shared/persistentState.ts | 4 +-
sdks/go.mod | 2 +-
sdks/go.sum | 12 +-
.../examples/inference/tfx_bsl}/__init__.py | 0
.../inference/tfx_bsl/build_tensorflow_model.py | 147 +
.../examples/inference/tfx_bsl/requirements.txt | 26 +-
.../tfx_bsl/tensorflow_image_classification.py | 194 ++
.../inference/tfx_bsl/tfx_bsl_inference_it_test.py | 99 +
.../examples/ml-orchestration/README.md | 22 +
.../kfp/components/ingestion/Dockerfile | 29 +-
.../kfp/components/ingestion/component.yaml | 41 +-
.../kfp/components/ingestion/requirements.txt | 23 +-
.../kfp/components/ingestion/src/ingest.py | 74 +
.../kfp/components/preprocessing/Dockerfile | 29 +-
.../kfp/components/preprocessing/component.yaml | 64 +
.../kfp/components/preprocessing/requirements.txt | 26 +-
.../kfp/components/preprocessing/src/preprocess.py | 208 ++
.../kfp/components/train/Dockerfile | 27 +-
.../kfp/components/train/component.yaml | 46 +-
.../kfp/components/train/requirements.txt | 23 +-
.../kfp/components/train/src/train.py | 83 +
.../examples/ml-orchestration/kfp/pipeline.json | 247 ++
.../examples/ml-orchestration/kfp/pipeline.py | 132 +
.../examples/ml-orchestration/kfp/requirements.txt | 23 +-
.../ml-orchestration/tfx/coco_captions_local.py | 141 +
.../ml-orchestration/tfx/coco_captions_utils.py | 87 +
.../examples/ml-orchestration/tfx/requirements.txt | 22 +-
.../apache_beam/io/gcp/bigquery_file_loads.py | 43 +-
.../portability/fn_api_runner/fn_runner_test.py | 2 +
.../apache_beam/runners/worker/worker_status.py | 2 +-
.../apache_beam/transforms/batch_dofn_test.py | 49 +-
sdks/python/apache_beam/transforms/core.py | 31 +-
.../apache_beam/transforms/periodicsequence.py | 13 +-
.../transforms/periodicsequence_it_test.py | 89 +
sdks/python/pytest.ini | 1 +
sdks/python/test-suites/direct/common.gradle | 31 +-
.../content/en/documentation/ml/data-processing.md | 2 +
.../content/en/documentation/ml/orchestration.md | 223 ++
.../site/content/en/documentation/ml/overview.md | 3 +-
.../content/en/documentation/programming-guide.md | 235 ++
.../partials/section-menu/en/documentation.html | 2 +
.../static/images/orchestrated-beam-pipeline.svg | 35 +
.../static/images/standalone-beam-pipeline.svg | 35 +
74 files changed, 11693 insertions(+), 503 deletions(-)
create mode 100644 examples/notebooks/beam-ml/dataframe_api_preprocessing.ipynb
copy learning/tour-of-beam/backend/integration_tests/{auth_test.go => auth_emulator.go} (73%)
copy learning/tour-of-beam/backend/internal/{storage/iface.go => service/pg_adapter.go} (56%)
create mode 100644 learning/tour-of-beam/backend/playground_api/api.pb.go
create mode 100644 learning/tour-of-beam/backend/playground_api/api_grpc.pb.go
copy learning/tour-of-beam/backend/{internal/storage/iface.go => playground_api/helper.go} (53%)
create mode 100644 learning/tour-of-beam/backend/playground_api/mock.go
create mode 100644 runners/spark/3/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineGroupedValuesTranslatorBatch.java
create mode 100644 runners/spark/3/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineGroupedValuesTest.java
copy {.test-infra/jenkins/dependency_check => sdks/python/apache_beam/examples/inference/tfx_bsl}/__init__.py (100%)
create mode 100644 sdks/python/apache_beam/examples/inference/tfx_bsl/build_tensorflow_model.py
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/inference/tfx_bsl/requirements.txt (59%)
create mode 100644 sdks/python/apache_beam/examples/inference/tfx_bsl/tensorflow_image_classification.py
create mode 100644 sdks/python/apache_beam/examples/inference/tfx_bsl/tfx_bsl_inference_it_test.py
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/README.md
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/ingestion/Dockerfile (62%)
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/ingestion/component.yaml (56%)
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/ingestion/requirements.txt (59%)
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/kfp/components/ingestion/src/ingest.py
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/preprocessing/Dockerfile (62%)
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/kfp/components/preprocessing/component.yaml
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/preprocessing/requirements.txt (62%)
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/kfp/components/preprocessing/src/preprocess.py
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/Dockerfile (62%)
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/component.yaml (52%)
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/requirements.txt (62%)
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/kfp/components/train/src/train.py
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/kfp/pipeline.json
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/kfp/pipeline.py
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/kfp/requirements.txt (62%)
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/tfx/coco_captions_local.py
create mode 100644 sdks/python/apache_beam/examples/ml-orchestration/tfx/coco_captions_utils.py
copy learning/tour-of-beam/backend/docker-compose.yml => sdks/python/apache_beam/examples/ml-orchestration/tfx/requirements.txt (62%)
create mode 100644 sdks/python/apache_beam/transforms/periodicsequence_it_test.py
create mode 100644 website/www/site/content/en/documentation/ml/orchestration.md
mode change 100755 => 100644 website/www/site/content/en/documentation/ml/overview.md
create mode 100644 website/www/site/static/images/orchestrated-beam-pipeline.svg
create mode 100644 website/www/site/static/images/standalone-beam-pipeline.svg