You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2024/10/01 17:40:04 UTC

(beam) branch dependabot/go_modules/sdks/github.com/fsouza/fake-gcs-server-1.50.0 updated (312db7d0b37 -> 7af05d6335b)

This is an automated email from the ASF dual-hosted git repository.

github-bot pushed a change to branch dependabot/go_modules/sdks/github.com/fsouza/fake-gcs-server-1.50.0
in repository https://gitbox.apache.org/repos/asf/beam.git


 discard 312db7d0b37 Bump github.com/fsouza/fake-gcs-server from 1.49.2 to 1.50.0 in /sdks
     add 42efefd9892 [GCSIO] Fix internal unit test failure (#32518)
     add 878ccceb038 ReadFromKafkaDoFn: handle failed seek (#32456)
     add 65b29361eb1 Set np printoptions (#32539)
     add c3be9f096f0 Add RowFilter utility (#32366)
     add 97cb452e4b9 Vllm model handler (#32410)
     add cfcccbb4718 Bump github.com/aws/aws-sdk-go-v2/credentials in /sdks (#32542)
     add d312e959fa0 Bump com.pswidersk.terraform-plugin from 1.0.0 to 1.1.1 (#32541)
     add 2d5e72c615d Bump go.mongodb.org/mongo-driver from 1.16.1 to 1.17.0 in /sdks (#32497)
     add 5d8b0418cef Update build.gradle to use 3.12 for metrics report (#32547)
     add c503ec4fda5 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#32543)
     add a79e7103450 Set a UUID when building a Schema object. (#32399)
     add 72d46ce0caf Revert "Bump com.pswidersk.terraform-plugin from 1.0.0 to 1.1.1 (#32541)" (#32551)
     add d93f93abf0e Fix backlog reported by periodicsequence in python and go sdks to not include future outputs (#32513)
     add faf884a7a1b Update Python Inference Benchmark Test to Python 3.10 (#32550)
     add e3c6f4747be Add RowStringInterpolator utility  (#32367)
     add 01c7caf92af SolaceIO: deduplicate records in the integration test (#32557)
     add 380ed7b7f74 fix: add topic existing validation (#32465)
     add 80960d19b61 Remove unused code (#32414)
     add aabae2729c0 fix NPE and other exceptions thrown when work fails to process before work execution (#32566)
     add c7fb9a0d5da [Dataflow Streaming] Use separate heartbeat streams based on job settings (#32511)
     add 8e8c9143ca5 [Dataflow Streaming] Use isolated windmill streams based on job settings (#32503)
     add 9053dbf72e9 [website][Go SDK] Add Go testing advice to test your pipeline. (#32549)
     add 11318ae4211 Bump cloud.google.com/go/bigquery from 1.62.0 to 1.63.0 in /sdks (#32555)
     add 1eddbdca4c5 Tour of Beam markdown touchups (#32536)
     add 91c9be4e9ce Log warning if process function return None (#32545)
     add 7c565c38c3c Add info log between executing work failed and cleanup (#32570)
     add 160dffd88e5 Revert "fix: add topic existing validation (#32465)" (#32572)
     add c2c640f8c33 Update CHANGES for recent fixes, improvements. (#32568)
     add 7f849eb49a9 Add test case
     add 4661b14017f reverse ordering of test
     add 52b86c52a29 Update is_consistent_with
     add ad55edc3249 Update failing unit test to use str instead of int
     add 22ce9feddb9 update a couple more tests
     add 4309675e266 Merge pull request #32516 from hjtran/permissive_num_types
     add 271ea431156 BigQuery trim schema with selected fields (#32514)
     add ee604a5eb8a Improve BQ <-> Avro conversions (#32482)
     add 9b28982fe4b Update python KafkaIO docstring to add use_deprecated_read option
     add e705e28ecfd Merge pull request #32589 from jaehyeon-kim/master
     add ac01dccfd28 represenation->representation (#32588)
     add 48f836aae06 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#32595)
     add e640b2578fb Force BQIO to output elements in the correct row (#32584)
     add 9487927adce [#32562] Incorporate Prism into the Beam Website. (#32563)
     add 7da6c89592b [Go] Update Go version used by Beam repo to go1.23.1 (#32575)
     add 543d2ce97de Keep string FQN as a way to add lineage information (#32585)
     add 9b85f718cb8 [yaml] Preserve windowing for windowed input when using FileIO Java providers (#32586)
     add 40a5524d65d Update environments.py
     add 111f4c34ab2 Merge pull request #32602 from apache/tvalentyn-patch-2
     add deb4b4ada3a Managed Iceberg dynamic destinations (#32565)
     add aadb695348d Fix a bug in _get_function_body_without_inners for module sdks.python.transforms.core (#32591)
     add 301286fd6a8 default null array to empty array (#32604)
     add c8c674e1c04 Add support for dynamic write in `MqttIO` (#32470)
     add dcc0bd2fb22 Build release candidate with Java 11 (#32573)
     add 4d254a0863b Fixes a transform upgrade compatibility issue related to BigqueryIO
     add 5aed5850619 Merge pull request #32567: Fixes a transform upgrade compatibility issue related to BigqueryIO
     add 3dee026a2ab Update staticcheck version to fix breakage. (#32614)
     add 7af05d6335b Bump github.com/fsouza/fake-gcs-server from 1.49.2 to 1.50.0 in /sdks

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (312db7d0b37)
            \
             N -- N -- N   refs/heads/dependabot/go_modules/sdks/github.com/fsouza/fake-gcs-server-1.50.0 (7af05d6335b)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

No new revisions were added by this update.

Summary of changes:
 .../test-properties.json                           |   2 +-
 .../actions/setup-environment-action/action.yml    |   2 +-
 .github/autolabeler.yml                            |   7 +-
 .../arc/images/Dockerfile                          |   2 +-
 .../IO_Iceberg_Integration_Tests.json              |   2 +-
 .github/trigger_files/beam_PostCommit_Python.json  |   4 +-
 ...tCommit_Python_ValidatesContainer_Dataflow.json |   3 +-
 .github/workflows/README.md                        |   4 +-
 .../beam_Inference_Python_Benchmarks_Dataflow.yml  |  12 +-
 .github/workflows/build_release_candidate.yml      |  24 +-
 .github/workflows/go_tests.yml                     |   4 +-
 .test-infra/jenkins/build.gradle                   |   3 -
 CHANGES.md                                         |  24 +-
 README.md                                          |   1 +
 build.gradle.kts                                   |   1 +
 .../org/apache/beam/gradle/BeamModulePlugin.groovy |   2 +-
 dev-support/docker/Dockerfile                      |   2 +-
 .../overview-pipeline/description.md               |   2 +-
 .../runner-concepts/description.md                 |   6 +-
 .../introduction/introduction-guide/description.md |  35 +-
 release/src/main/Dockerfile                        |   4 +-
 .../options/DataflowStreamingPipelineOptions.java  |  10 +-
 .../dataflow/worker/StreamingDataflowWorker.java   |  62 ++-
 .../worker/StreamingModeExecutionContext.java      |  25 +-
 .../streaming/config/FakeGlobalConfigHandle.java   |  52 +++
 .../windmill/client/grpc/GrpcDispatcherClient.java |  66 ++-
 .../windmill/client/grpc/GrpcWindmillServer.java   |  16 +-
 .../grpc/stubs/WindmillStubFactoryFactory.java}    |  12 +-
 .../grpc/stubs/WindmillStubFactoryFactoryImpl.java |  54 +++
 .../work/processing/StreamingWorkScheduler.java    |   1 +
 .../work/refresh/StreamPoolHeartbeatSender.java    |  45 +-
 .../streaming/ComputationWorkExecutorTest.java     |  72 ++++
 .../FanOutStreamingEngineWorkerHarnessTest.java    |   9 +-
 .../client/grpc/GrpcDispatcherClientTest.java      | 154 +++++++
 .../client/grpc/GrpcWindmillServerTest.java        |  11 +-
 .../testing/FakeWindmillStubFactoryFactory.java    |  23 +-
 .../refresh/StreamPoolHeartbeatSenderTest.java     | 132 ++++++
 runners/prism/java/build.gradle                    |   5 -
 .../beam/runners/prism/PrismArtifactResolver.java  | 110 -----
 .../beam/runners/prism/PrismArtifactStager.java    | 173 --------
 .../apache/beam/runners/prism/PrismJobManager.java | 160 -------
 .../apache/beam/runners/prism/StateWatcher.java    | 146 -------
 .../apache/beam/runners/prism/WorkerService.java   | 116 ------
 .../runners/prism/PrismArtifactResolverTest.java   |  45 --
 .../runners/prism/PrismArtifactStagerTest.java     | 143 -------
 .../beam/runners/prism/PrismJobManagerTest.java    | 211 ----------
 .../beam/runners/prism/StateWatcherTest.java       | 136 ------
 .../beam/runners/prism/WorkerServiceTest.java      |  85 ----
 sdks/go.mod                                        |  48 +--
 sdks/go.sum                                        |  96 ++---
 sdks/go/pkg/beam/core/runtime/harness/harness.go   |   2 +-
 sdks/go/pkg/beam/transforms/periodic/periodic.go   |  13 +-
 .../pkg/beam/transforms/periodic/periodic_test.go  |  36 ++
 sdks/go/run_with_go_version.sh                     |   2 +-
 .../java/org/apache/beam/sdk/metrics/Lineage.java  |  10 +-
 .../java/org/apache/beam/sdk/schemas/Schema.java   |  75 ++--
 .../apache/beam/sdk/schemas/SchemaTranslation.java |  67 ++-
 .../beam/sdk/schemas/logicaltypes/OneOfType.java   |   3 +-
 .../java/org/apache/beam/sdk/util/RowFilter.java   | 421 +++++++++++++++++++
 .../beam/sdk/util/RowStringInterpolator.java       | 167 ++++++++
 .../beam/sdk/schemas/SchemaTranslationTest.java    |  46 +-
 .../sdk/schemas/logicaltypes/LogicalTypesTest.java |  10 +
 .../org/apache/beam/sdk/util/RowFilterTest.java    | 353 ++++++++++++++++
 .../beam/sdk/util/RowStringInterpolatorTest.java   | 205 +++++++++
 .../extensions/python/PythonExternalTransform.java |   3 -
 .../csv/providers/CsvWriteTransformProvider.java   |  17 +-
 .../io/gcp/bigquery/BeamRowToStorageApiProto.java  |   3 +
 .../sdk/io/gcp/bigquery/BigQueryAvroUtils.java     | 464 +++++++++------------
 .../beam/sdk/io/gcp/bigquery/BigQueryIO.java       |  33 +-
 .../sdk/io/gcp/bigquery/BigQueryIOTranslation.java |  32 +-
 .../io/gcp/bigquery/BigQueryQuerySourceDef.java    |   6 +-
 .../sdk/io/gcp/bigquery/BigQuerySourceBase.java    |   3 +-
 .../sdk/io/gcp/bigquery/BigQuerySourceDef.java     |   7 +-
 .../io/gcp/bigquery/BigQueryStorageSourceBase.java |  46 +-
 .../io/gcp/bigquery/BigQueryTableSourceDef.java    |   6 +-
 .../beam/sdk/io/gcp/bigquery/BigQueryUtils.java    | 103 ++++-
 .../gcp/bigquery/BeamRowToStorageApiProtoTest.java |  23 +-
 .../sdk/io/gcp/bigquery/BigQueryAvroUtilsTest.java | 213 +++++++---
 .../gcp/bigquery/BigQueryIOStorageQueryTest.java   |  12 +-
 .../io/gcp/bigquery/BigQueryIOStorageReadTest.java |  15 +-
 .../sdk/io/gcp/bigquery/BigQueryUtilsTest.java     |  25 ++
 ...QueryDirectReadSchemaTransformProviderTest.java |   3 +-
 .../beam/sdk/io/iceberg/AssignDestinations.java    |  30 +-
 .../beam/sdk/io/iceberg/DynamicDestinations.java   |  13 +-
 .../org/apache/beam/sdk/io/iceberg/IcebergIO.java  |  22 +-
 .../IcebergWriteSchemaTransformProvider.java       |  32 +-
 .../io/iceberg/OneTableDynamicDestinations.java    |  31 +-
 .../io/iceberg/PortableIcebergDestinations.java    |  81 ++++
 .../sdk/io/iceberg/WriteGroupedRowsToFiles.java    |  12 +-
 .../beam/sdk/io/iceberg/WriteToDestinations.java   |  43 +-
 .../sdk/io/iceberg/WriteUngroupedRowsToFiles.java  |  12 +-
 .../apache/beam/sdk/io/iceberg/IcebergIOIT.java    | 196 +++++++--
 .../beam/sdk/io/iceberg/IcebergIOWriteTest.java    |  43 +-
 .../IcebergWriteSchemaTransformProviderTest.java   | 208 ++++++++-
 .../json/providers/JsonWriteTransformProvider.java |  13 +-
 .../beam/sdk/io/kafka/ReadFromKafkaDoFn.java       |  33 +-
 .../beam/sdk/io/kafka/ReadFromKafkaDoFnTest.java   |  36 ++
 .../java/org/apache/beam/sdk/io/mqtt/MqttIO.java   | 139 ++++--
 .../org/apache/beam/sdk/io/mqtt/MqttIOTest.java    | 220 ++++++++++
 .../apache/beam/sdk/io/solace/it/SolaceIOIT.java   |   1 +
 .../ManagedSchemaTransformTranslationTest.java     |  49 ++-
 sdks/python/apache_beam/dataframe/doctests.py      |   5 +-
 .../apache_beam/examples/inference/README.md       |  80 ++++
 .../examples/inference/vllm_text_completion.py     | 162 +++++++
 sdks/python/apache_beam/io/gcp/bigquery.py         |  59 +--
 .../apache_beam/io/gcp/bigquery_write_it_test.py   |   1 +
 sdks/python/apache_beam/io/kafka.py                |  12 +
 .../ml/inference/test_resources/vllm.dockerfile    |  47 +++
 .../apache_beam/ml/inference/vllm_inference.py     | 312 ++++++++++++++
 .../python/apache_beam/options/pipeline_options.py |   5 +
 .../apache_beam/options/pipeline_options_test.py   |  47 ++-
 sdks/python/apache_beam/transforms/core.py         |  31 +-
 sdks/python/apache_beam/transforms/core_test.py    |  38 ++
 sdks/python/apache_beam/transforms/environments.py |   6 +-
 .../apache_beam/transforms/periodicsequence.py     |  21 +-
 .../transforms/periodicsequence_test.py            |  20 +
 .../apache_beam/transforms/ptransform_test.py      |   6 +-
 .../apache_beam/typehints/typed_pipeline_test.py   |  10 +-
 sdks/python/apache_beam/typehints/typehints.py     |   6 +
 .../python/apache_beam/typehints/typehints_test.py |  10 +-
 sdks/python/setup.py                               |   1 +
 .../tensorrt_runinference/tensor_rt.dockerfile     |   6 +-
 sdks/python/test-suites/dataflow/common.gradle     |  39 ++
 website/www/site/content/en/blog/beam-2.59.0.md    |   2 +-
 .../documentation/pipelines/test-your-pipeline.md  | 119 +++++-
 .../content/en/documentation/runners/direct.md     |   1 +
 .../site/content/en/documentation/runners/prism.md | 145 +++++++
 website/www/site/content/en/roadmap/_index.md      |   4 +-
 .../www/site/content/en/roadmap/prism-runner.md    |  35 ++
 .../www/site/data/en/documentation_runners.yaml    |   3 +
 .../layouts/partials/section-menu/en/roadmap.html  |   1 +
 .../layouts/partials/section-menu/en/runners.html  |   1 +
 132 files changed, 4919 insertions(+), 2318 deletions(-)
 create mode 100644 runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/streaming/config/FakeGlobalConfigHandle.java
 rename runners/{prism/java/src/main/java/org/apache/beam/runners/prism/StateListener.java => google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/stubs/WindmillStubFactoryFactory.java} (69%)
 create mode 100644 runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/stubs/WindmillStubFactoryFactoryImpl.java
 create mode 100644 runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/streaming/ComputationWorkExecutorTest.java
 create mode 100644 runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/client/grpc/GrpcDispatcherClientTest.java
 copy sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/DynamicDestinations.java => runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/testing/FakeWindmillStubFactoryFactory.java (54%)
 create mode 100644 runners/google-cloud-dataflow-java/worker/src/test/java/org/apache/beam/runners/dataflow/worker/windmill/work/refresh/StreamPoolHeartbeatSenderTest.java
 delete mode 100644 runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismArtifactResolver.java
 delete mode 100644 runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismArtifactStager.java
 delete mode 100644 runners/prism/java/src/main/java/org/apache/beam/runners/prism/PrismJobManager.java
 delete mode 100644 runners/prism/java/src/main/java/org/apache/beam/runners/prism/StateWatcher.java
 delete mode 100644 runners/prism/java/src/main/java/org/apache/beam/runners/prism/WorkerService.java
 delete mode 100644 runners/prism/java/src/test/java/org/apache/beam/runners/prism/PrismArtifactResolverTest.java
 delete mode 100644 runners/prism/java/src/test/java/org/apache/beam/runners/prism/PrismArtifactStagerTest.java
 delete mode 100644 runners/prism/java/src/test/java/org/apache/beam/runners/prism/PrismJobManagerTest.java
 delete mode 100644 runners/prism/java/src/test/java/org/apache/beam/runners/prism/StateWatcherTest.java
 delete mode 100644 runners/prism/java/src/test/java/org/apache/beam/runners/prism/WorkerServiceTest.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowFilter.java
 create mode 100644 sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowStringInterpolator.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/util/RowFilterTest.java
 create mode 100644 sdks/java/core/src/test/java/org/apache/beam/sdk/util/RowStringInterpolatorTest.java
 create mode 100644 sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/PortableIcebergDestinations.java
 create mode 100644 sdks/python/apache_beam/examples/inference/vllm_text_completion.py
 create mode 100644 sdks/python/apache_beam/ml/inference/test_resources/vllm.dockerfile
 create mode 100644 sdks/python/apache_beam/ml/inference/vllm_inference.py
 create mode 100644 website/www/site/content/en/documentation/runners/prism.md
 create mode 100644 website/www/site/content/en/roadmap/prism-runner.md