You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2023/07/28 04:22:38 UTC
[beam] branch nightly-refs/heads/master updated (dc1cfe54bfa -> 90f2b3445ff)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from dc1cfe54bfa Add a Dataflow Go postcommit suite on running on ARM machines. #27673
add 907c5110163 Argument fix on Build and Test step (#27684)
add d822fc9a1d3 Bump certifi from 2023.5.7 to 2023.7.22 in /sdks/python/container/py38 (#27654)
add 21e040c14e4 [AWS] Fix bug in KinesisIOReadTest (closes #27666) (#27686)
add e696414b887 Bump github.com/aws/aws-sdk-go-v2/config in /sdks (#27680)
add 8508e05b35e Add worfklow_dispatch triggers to all new jobs (#27688)
add aa6c8dc6fcd Add MLTransform to __init__ and change input types of transforms (#27687)
add 505d7c163dd Bump actions/upload-artifact from 2 to 3 (#27677)
add 42393baa989 Update Gradle Wrapper to 7.6.2
add c032a388608 Reconfigure Checkstyle plugin to remove absolute path inputs
add c754e585c48 Merge pull request #24764: Upgrade gradle to 7.6.2 and optimize build
add bb6a1730fff Bump certifi from 2023.5.7 to 2023.7.22 in /sdks/python/container/py310 (#27656)
add f174f2aece6 Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#27683)
add 85078169fd0 Bump cloud.google.com/go/bigquery from 1.52.0 to 1.53.0 in /sdks (#27652)
add b788d09bb37 Add NLP TensorFlow streaming use case (#27659)
add f35a6365e8a explicitly use >=41.0.2 for cryptography (#27608)
add d241eaeeb94 Make all Github Action tests to run at minute 0 of every 6 hours (#27701)
add 2e958499e5d lowercase everything
add 1f4defab42a Merge pull request #27699: Don't make StorageAPI tablerow names case-sensitive
add d3ea23250b5 Enable FirestoreV1IT to run against a non default host (#27256)
add 4d712ebfde2 [Documentation] Specify version of HCLS API in Documentation (#27672)
add cf438d3141c Fetch buffer and add it to pcoll_buffer to avoid element duplication when buffer is None (#27676)
add 2aabc264cf1 Bump word-wrap (#27557)
add 1c53cc7c1fb Add trigger phrases to workflow names (#27669)
add 27424a403a2 Change how we interrupt in progress runs (#27671)
add af5e43defdd Add instructions on adding new Actions workflows (#27689)
add f50d2098ef6 Bump google.golang.org/api from 0.133.0 to 0.134.0 in /sdks (#27705)
add dc321f97fa8 Bump certifi from 2023.5.7 to 2023.7.22 in /sdks/python/container/py39 (#27657)
add 41790473fcd Bump cloud.google.com/go/datastore from 1.12.1 to 1.13.0 in /sdks (#27704)
add bbd49b8891b Add It_Framework PreCommit (#27710)
add 798e0f88aaf Add CassandraResourceManager for managing cassandra resources in tests. (#27664)
add fa685b0fbf6 moved --dataflowEndpoint to public pipeline option set
add 06ff272bb72 minor linting
add 03da178ecb5 imported Dataflow to DataflowPipelineOptions
add 45781651cb5 added Override decorators
add c6296ba3a17 Merge pull request #27693: Make --dataflowEndpoint a public option
add 8d4b0bd86d4 [prism] move test functions and tests to internal_test (#27714)
add 5a54ee6ddd8 [#27558] Update required Go version to 1.20. (#27559)
add c3f16750192 Register create functions to pass on prism. (#27713)
add 66f0592f8b4 Add flags to build multiarch Go SDK containers and fix Go Dataflow ARM github actions test (#27716)
add 5989a662dc9 populate name map during snapshot restore
add 90f2b3445ff Merge pull request #27638: Populate name map during snapshot restore
No new revisions were added by this update.
Summary of changes:
.../test-properties.json | 2 +-
.github/workflows/README.md | 90 +-
.../workflows/beam_PostCommit_Go_Dataflow_ARM.yml | 12 +-
.../workflows/beam_PreCommit_CommunityMetrics.yml | 9 +-
.github/workflows/beam_PreCommit_Go.yml | 8 +-
.github/workflows/beam_PreCommit_ItFramework.yml | 103 ++
.../beam_PreCommit_Java_Examples_Dataflow.yml | 7 +-
.github/workflows/beam_PreCommit_PythonDocs.yml | 11 +-
.../beam_PreCommit_Python_Integration.yml | 7 +-
.github/workflows/beam_PreCommit_RAT.yml | 11 +-
.github/workflows/beam_PreCommit_SQL_Java11.yml | 11 +-
.github/workflows/beam_PreCommit_Typescript.yml | 7 +-
.github/workflows/beam_PreCommit_Website.yml | 8 +-
.github/workflows/beam_PreCommit_Whitespace.yml | 8 +-
.github/workflows/build_wheels.yml | 2 +-
.github/workflows/dask_runner_tests.yml | 2 +-
.github/workflows/go_tests.yml | 2 +-
.github/workflows/java_tests.yml | 2 +-
.github/workflows/job_PreCommit_Python.yml | 7 +-
.../job_PreCommit_PythonAutoformatter.yml | 7 +-
.github/workflows/job_PreCommit_PythonLint.yml | 7 +-
.../workflows/job_PreCommit_Python_Coverage.yml | 7 +-
.../workflows/job_PreCommit_Python_Dataframes.yml | 7 +-
.../workflows/job_PreCommit_Python_DockerBuild.yml | 7 +-
.../workflows/job_PreCommit_Python_Examples.yml | 7 +-
.github/workflows/job_PreCommit_Python_Runners.yml | 6 +-
.../workflows/job_PreCommit_Python_Transforms.yml | 6 +-
.github/workflows/local_env_tests.yml | 2 +-
.github/workflows/playground_frontend_test.yml | 2 +-
.github/workflows/python_dependency_tests.yml | 2 +-
.github/workflows/python_tests.yml | 2 +-
.github/workflows/tour_of_beam_backend.yml | 2 +-
.../workflows/tour_of_beam_backend_integration.yml | 2 +-
.github/workflows/tour_of_beam_frontend_test.yml | 2 +-
.github/workflows/typescript_tests.yml | 2 +-
CHANGES.md | 3 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 17 +-
dev-support/docker/Dockerfile | 2 +-
.../beam-ml/nlp_tensorflow_streaming.ipynb | 1110 ++++++++++++++++++++
gradle/wrapper/gradle-wrapper.jar | Bin 59536 -> 61624 bytes
gradle/wrapper/gradle-wrapper.properties | 3 +-
gradlew | 18 +-
gradlew.bat | 15 +-
.../extensions/arrow => it/cassandra}/build.gradle | 24 +-
.../it/cassandra/CassandraResourceManager.java | 313 ++++++
.../CassandraResourceManagerException.java} | 12 +-
.../cassandra/CassandraResourceManagerUtils.java | 52 +
.../it/cassandra/matchers/CassandraAsserts.java} | 42 +-
.../beam/it/cassandra}/matchers/package-info.java | 5 +-
.../apache/beam/it/cassandra}/package-info.java | 4 +-
.../it/cassandra/CassandraResourceManagerIT.java | 73 ++
.../it/cassandra/CassandraResourceManagerTest.java | 121 +++
.../it/gcp/artifacts/utils/ParquetTestUtil.java | 3 +-
.../beam/runners/dataflow/DataflowRunner.java | 1 +
.../dataflow/options/DataflowPipelineOptions.java | 20 +
sdks/go.mod | 18 +-
sdks/go.sum | 34 +-
sdks/go/container/build.gradle | 7 +-
sdks/go/pkg/beam/create.go | 6 +
sdks/go/pkg/beam/create_test.go | 9 +
.../beam/runners/prism/internal/execute_test.go | 5 +-
.../beam/runners/prism/internal/separate_test.go | 2 +-
.../pkg/beam/runners/prism/internal/testdofns.go | 362 -------
.../beam/runners/prism/internal/testdofns_test.go | 332 +++++-
.../runners/prism/internal/unimplemented_test.go | 2 +-
sdks/go/run_with_go_version.sh | 2 +-
sdks/go/test/run_validatesrunner_tests.sh | 10 +-
.../resources/beam/{ => checkstyle}/checkstyle.xml | 4 +-
.../beam/{ => checkstyle}/suppressions.xml | 0
.../sdk/io/aws2/kinesis/KinesisIOReadTest.java | 9 +-
sdks/java/io/google-cloud-platform/build.gradle | 4 +
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 6 +-
.../sdk/io/gcp/firestore/FirestoreOptions.java | 18 +
.../FirestoreStatefulComponentFactory.java | 2 +-
.../apache/beam/sdk/io/gcp/healthcare/DicomIO.java | 2 +-
.../apache/beam/sdk/io/gcp/healthcare/FhirIO.java | 6 +-
.../io/gcp/healthcare/FhirIOPatientEverything.java | 2 +-
.../apache/beam/sdk/io/gcp/healthcare/HL7v2IO.java | 10 +-
.../sdk/io/gcp/healthcare/HealthcareApiClient.java | 4 +-
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 2 +-
.../sdk/io/gcp/firestore/it/BaseFirestoreIT.java | 26 +-
.../gcp/firestore/it/FirestoreTestingHelper.java | 10 +-
.../sdk/io/gcp/firestore/it/FirestoreV1IT.java | 3 +-
sdks/python/apache_beam/ml/__init__.py | 2 +
sdks/python/apache_beam/ml/transforms/tft.py | 11 +-
.../apache-beam-jupyterlab-sidepanel/yarn.lock | 6 +-
.../runners/portability/fn_api_runner/fn_runner.py | 10 +-
.../portability/fn_api_runner/fn_runner_test.py | 9 +
.../container/py310/base_image_requirements.txt | 2 +-
.../container/py38/base_image_requirements.txt | 2 +-
.../container/py39/base_image_requirements.txt | 2 +-
sdks/python/setup.py | 2 +-
settings.gradle.kts | 1 +
.../content/en/documentation/programming-guide.md | 2 +-
.../site/content/en/get-started/quickstart-go.md | 2 +-
95 files changed, 2582 insertions(+), 583 deletions(-)
create mode 100644 .github/workflows/beam_PreCommit_ItFramework.yml
create mode 100644 examples/notebooks/beam-ml/nlp_tensorflow_streaming.ipynb
copy {sdks/java/extensions/arrow => it/cassandra}/build.gradle (61%)
create mode 100644 it/cassandra/src/main/java/org/apache/beam/it/cassandra/CassandraResourceManager.java
copy it/{google-cloud-platform/src/main/java/org/apache/beam/it/gcp/datastore/DatastoreResourceManagerException.java => cassandra/src/main/java/org/apache/beam/it/cassandra/CassandraResourceManagerException.java} (73%)
create mode 100644 it/cassandra/src/main/java/org/apache/beam/it/cassandra/CassandraResourceManagerUtils.java
copy it/{google-cloud-platform/src/main/java/org/apache/beam/it/gcp/datastore/matchers/DatastoreAsserts.java => cassandra/src/main/java/org/apache/beam/it/cassandra/matchers/CassandraAsserts.java} (53%)
copy it/{google-cloud-platform/src/main/java/org/apache/beam/it/gcp/datastore => cassandra/src/main/java/org/apache/beam/it/cassandra}/matchers/package-info.java (87%)
copy it/{google-cloud-platform/src/main/java/org/apache/beam/it/gcp/datastore => cassandra/src/main/java/org/apache/beam/it/cassandra}/package-info.java (88%)
create mode 100644 it/cassandra/src/test/java/org/apache/beam/it/cassandra/CassandraResourceManagerIT.java
create mode 100644 it/cassandra/src/test/java/org/apache/beam/it/cassandra/CassandraResourceManagerTest.java
delete mode 100644 sdks/go/pkg/beam/runners/prism/internal/testdofns.go
rename sdks/java/build-tools/src/main/resources/beam/{ => checkstyle}/checkstyle.xml (99%)
rename sdks/java/build-tools/src/main/resources/beam/{ => checkstyle}/suppressions.xml (100%)