You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/09/15 05:13:22 UTC
[beam] branch nightly-refs/heads/master updated (d9405959199 -> 26c94b20f74)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from d9405959199 Don't depend on java 11 docker container for go test (#23197)
add d635d44ad3d Properly close Spark (streaming) context if Pipeline translation fails (fixes #23203)
add bee135e12e9 Merge pull request #23204: Properly close Spark (streaming) context if Pipeline translation fails
add 111878505e6 Annotate stateful VR test in TestStreamTest with UsesStatefulParDo (related to #22472)
add ebacef90771 Merge pull request #23202: Annotate stateful VR test in TestStreamTest with UsesStatefulParDo (related to #22472)
add c654e41cb40 [Playground] [Backend] Datastore queries and mappers to get precompiled objects (#22868)
add 66d1af8beef Open Allow and test pyarrow 8.x and 9.x (#22997)
add 634dbfd470e (BQ Python) Pass project field from options or parameter when writing with dynamic destinations (#23011)
add e2e7265356a Update python-machine-learning.md (#23209)
add 2a9327dd433 Pin the version of cloudpickle to 2.1.x (#23120)
add 6f25fba3fea Add streaming test for Write API sink (#21903)
add abdaa6bd69c [Go SDK] Proto changes for timer param (#23216)
add 6d50a333beb Bump github.com/testcontainers/testcontainers-go in /sdks (#23201)
add aecbc7cb45e Update to objsize to 0.5.2 which is under BSD-3 license (fixes #23096) (#23211)
add adff92bf472 Exclude insignificant whitespace from cloud object (#23217)
add 9be9a43c1b8 Trying out property-based tests for Beam python coders (#22233)
add 9d9db56ca63 Publish results of JMH benchmark runs (Java SDK) to InfluxDB (part of #22238). (#23041)
add 70f1c715259 Exclude protobuf 3.20.2 (#23226)
add 66bbee84ed4 Fix IllegalStateException in StorageApiWriteUnshardedRecords error handling. (#23205)
add 2eacbcf13d5 Fix assignees check
add 935eeadd45a Merge pull request #23229: Fix assignees check in alert email
add 60c4fae7302 Update cibuildwheel (#23024)
add 2d4f61c9325 Add section to docs on resource hints/RunInference (#23215)
add ac37784821e (BQ Python) Perform job waits in finish_bundle to allow BQ streaming writes with large batch loads (#23012)
add 9b83b79088d Update to newest version of CloudPickle. (#23223)
add 008fe94cbcf Resolve script parsing error when changing from bash to sh. (#23199)
add 2791d7b383e Bump cloud.google.com/go/bigquery from 1.39.0 to 1.40.0 in /sdks (#23122)
add d9ccb7c8d05 Bump github.com/google/go-cmp from 0.5.8 to 0.5.9 in /sdks (#23123)
add bb7468ff136 Update google-cloud-bigquery requirement from <3,>=1.6.0 to >=1.6.0,<4 in /sdks/python (#22378)
add 26c94b20f74 Optimize varint reading and writing for small ints. (#23192)
No new revisions were added by this update.
Summary of changes:
.github/workflows/build_wheels.yml | 6 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 13 +-
.../backend/internal/db/datastore/datastore_db.go | 210 +++++++++++
.../internal/db/datastore/datastore_db_test.go | 369 ++++++++++++++++++-
playground/backend/internal/db/db.go | 12 +
.../internal/db/entity/{id_meta.go => common.go} | 4 +
playground/backend/internal/db/mapper/mapper.go | 2 +
.../db/mapper/precompiled_object_mapper.go | 32 ++
.../db/mapper/precompiled_object_mapper_test.go | 95 +++++
.../db/schema/migration/migration_v001_test.go | 1 +
release/src/main/scripts/cut_release_branch.sh | 2 +-
release/src/main/scripts/verify_release_build.sh | 2 +-
.../util/SchemaCoderCloudObjectTranslator.java | 4 +-
.../SparkRunnerStreamingContextFactory.java | 24 +-
scripts/ci/issue-report/generateReport.js | 8 +-
sdks/go.mod | 41 ++-
sdks/go.sum | 390 +++++++++++++++++++--
sdks/go/pkg/beam/core/runtime/graphx/v1/v1.pb.go | 296 ++++++++--------
sdks/go/pkg/beam/core/runtime/graphx/v1/v1.proto | 2 +
.../apache/beam/sdk/testing/TestStreamTest.java | 7 +-
.../bigquery/StorageApiWriteUnshardedRecords.java | 11 +-
.../io/gcp/bigquery/BigQueryIOStorageWriteIT.java | 55 ++-
sdks/java/testing/test-utils/build.gradle | 1 +
.../org/apache/beam/sdk/testutils/jmh/Main.java | 190 ++++++++++
.../beam/sdk/testutils/jmh}/package-info.java | 4 +-
.../testutils/publishing/InfluxDBPublisher.java | 11 +
.../coders/coders_property_based_test.py | 156 +++++++++
sdks/python/apache_beam/coders/stream.pyx | 24 +-
sdks/python/apache_beam/io/gcp/bigquery.py | 1 +
.../apache_beam/io/gcp/bigquery_file_loads.py | 178 +++++-----
.../apache_beam/io/gcp/bigquery_file_loads_test.py | 166 +++++++--
.../testing/load_tests/load_test_metrics_utils.py | 6 +-
sdks/python/build-requirements.txt | 1 +
.../container/py37/base_image_requirements.txt | 4 +-
.../container/py38/base_image_requirements.txt | 4 +-
.../container/py39/base_image_requirements.txt | 4 +-
sdks/python/setup.py | 15 +-
sdks/python/test-suites/tox/py38/build.gradle | 8 +
sdks/python/tox.ini | 4 +-
.../documentation/sdks/python-machine-learning.md | 24 +-
40 files changed, 1987 insertions(+), 400 deletions(-)
rename playground/backend/internal/db/entity/{id_meta.go => common.go} (92%)
create mode 100644 sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/jmh/Main.java
copy {examples/java/src/main/java/org/apache/beam/examples/complete/kafkatopubsub/options => sdks/java/testing/test-utils/src/main/java/org/apache/beam/sdk/testutils/jmh}/package-info.java (88%)
create mode 100644 sdks/python/apache_beam/coders/coders_property_based_test.py