You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2022/12/02 04:34:39 UTC
[beam] branch nightly-refs/heads/master updated (8349ee6c12a -> 0dd083f81e4)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch nightly-refs/heads/master
in repository https://gitbox.apache.org/repos/asf/beam.git
from 8349ee6c12a Fix BlobstorageIO.checksum Attribute Error (#24442)
add 61f966740e9 Bump github.com/tetratelabs/wazero in /sdks (#24453)
add 57e5b69f45b [BEAM-12164] Support querying against Postgres for the SpannerIO change streams connector (#24390)
add 5af27337427 Bump cloud.google.com/go/bigquery from 1.43.0 to 1.44.0 in /sdks (#24451)
add 2e71061e7b9 ML notebook formatting and text updates (#24437)
add 263a7d8f432 lint fixes (#24455)
add 9b917fae3ca Bump cloud.google.com/go/pubsub from 1.26.0 to 1.27.0 in /sdks (#24450)
add d93a0d93b73 Install venv dependencies in local env setup (#24461)
add b08d72c1762 Don't set BigQuery services in schema transform configuration (#24316)
add ec2a07b38c1 Apache playground blog (#24431)
add 4718cdff87f Sort SchemaTransform configuration schema fields by name to establish consistency (#24344)
add 860d9af6979 Rename from default-pool to pool-1 (#24466)
add 6c807b3794d Moving to 2.45.0-SNAPSHOT on master branch.
add e9ed5fd0ae8 [Website] change svg to png (#24268)
add 14d3dc87ea2 Fix error messages in cred rotation email (#24474)
add f378533030c Add integer to NUMERIC and BIGNUMERIC conversion support (#24447)
add f9e21a68e8c Update jackson dep. (#24445)
add 03052670499 Reduce calls to FileSystem.match and API calls in FileSystem._list (#24317)
add b9f2055897b Capture full response context to provide complete error information (#24416)
add 0dd083f81e4 Pandas 1.5 support (#23973)
No new revisions were added by this update.
Summary of changes:
.github/workflows/local_env_tests.yml | 9 +-
.../job_IODatastoresCredentialsRotation.groovy | 4 +-
.../jenkins/job_MetricsCredentialsRotation.groovy | 2 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 4 +-
.../beam-ml/custom_remote_inference.ipynb | 50 +-
.../beam-ml/dataframe_api_preprocessing.ipynb | 82 ++--
.../notebooks/beam-ml/run_custom_inference.ipynb | 17 +-
.../beam-ml/run_inference_multi_model.ipynb | 74 +--
.../notebooks/beam-ml/run_inference_pytorch.ipynb | 32 +-
.../run_inference_pytorch_tensorflow_sklearn.ipynb | 57 +--
.../notebooks/beam-ml/run_inference_sklearn.ipynb | 30 +-
.../beam-ml/run_inference_tensorflow.ipynb | 42 +-
gradle.properties | 4 +-
local-env-setup.sh | 4 +
sdks/go.mod | 10 +-
sdks/go.sum | 26 +-
sdks/go/pkg/beam/core/core.go | 2 +-
sdks/go/pkg/beam/core/runtime/exec/window.go | 7 +
.../container/license_scripts/dep_urls_java.yaml | 2 +-
.../java/org/apache/beam/sdk/schemas/Schema.java | 15 +
.../transforms/TypedSchemaTransformProvider.java | 3 +-
.../org/apache/beam/sdk/schemas/SchemaTest.java | 82 ++++
.../bigquery/StorageApiWriteUnshardedRecords.java | 19 +-
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 6 +
.../BigQueryDirectReadSchemaTransformProvider.java | 15 +-
.../apache/beam/sdk/io/gcp/spanner/SpannerIO.java | 36 +-
.../gcp/spanner/changestreams/NameGenerator.java | 21 +-
.../action/QueryChangeStreamAction.java | 2 +-
.../spanner/changestreams/dao/ChangeStreamDao.java | 73 ++-
.../changestreams/dao/ChangeStreamResultSet.java | 18 +
.../gcp/spanner/changestreams/dao/DaoFactory.java | 22 +-
.../dao/PartitionMetadataAdminDao.java | 119 +++--
.../changestreams/dao/PartitionMetadataDao.java | 228 ++++++---
.../mapper/ChangeStreamRecordMapper.java | 283 +++++++++--
.../changestreams/mapper/MapperFactory.java | 9 +-
.../bigquery/TableRowToStorageApiProtoTest.java | 72 ++-
...QueryDirectReadSchemaTransformProviderTest.java | 8 +-
.../spanner/changestreams/NameGeneratorTest.java | 7 +-
.../SpannerChangeStreamErrorTest.java | 128 +++--
.../action/QueryChangeStreamActionTest.java | 8 +-
.../dao/PartitionMetadataAdminDaoTest.java | 29 +-
.../dao/PartitionMetadataDaoTest.java | 7 +-
.../changestreams/it/IntegrationTestEnv.java | 176 +++++--
...mIT.java => SpannerChangeStreamPostgresIT.java} | 106 +----
.../mapper/ChangeStreamRecordMapperTest.java | 520 +++++++++++++++++----
.../spanner/changestreams/util/TestJsonMapper.java | 246 ++++++++++
sdks/python/apache_beam/dataframe/frames.py | 42 +-
.../apache_beam/dataframe/pandas_doctests_test.py | 16 +-
sdks/python/apache_beam/io/aws/s3filesystem.py | 4 +-
.../python/apache_beam/io/aws/s3filesystem_test.py | 30 +-
sdks/python/apache_beam/io/aws/s3io.py | 58 ++-
.../apache_beam/io/azure/blobstoragefilesystem.py | 4 +-
.../io/azure/blobstoragefilesystem_test.py | 32 +-
sdks/python/apache_beam/io/azure/blobstorageio.py | 49 +-
sdks/python/apache_beam/io/filebasedsource.py | 5 +-
sdks/python/apache_beam/io/gcp/gcsfilesystem.py | 4 +-
.../apache_beam/io/gcp/gcsfilesystem_test.py | 29 +-
sdks/python/apache_beam/io/gcp/gcsio.py | 60 ++-
sdks/python/apache_beam/version.py | 2 +-
sdks/python/setup.py | 8 +-
sdks/python/test-suites/tox/py38/build.gradle | 4 +
sdks/python/tox.ini | 4 +-
website/www/site/assets/icons/linkedin-icon.png | Bin 0 -> 4341 bytes
website/www/site/assets/icons/linkedin-icon.svg | 21 -
.../www/site/content/en/blog/ApachePlayground.md | 47 ++
website/www/site/content/en/case-studies/hop.md | 4 +-
website/www/site/content/en/case-studies/neo4j.md | 2 +-
.../www/site/content/en/case-studies/spotify.md | 2 +-
website/www/site/content/en/case-studies/talend.md | 2 +-
website/www/site/data/en/pillars_social.yaml | 2 +-
.../www/site/static/images/blog/BeamPlayground.gif | Bin 0 -> 3581157 bytes
.../static/images/logos/powered-by/Trustpilot.png | Bin 25458 -> 9526 bytes
.../static/images/logos/powered-by/Wayfair.png | Bin 82679 -> 30400 bytes
.../site/static/images/logos/powered-by/eBay.png | Bin 90243 -> 20607 bytes
.../site/static/images/logos/powered-by/goga.png | Bin 44510 -> 14952 bytes
.../site/static/images/logos/powered-by/hop.png | Bin 0 -> 18210 bytes
.../site/static/images/logos/powered-by/hop.svg | 71 ---
.../site/static/images/logos/powered-by/kio.png | Bin 89139 -> 23035 bytes
.../site/static/images/logos/powered-by/klio.png | Bin 9212 -> 5619 bytes
.../static/images/logos/powered-by/liveramp.png | Bin 9653 -> 5324 bytes
.../site/static/images/logos/powered-by/lyft.png | Bin 50028 -> 42085 bytes
.../site/static/images/logos/powered-by/neo4j.png | Bin 0 -> 12965 bytes
.../site/static/images/logos/powered-by/neo4j.svg | 19 -
.../site/static/images/logos/powered-by/oriel.png | Bin 76963 -> 25382 bytes
.../static/images/logos/powered-by/paloalto.png | Bin 20154 -> 9471 bytes
.../static/images/logos/powered-by/ricardo.png | Bin 25264 -> 10302 bytes
.../site/static/images/logos/powered-by/scio.png | Bin 28972 -> 10494 bytes
.../site/static/images/logos/powered-by/seznam.png | Bin 72223 -> 23604 bytes
.../static/images/logos/powered-by/spotify.png | Bin 0 -> 5254 bytes
.../static/images/logos/powered-by/spotify.svg | 17 -
.../site/static/images/logos/powered-by/talend.png | Bin 0 -> 5663 bytes
.../site/static/images/logos/powered-by/talend.svg | 121 -----
.../site/static/images/logos/powered-by/zillow.png | Bin 26095 -> 13697 bytes
93 files changed, 2297 insertions(+), 1078 deletions(-)
copy sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/it/{SpannerChangeStreamIT.java => SpannerChangeStreamPostgresIT.java} (69%)
create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/util/TestJsonMapper.java
create mode 100644 website/www/site/assets/icons/linkedin-icon.png
delete mode 100644 website/www/site/assets/icons/linkedin-icon.svg
create mode 100644 website/www/site/content/en/blog/ApachePlayground.md
create mode 100644 website/www/site/static/images/blog/BeamPlayground.gif
create mode 100644 website/www/site/static/images/logos/powered-by/hop.png
delete mode 100644 website/www/site/static/images/logos/powered-by/hop.svg
create mode 100644 website/www/site/static/images/logos/powered-by/neo4j.png
delete mode 100644 website/www/site/static/images/logos/powered-by/neo4j.svg
create mode 100644 website/www/site/static/images/logos/powered-by/spotify.png
delete mode 100644 website/www/site/static/images/logos/powered-by/spotify.svg
create mode 100644 website/www/site/static/images/logos/powered-by/talend.png
delete mode 100644 website/www/site/static/images/logos/powered-by/talend.svg