You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2023/02/27 14:21:51 UTC
[beam] branch dependabot/go_modules/sdks/cloud.google.com/go/spanner-1.44.0 updated (010cdc45bd2 -> f397cba3f64)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/go_modules/sdks/cloud.google.com/go/spanner-1.44.0
in repository https://gitbox.apache.org/repos/asf/beam.git
discard 010cdc45bd2 Bump cloud.google.com/go/spanner from 1.43.0 to 1.44.0 in /sdks
add 5b4edaf83bf [#24789][prism] V0 Go Direct Runner Replacement, Prism. (#25568)
add a6bd21a5616 Add retry logic to Python boot script.
add e99999509f8 More verbose messaging.
add 8cfee7d05e6 Merge pull request #25473 Add retry logic to Python boot script.
add d8e5047dad3 Bump github.com/testcontainers/testcontainers-go in /sdks (#25517)
add 95a6566107e [Avro] Use "extensions/avro" instead of avro from"core" in Java SDK modules (#24992)
add 83cfdb829b8 PCollection data sampling for Java SDK harness #25064 (#25354)
add 85817b6b343 Add partitioned reads to JDBC SchemaIO (#25577)
add 42827acbb64 Deprecate Avro API classes in "sdks/java/core"
add 5b59377ff61 Merge pull request #25534: [Avro] Deprecate Avro API classes in "sdks/java/core"
add cebe23b37e2 Bump github.com/aws/aws-sdk-go from 1.33.0 to 1.34.0 in /sdks (#25469)
add 50e5d728fc1 ML model evaluation website page (#25589)
add 2e7584cc43a Add Data Sampling capability URN to Java SDK (#25593)
add 93c1e92f371 Add anchor link for the logo wall (#25594)
add 4ce8eeda196 Read API Source v2 (#25392)
add 27c55107e56 Moving to 2.47.0-SNAPSHOT on master branch.
add 011296c1465 Apply shallow clone for jenkins (#25595)
add a1f09f7038e Bump golang.org/x/net from 0.5.0 to 0.7.0 in /sdks (#25543)
add a34f4591de4 Bump github.com/containerd/containerd from 1.6.12 to 1.6.18 in /sdks (#25512)
add 19e63e1da78 Add mock modules (#25603)
add 96c68ab058d Add missing entries to mass_comment.py (#25609)
add 948f7cdf489 Fix go postcommit naming (#25613)
add 160be6b14a4 Bump to Dataproc 2.1 (#25616)
add dcfadddf52a Prioritise Avro providers from "extensions/core" (#25611)
add 441d44acb67 Bump golang.org/x/oauth2 in /sdks (#25621)
add bddfd86afd2 restructure ml overview website page (#25607)
add de71d132f6a [Beam#25046]add support for flink 1.16 for Beam Flink Runner (#25584)
add 5a46e0083f5 Update CHANGE.md to 2.47.0 (#25628)
add 2bd1cebfd78 Bump google.golang.org/api from 0.109.0 to 0.110.0 in /sdks (#25619)
add 21baee4fa55 Bump go.mongodb.org/mongo-driver from 1.11.1 to 1.11.2 in /sdks (#25605)
add 0bb5be45bf1 Fix typo in docstring (#25630)
add 124e1d6a5f1 Merge pull request #25627: #25626 Use the correct constructor when creating StreamWriter
add 9d4df48864d Bump github.com/aws/aws-sdk-go-v2/feature/s3/manager in /sdks (#25637)
add f397cba3f64 Bump cloud.google.com/go/spanner from 1.43.0 to 1.44.0 in /sdks
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (010cdc45bd2)
\
N -- N -- N refs/heads/dependabot/go_modules/sdks/cloud.google.com/go/spanner-1.44.0 (f397cba3f64)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.test-infra/dataproc/flink_cluster.sh | 3 +-
.test-infra/jenkins/CommonJobProperties.groovy | 4 +
.test-infra/jenkins/job_PostCommit_Go.groovy | 2 +-
CHANGES.md | 39 +-
build.gradle.kts | 4 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
examples/java/build.gradle | 1 +
.../beam/examples/complete/AutoComplete.java | 2 +-
.../beam/examples/complete/TrafficMaxLaneFlow.java | 2 +-
.../beam/examples/complete/TrafficRoutes.java | 2 +-
.../beam/examples/complete/game/UserScore.java | 2 +-
.../complete/kafkatopubsub/avro/AvroDataClass.java | 2 +-
.../kafkatopubsub/transforms/FormatTransform.java | 2 +-
.../apache/beam/examples/snippets/Snippets.java | 2 +-
.../transforms/io/gcp/bigquery/BigQueryMyData.java | 2 +-
.../examples/subprocess/utils/ExecutableFile.java | 2 +-
.../examples/complete/game/LeaderBoardTest.java | 2 +-
.../complete/game/StatefulTeamScoreTest.java | 2 +-
examples/kotlin/build.gradle | 1 +
.../beam/examples/kotlin/snippets/Snippets.kt | 2 +-
gradle.properties | 6 +-
release/src/main/scripts/mass_comment.py | 53 +-
runners/core-construction-java/build.gradle | 2 +
.../construction/AvroGenericCoderRegistrar.java | 2 +-
.../construction/AvroGenericCoderTranslator.java | 2 +-
.../runners/core/construction/Environments.java | 1 +
.../core/construction/CoderTranslationTest.java | 2 +-
.../core/construction/EnvironmentsTest.java | 3 +
runners/direct-java/build.gradle | 4 +-
.../sketching => runners/flink/1.16}/build.gradle | 31 +-
.../job-server-container/build.gradle | 0
.../flink/1.16/job-server/build.gradle | 27 +-
.../runners/flink/ReadSourceStreamingTest.java | 3 +-
.../apache/beam/runners/flink/ReadSourceTest.java | 3 +-
.../flink/streaming/GroupByNullKeyTest.java | 3 +-
.../flink/streaming/TopWikipediaSessionsTest.java | 3 +-
runners/google-cloud-dataflow-java/build.gradle | 2 +
.../util/AvroCoderCloudObjectTranslator.java | 2 +-
.../runners/dataflow/util/CloudObjectsTest.java | 2 +-
.../google-cloud-dataflow-java/worker/build.gradle | 1 +
.../runners/dataflow/worker/AvroByteReader.java | 4 +-
.../translation/SparkSessionFactory.java | 16 +-
runners/spark/spark_runner.gradle | 2 +
.../beam/runners/spark/io/AvroPipelineTest.java | 2 +-
sdks/go.mod | 79 +-
sdks/go.sum | 853 +++---------
sdks/go/pkg/beam/core/core.go | 2 +-
sdks/go/pkg/beam/runners/prism/README.md | 1 +
.../pkg/beam/runners/prism/internal/coders_test.go | 6 +-
sdks/go/pkg/beam/runners/prism/internal/execute.go | 278 +++-
.../beam/runners/prism/internal/execute_test.go | 417 ++++++
.../beam/runners/prism/internal/separate_test.go | 595 ++++++++
sdks/go/pkg/beam/runners/prism/internal/stage.go | 400 ++++++
sdks/go/pkg/beam/runners/prism/prism.go | 48 +
.../java/org/apache/beam/sdk/coders/AvroCoder.java | 5 +
.../apache/beam/sdk/coders/AvroGenericCoder.java | 10 +-
.../org/apache/beam/sdk/coders/DefaultCoder.java | 2 +-
.../main/java/org/apache/beam/sdk/io/AvroIO.java | 6 +
.../apache/beam/sdk/io/AvroSchemaIOProvider.java | 6 +
.../main/java/org/apache/beam/sdk/io/AvroSink.java | 10 +-
.../java/org/apache/beam/sdk/io/AvroSource.java | 5 +
.../beam/sdk/io/ConstantAvroDestination.java | 10 +-
.../beam/sdk/io/DynamicAvroDestinations.java | 6 +
.../beam/sdk/io/SerializableAvroCodecFactory.java | 7 +
.../apache/beam/sdk/schemas/AvroRecordSchema.java | 6 +
.../org/apache/beam/sdk/schemas/io/Providers.java | 30 +-
.../io/payloads/AvroPayloadSerializerProvider.java | 8 +
.../beam/sdk/schemas/utils/AvroByteBuddyUtils.java | 8 +
.../apache/beam/sdk/schemas/utils/AvroUtils.java | 6 +
.../org/apache/beam/sdk/io/TextIOWriteTest.java | 30 +-
.../org/apache/beam/sdk/transforms/LatestTest.java | 4 +-
sdks/java/extensions/avro/build.gradle | 4 +-
.../extensions/avro/coders/DefaultCoderTest.java | 61 +
.../extensions/avro/schemas/SchemaCoderTest.java | 196 +++
.../avro/schemas/transforms/ConvertTest.java | 147 ++
.../ExternalSchemaIOTransformRegistrar.java | 60 +-
sdks/java/extensions/sketching/build.gradle | 1 +
.../sketching/ApproximateDistinctTest.java | 2 +-
.../sketching/SketchFrequenciesTest.java | 2 +-
sdks/java/extensions/sql/build.gradle | 1 +
.../sql/meta/provider/avro/AvroTableProvider.java | 4 +-
.../sql/meta/provider/parquet/ParquetTable.java | 2 +-
.../provider/kafka/BeamKafkaTableAvroTest.java | 4 +-
.../meta/provider/kafka/KafkaTableProviderIT.java | 2 +-
.../provider/pubsub/PubsubTableProviderIT.java | 4 +-
sdks/java/harness/build.gradle | 1 +
.../java/org/apache/beam/fn/harness/FnHarness.java | 14 +-
.../fn/harness/control/ProcessBundleHandler.java | 18 +-
.../harness/data/PCollectionConsumerRegistry.java | 59 +-
.../apache/beam/fn/harness/debug/DataSampler.java | 125 ++
.../beam/fn/harness/debug/OutputSampler.java | 129 ++
.../beam/fn/harness/debug/package-info.java} | 17 +-
.../harness/control/ProcessBundleHandlerTest.java | 48 +-
.../data/PCollectionConsumerRegistryTest.java | 60 +
.../beam/fn/harness/debug/DataSamplerTest.java | 273 ++++
.../beam/fn/harness/debug/OutputSamplerTest.java | 188 +++
sdks/java/io/amazon-web-services/build.gradle | 1 +
.../sdk/io/aws/dynamodb/DynamoDBIOWriteTest.java | 2 +-
sdks/java/io/amazon-web-services2/build.gradle | 1 +
.../sdk/io/aws2/dynamodb/DynamoDBIOWriteTest.java | 2 +-
sdks/java/io/cdap/build.gradle | 1 +
.../apache/beam/sdk/io/cdap/TestRowDBWritable.java | 2 +-
sdks/java/io/file-based-io-tests/build.gradle | 1 +
.../java/org/apache/beam/sdk/io/avro/AvroIOIT.java | 4 +-
.../apache/beam/sdk/io/parquet/ParquetIOIT.java | 2 +-
sdks/java/io/file-schema-transform/build.gradle | 1 +
.../AvroWriteSchemaTransformFormatProvider.java | 6 +-
.../FileWriteSchemaTransformFormatProviders.java | 2 +-
.../ParquetWriteSchemaTransformFormatProvider.java | 2 +-
...FileWriteSchemaTransformFormatProviderTest.java | 6 +-
...FileWriteSchemaTransformFormatProviderTest.java | 2 +-
sdks/java/io/google-cloud-platform/build.gradle | 2 +
.../AvroGenericRecordToStorageApiProto.java | 2 +-
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 361 +++--
.../beam/sdk/io/gcp/bigquery/BigQueryOptions.java | 11 +
.../sdk/io/gcp/bigquery/BigQueryQuerySource.java | 2 +-
.../io/gcp/bigquery/BigQueryQuerySourceDef.java | 2 +-
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 4 +-
.../sdk/io/gcp/bigquery/BigQuerySourceBase.java | 2 +-
.../sdk/io/gcp/bigquery/BigQuerySourceDef.java | 2 +-
.../gcp/bigquery/BigQueryStorageArrowReader.java | 2 +-
.../io/gcp/bigquery/BigQueryStorageSourceBase.java | 67 +-
.../BigQueryStorageStreamBundleSource.java | 381 ++++++
.../sdk/io/gcp/bigquery/BigQueryTableSource.java | 2 +-
.../io/gcp/bigquery/BigQueryTableSourceDef.java | 2 +-
...StorageApiDynamicDestinationsGenericRecord.java | 2 +-
.../beam/sdk/io/gcp/pubsub/PubsubClient.java | 2 +-
.../apache/beam/sdk/io/gcp/pubsub/PubsubIO.java | 4 +-
.../PubsubLiteReadSchemaTransformProvider.java | 2 +-
.../PubsubLiteWriteSchemaTransformProvider.java | 2 +-
.../model/ChangeStreamRecordMetadata.java | 2 +-
.../changestreams/model/ChildPartition.java | 2 +-
.../changestreams/model/ChildPartitionsRecord.java | 2 +-
.../spanner/changestreams/model/ColumnType.java | 2 +-
.../changestreams/model/DataChangeRecord.java | 2 +-
.../changestreams/model/HeartbeatRecord.java | 2 +-
.../io/gcp/spanner/changestreams/model/Mod.java | 2 +-
.../gcp/spanner/changestreams/model/ModType.java | 2 +-
.../changestreams/model/PartitionMetadata.java | 2 +-
.../gcp/spanner/changestreams/model/TypeCode.java | 2 +-
.../changestreams/model/ValueCaptureType.java | 2 +-
.../AvroGenericRecordToStorageApiProtoTest.java | 6 +-
.../sdk/io/gcp/bigquery/BigQueryAvroUtilsTest.java | 2 +-
.../sdk/io/gcp/bigquery/BigQueryIOReadTest.java | 2 +-
.../io/gcp/bigquery/BigQueryIOStorageReadTest.java | 2 +-
...ryIOStorageReadWithStreamBundleSourceTest.java} | 1418 +++++++++-----------
.../sdk/io/gcp/bigquery/BigQueryUtilsTest.java | 2 +-
.../beam/sdk/io/gcp/pubsub/PubsubIOTest.java | 2 +-
.../PubsubReadSchemaTransformProviderTest.java | 2 +-
.../sdk/io/gcp/pubsub/PubsubRowToMessageTest.java | 2 +-
...bsubSchemaTransformMessageToRowFactoryTest.java | 2 +-
.../PubsubWriteSchemaTransformProviderTest.java | 2 +-
.../encoder/TimestampEncodingTest.java | 2 +-
sdks/java/io/hadoop-format/build.gradle | 1 +
.../apache/beam/sdk/io/hadoop/format/Employee.java | 2 +-
.../io/hadoop/format/HadoopFormatIOReadTest.java | 2 +-
.../sdk/io/hadoop/format/TestRowDBWritable.java | 2 +-
sdks/java/io/influxdb/build.gradle | 1 +
.../org/apache/beam/sdk/io/influxdb/Model.java | 2 +-
sdks/java/io/jdbc/build.gradle | 1 +
.../beam/sdk/io/jdbc/JdbcSchemaIOProvider.java | 63 +-
.../beam/sdk/io/jdbc/JdbcSchemaIOProviderTest.java | 129 ++
.../apache/beam/sdk/io/jdbc/SchemaUtilTest.java | 2 +-
sdks/java/io/kafka/build.gradle | 2 +
...onfluentSchemaRegistryDeserializerProvider.java | 2 +-
.../beam/sdk/io/kafka/KafkaCheckpointMark.java | 2 +-
.../java/org/apache/beam/sdk/io/kafka/KafkaIO.java | 2 +-
.../io/kafka/KafkaReadSchemaTransformProvider.java | 4 +-
.../beam/sdk/io/kafka/KafkaUnboundedSource.java | 2 +-
.../kafka/KafkaWriteSchemaTransformProvider.java | 2 +-
...uentSchemaRegistryDeserializerProviderTest.java | 2 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOIT.java | 2 +-
sdks/java/io/parquet/build.gradle | 2 +
.../org/apache/beam/sdk/io/parquet/ParquetIO.java | 4 +-
.../apache/beam/sdk/io/parquet/ParquetIOTest.java | 4 +-
sdks/java/io/snowflake/build.gradle | 1 +
.../test/unit/read/SnowflakeIOReadTest.java | 2 +-
sdks/java/testing/expansion-service/build.gradle | 1 +
.../testing/expansion/TestExpansionService.java | 2 +-
sdks/java/testing/nexmark/build.gradle | 1 +
.../apache/beam/sdk/nexmark/NexmarkLauncher.java | 2 +-
.../org/apache/beam/sdk/nexmark/NexmarkUtils.java | 2 +-
sdks/python/apache_beam/coders/coder_impl.pxd | 4 +
sdks/python/apache_beam/coders/coder_impl.py | 18 +-
sdks/python/apache_beam/coders/coders.py | 19 +
.../apache_beam/coders/coders_test_common.py | 1 +
sdks/python/apache_beam/coders/row_coder.py | 3 +
sdks/python/apache_beam/coders/slow_stream.py | 6 +
sdks/python/apache_beam/coders/stream.pxd | 3 +
sdks/python/apache_beam/coders/stream.pyx | 16 +
sdks/python/apache_beam/coders/stream_test.py | 9 +
.../io/external/xlang_jdbcio_it_test.py | 18 +
sdks/python/apache_beam/io/jdbc.py | 37 +-
.../python/apache_beam/options/pipeline_options.py | 2 +-
sdks/python/apache_beam/pipeline.py | 2 +-
sdks/python/apache_beam/version.py | 2 +-
sdks/python/container/boot.go | 49 +-
sdks/python/scripts/generate_pydoc.sh | 2 +-
sdks/python/tox.ini | 5 -
settings.gradle.kts | 4 +
.../en/documentation/ml/model-evaluation.md | 85 ++
.../site/content/en/documentation/ml/overview.md | 20 +-
website/www/site/layouts/case-studies/list.html | 2 +-
.../partials/section-menu/en/documentation.html | 7 +-
website/www/site/static/images/ml-workflows.svg | 2 +-
205 files changed, 5513 insertions(+), 1920 deletions(-)
copy {sdks/java/extensions/sketching => runners/flink/1.16}/build.gradle (50%)
copy runners/flink/{1.12 => 1.16}/job-server-container/build.gradle (100%)
copy sdks/java/core/src/main/java/org/apache/beam/sdk/coders/AvroGenericCoder.java => runners/flink/1.16/job-server/build.gradle (52%)
create mode 100644 sdks/go/pkg/beam/runners/prism/internal/execute_test.go
create mode 100644 sdks/go/pkg/beam/runners/prism/internal/separate_test.go
create mode 100644 sdks/go/pkg/beam/runners/prism/internal/stage.go
create mode 100644 sdks/go/pkg/beam/runners/prism/prism.go
create mode 100644 sdks/java/extensions/avro/src/test/java/org/apache/beam/sdk/extensions/avro/coders/DefaultCoderTest.java
create mode 100644 sdks/java/extensions/avro/src/test/java/org/apache/beam/sdk/extensions/avro/schemas/SchemaCoderTest.java
create mode 100644 sdks/java/extensions/avro/src/test/java/org/apache/beam/sdk/extensions/avro/schemas/transforms/ConvertTest.java
create mode 100644 sdks/java/harness/src/main/java/org/apache/beam/fn/harness/debug/DataSampler.java
create mode 100644 sdks/java/harness/src/main/java/org/apache/beam/fn/harness/debug/OutputSampler.java
copy sdks/java/{io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/model/ModType.java => harness/src/main/java/org/apache/beam/fn/harness/debug/package-info.java} (66%)
create mode 100644 sdks/java/harness/src/test/java/org/apache/beam/fn/harness/debug/DataSamplerTest.java
create mode 100644 sdks/java/harness/src/test/java/org/apache/beam/fn/harness/debug/OutputSamplerTest.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryStorageStreamBundleSource.java
copy sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/{BigQueryIOStorageReadTest.java => BigQueryIOStorageReadWithStreamBundleSourceTest.java} (64%)
create mode 100644 sdks/java/io/jdbc/src/test/java/org/apache/beam/sdk/io/jdbc/JdbcSchemaIOProviderTest.java
create mode 100755 website/www/site/content/en/documentation/ml/model-evaluation.md
mode change 100644 => 100755 website/www/site/layouts/partials/section-menu/en/documentation.html