You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by ke...@apache.org on 2021/03/30 18:55:31 UTC
[beam] branch release-2.29.0 updated (85e93ca -> 2105fe8)
This is an automated email from the ASF dual-hosted git repository.
kenn pushed a change to branch release-2.29.0
in repository https://gitbox.apache.org/repos/asf/beam.git.
omit 85e93ca Merge pull request #14306: [BEAM-11862] Updates ExternalTransformFinder to consider external leaf transforms
omit e334045 [BEAM-11862] Updates ExternalTransformFinder to consider external leaf transforms (#14293)
omit adf43d9 Merge pull request #14304: Revert "Add License file for the Go SDK for pkg.go.dev" (CP #14267)
omit 2634d3b Merge pull request #14267 from apache/revert-11657-lostluck-patch-2
omit 43b259c Merge pull request #14223: [BEAM-11967] Cherrypick #14208 to release 2.9.0 branch: Fix Dataflow V2 metrics
omit 69c9e47 Merge pull request #14240: [BEAM-11613] Limits Dataflow GCR container image overriding
omit 9a2fbcb Limits Dataflow GCR container image overriding to external Apache Beam container images
omit 269c9c5 Unsickbay metrics tests in runner v2
omit bf3e38a Attach portable proto to DataflowPipelineJob
omit e7cfa2f Merge #14209: Cherrypick #14120 to 2.29.0 release branch, fixing Flink merging windowing
omit 9a11c22 Set Dataflow container version for 2.29.0 release branch
omit 7fdcbec [BEAM-11887] Change SortingFlinkCombineRunner usage to only for Sessions
add b8b5b58 Moving to 2.30.0-SNAPSHOT on master branch.
add 2da7448 [BEAM-11887] Change SortingFlinkCombineRunner usage to only for Sessions (#14120)
add 40e8267 Only clone to depth 1 for cutting release branch
add fffb85a Merge pull request #14210: Only clone to depth 1 for cutting release branch
add d2259fa Attach portable proto to DataflowPipelineJob
add c818da0 Unsickbay metrics tests in runner v2
add e330267 Merge pull request #14208: [BEAM-11967] Attach portable proto to DataflowPipelineJob
add 1fc801d [BEAM-10961] enable strict dependency checking for sdks/java/maven-archetypes/*
add 0da0a1b [BEAM-10961] remove dependencies blocks containing, which only contained unusedDeclared dependencies
add adf3dd4 Merge pull request #14196 from sonam-vend/sdks-java-maven-archetypes
add b0a33da [BEAM-1251] Use Python 3 semantics in Cython-compiled modules. (#14198)
add 1db92d3 Set Dataflow container version correctly in set_version.sh
add 7f925ab Set Go SDK version in set_version.sh
add b8d0f87 Explain set_version.sh in comments more clearly
add 04e34e1 Add knowledge to `git add` to set_version.sh
add b1b3d7b Merge #14212: Fix set_version.sh in release scripts
add 86c55f4 Add more comments to describe PubSubReadPayload and PubSubWritePayload.
add 5afe677 Merge pull request #14204 from Add more comments to describe PubSubReadPayload and PubSubWritePayload.
add 0ab785a [BEAM-11797] Fixed the flaky test (#14220)
add 79da783 [BEAM-11926] Improve error when missing Beam schema for BigqueryIO
add 0af8c3b Merge pull request #14152 from [BEAM-11926] Improve error when missing Beam schema for BigqueryIO
add 0f6aacf [BEAM-11910] Increase the bag page limit for continuation pages from 8MB to 32MB.
add c925ff8 Merge pull request #14154: [BEAM-11910] Increase the bag page limit for continuation pages
add 9c0f608 [BEAM-9282] Move structured streaming runner into Spark 2 specific module
add 47fd176 [BEAM-9282] Separate modules for Spark 2/3
add 17710e5 [BEAM-9282] Separate modules for Spark 2/3 job-server
add e86862c [BEAM-9282] Separate modules for Spark 2/3 job-server container
add 44b7a87 [BEAM-7092] Run PostCommit tests for Spark 3 module too
add cc29b5b [BEAM-7092] Update tests invocation for Spark 2 module
add b234bb4 [BEAM-9283] Add Spark 3 test jobs to the CI (Java 11)
add 07e52fc [BEAM-11654] Publish Spark 2 and 3 specific Job-Server containers
add d9f3157 [BEAM-7092] Add paranamer 2.8 license to container (Spark 3 / Avro)
add 153876f Merge pull request #14216: [BEAM-7093] Support Spark 3 in Spark runner
add b388776 [BEAM-7092] Upgrade Spark 3 to version 3.1.1
add f06ffcb [BEAM-7092] Fix artifact name for Spark 2 runner to be the same as before
add 1fdf315 [BEAM-7092] Make Spark 3 runner name consistent with other modules '-$VERSION'
add 88d687b Merge pull request #14228: [BEAM-7092] Fix artifact names for Spark separated modules + upgrade to Spark 3.1.1
add b2ce15e [BEAM-8221] Fix NPE while reading from non-existent Kafka topic
add 08a9d54 Merge pull request #14217: [BEAM-8221] Fix NPE while reading from non-existent Kafka topic
add 15ef51f [BEAM-11963] Fix KafkaTableProviderIT (#14205)
add 0ad65c6 [BEAM-9547] Implement count() on deferred DataFrame, Series (#14178)
add 91c8160 [BEAM-8787] Don't recommend './gradlew check' to new contributors.
add ee83d09 [BEAM-8787] Add instructions for cloning the Beam repo.
add 7a72101 [BEAM-8787] cd into the newly created beam repo.
add ad01046 Merge pull request #14225 from ibzib/BEAM-8787
add 8aa0df4 Update partitioning ordering
add cfa75ae Update PartitioningSession
add 573fa43 Explicitly define the requires/preserves semantics
add a524931 Reverse comparison detecting when elementwise expressions need Index
add fe198bb Bulk update preserves=
add f2ba501 nunique update
add 61d7572 TODO updates
add bdc9ade Strong preservation assertion for PartitioningSession
add db9355a Update stage fusion logic
add 3e24899 Actually run index partitioning checks
add f0a9c73 Fix Index[i,j] subpartitioing
add be6d5f6 Fix output_partitioning, add tests
add a1d8b85 fix callable, add test
add fb5be4b Bulk rename Nothing() to Arbitrary()
add a273061 Allow preserves=Arbitrary, bulk replace preserves=Index() with preserves=Arbitrary()
add 1b3e231 Fix pd.concat(keys=) partitioning
add ba61708 [BEAM-11881] Fix partitioning ordering for DataFrames (#14135)
add b8066d5 Few small updates to the release guide
add 5a9bb83 Merge pull request #14230: Few small updates to the release guide
add 8d0b19f [BEAM-11972] Close all opened channels/readers on ParquetIO
add 916c156 Merge pull request #14229: [BEAM-11972] Close all opened channels/readers on ParquetIO
add 23ba50b [BEAM-10943] Add builtin functions in the Calcite planner.
add 3a9591f Merge pull request #14222 from ibzib/BEAM-10943-builtins
add 7f3d8dc add environment caching to custom container instructions
add f600108 Merge pull request #14234 from emilymye/editflinkdocs
add fdb0fd7 Limits Dataflow GCR container image overriding to external Apache Beam container images
add baecb10 Merge pull request #14224: [BEAM-11613] Limits Dataflow GCR container image overriding
add a4f0bc9 [BEAM-5628] remove vcfio.py and concerning resources (#14185)
add f515758 Removes non-interactive from SVN commands
add 56d7e1b Merge pull request #14231: Removes non-interactive from SVN commands
add 915441f Added Kio to powered by list
add 066efd4 Merge pull request #14245: Added Kio to powered by list
add 7f989ca Preparing vendored gRPC 1.36.0
add 77239e4 Merge pull request #14242: [BEAM-11227] Preparing vendored gRPC 1.36.0
add 7ff7ceb Create 2.30.0 entry in CHANGES.md (#14250)
add 3d4518f [BEAM-11726] Bump Clickhouse version to "0.2.6"
add 25bad0e Merge pull request #14246: [BEAM-11726] Bump Clickhouse version to 0.2.6
add 7af3d13 [BEAM-11764] Bump com.amazonaws version to 1.11.974
add 8d6fa737 Merge pull request #14247: [BEAM-11764] Bump com.amazonaws version to 1.11.974
add 03017b6 Replacing use of TFT's deprecated dataset_schema.from_feature_spec (#14243)
add 7c62484 Reverting perfmark relocation
add 2a24933 Relocating com.google.geo.type in proto_google_common_protos
add 67d90e6 check task to confirm the vendored JAR file
add c445ac1 Merge pull request #14257: [BEAM-11227] Fixed class relocation for vendored gRPC 1.36.0 module
add e1af6ed [BEAM-10925] Re-enable Java UDF tests.
add 8ecff38 [BEAM-10925] Reuse existing config fields.
add 1197452 [BEAM-10925] Fix comment.
add 4ff08fe Merge pull request #14235 from ibzib/udf-test
add ee62cbe Merge pull request #14252 from Add types to FileMatching
add 93d0be0 [BEAM-11969] Adds an option for setting row-group size in ParquetIO (#14227)
add 4565a31 [BEAM-11659] Add new schema types to Pub/Sub SQL (#13980)
add 3014959 [BEAM-11628] Switch to pd.testing.assert* in transforms_test (#14256)
add d559544 [BEAM-10708] Clean up pipeline instrument (#14176)
add 9e09977 [BEAM-9615] Embedded structs and Deterministic map encoding, and Logical Types. (#14192)
add e8f9c68 [BEAM-10937] Tour of Beam: Reading and writing data notebook (#14045)
add 99a1b3a Keep dev/SNAPSHOT version on release branch; set to non-SNAPSHOT version only for RCs
add 58eab9e Update instructions for cutting release branch
add 8a91ca3 Merge pull request #14239: Update script and instructions for cutting release branch
add e3a7c26 Add script to tag a commit for building RC
add b817a1c Adjust build_release_candidate.sh to use existing tag
add cf4bbf3 Update instructions to include tagging an RC commit
add d04319d Merge pull request #14213: Add release script to choose a commit for an RC
add ab79451 Add example of Spark rest URL.
add 15c89fd Merge pull request #14221: Add example of Spark rest URL.
add ee29766 Better error for missing java.
add 6e799fa Merge pull request #14266 from robertwb/java-error
add f561a83 Revert "Add License file for the Go SDK for pkg.go.dev"
add ec64a81 Merge pull request #14267 from apache/revert-11657-lostluck-patch-2
add d6436b3 [BEAM-11979] Ignore not serializable filter fields in python MongoDBIO display data
add 9791ef9 Merge pull request #14237 from [BEAM-11979] Ignore not serializable filter fields in python MongoDBI…
add abc7b20 [BEAM-11992] Run CrossLanguage ValidatesRunner tests only for Spark 2
add 540957e Merge pull request #14253: [BEAM-11992] Run CrossLanguage ValidatesRunner tests only for Spark 2
add 6db22c6 [BEAM-8778] Bump software.amazon.awssdk version to 2.15.31
add c4f62ca Merge pull request #14264: [BEAM-8778] Bump software.amazon.awssdk version to 2.15.31
add 3fc2ab1 [BEAM-9547] Add NDFrame to doctests, implement a few more operations (#14236)
add 9113c6f [BEAM-9283] Add Java 11 Jpms compatibility tests for Spark runner
add aee24ad Merge pull request #14270: [BEAM-9283] Add Java 11 Jpms compatibility tests for Spark runner
add c4ddb59 Remove change detector tests for Java-on-JRH
add 1b4d2ff Make beam_fn_api, use_runner_v2, and use_unified_worker all trigger the same behavior in the SDK
add 0619de8 Merge pull request #14226: Eliminate use_runner_v2, and use_unified_worker in the Java SDK, since beam_fn_api suffices
add e96f69a Revert "Increase the timeout for ValidatesContainer suite" (#14219)
add ff4366d [BEAM-11023] Fix testGroupByKeyWithBadEqualsHashCode failing on Spark Structured Streaming runner
add 96bf00a [BEAM-11023] Fix testLargeKeys100MB on Spark Structured Streaming runner
add 3ac902a [BEAM-11023] Change access level GroupByKeyTest
add 5c9c8c0 Merge pull request #14275: [BEAM-11023] Fix GroupByKeyTest testLargeKeys100MB and testGroupByKeyWithBadEqualsHashCode failing on Spark Structured Streaming runner
add e67ba2b [BEAM-5949] Remove trivial __ne__ implementations. (#13887)
add 7a60699 Returning successes from FhirIO executeBundles. Needed for healthcare solutions accuracy in logging what was written to the fhir store.
add 7f891c3 Syncing with milenas change
add ba4a111 Undo formatting changes from Google auto-formatter.
add e4768a4 Adding the tuple tag check for FhirIO.Write.Result creation.
add 95201ac Updating the contains TupleTag check to use the PCollectionTuple .has() method, casting TupleTagList -> Collection creates an exception for some.
add 05f6aaf Fix build
add 655b29b Running spotless apply
add 61004f3 Merge branch 'master' of https://github.com/apache/beam
add 5de886d Merge remote-tracking branch 'upstream/master'
add 03d5364 adding LRO counters to import/export/deidentify
add 8611174 running spotless apply
add 62320cf changing NAME -> KEY after feedback from an internal review
add 3bef3da Adding a log statement for when failures occur.
add 5f7358d Removing ignores from tests
add 1a4cb59 spotless apply
add 14f2371 Updating counter names
add b2f84d7 Merge pull request #14233 from [BEAM-11733] Adding LRO counters to FhirIO and re-enable FhirIO tests
add 93417ce [BEAM-9038] Fix typo for setting_global_window example
add 195bbf3 Merge pull request #14271 from [BEAM-9038] Fix typo for setting_global_window example
add aa6c075 [BEAM-12010] Create separate merge rules for different Calc implementations.
add 3a096ee [BEAM-12010] Use simpler rule constructor so we don't have to pass null.
add 49b9d1b Merge pull request #14265 from ibzib/BEAM-12010
add 2e68952 Sickbay committed metrics, not attempted metrics, for Dataflow streaming V2
add ff74296 Merge pull request #14277: Sickbay committed metrics, not attempted metrics, for Dataflow streaming V2
add 22a8b18 Add an option to create Dataflow piplines from a snapshot for python sdk (#14278)
add ba14ca2 [BEAM-11992] Run CrossLanguage ValidatesRunner for Spark 3
add 40eef35 Merge pull request #14269: [BEAM-11992] Run CrossLanguage ValidatesRunner for Spark 3
add 0408122 Update names.py
add d6b020a Merge pull request #14279 from apache/tvalentyn-patch-1
add 209fd5d [BEAM-7093] Update some out-of-date Gradle Spark instructions.
add 8ac1eb5 Merge pull request #14282: [BEAM-7093] Update some out-of-date Gradle Spark instructions.
add b661b87 [BEAM-7078] Bump com.amazonaws:amazon-kinesis-client to version 1.14.2
add 5bfdc3b Merge pull request #14283: [BEAM-7078] Bump com.amazonaws:amazon-kinesis-client to version 1.14.2
add 9c5d644 Add ListCoder distinct from IterableCoder.
add 6dd0475 Make ListCoder public, fix pydoc error.
add a0447e4 Merge pull request #14110 Add ListCoder distinct from IterableCoder.
add 0ca0088 Merge pull request #11767: [BEAM-11648] BigQuery Storage API sink
add 9e3a2f0 Minor FhirIO Search changes
add 170d581 Merge pull request #14249 from Minor FhirIO Search changes
add a954e6f Remove @Hidden and @Experimental annotation of CreateFromSnapshot pipelien option.
add 38c9caa Merge pull request #14296 from andyxiexu/java-snapshot
add 43bf9b0 Fix a typo in license names
add 41e9e04 Merge pull request #14289 from apache/aaltay-patch-1
add 6c7dff9 Revert "Merge pull request #11767: [BEAM-11648] BigQuery Storage API sink"
add ec6035c Merge pull request #14299: Revert "[BEAM-11648] BigQuery Storage API sink"
add 735b896 Increase Dataflow streaming ValidatesRunner timeout
add d9ef128 Merge pull request #14303: Increase Dataflow streaming ValidatesRunner timeout
add 6075867 Sickbay failing MapState and SetState tests for Dataflow streaming
add eb65c5a Exclude unsupported committed metrics for Dataflow streaming
add 67dc5fe Merge pull request #14302: [BEAM-11886] Sickbay unsupported MapState, SetState, committed metrics for Dataflow streaming
add f218e7a Revert "Always use portable job submission for Dataflow runner v2"
add 19b7a3b Merge pull request #14300: Revert 59906f6 ("Always use portable job submission for Dataflow runner v2")
add e1664bf [BEAM-11862] Updates ExternalTransformFinder to consider external leaf transforms (#14293)
add a57a58d [BEAM-12022] Touch up Go ValidatesRunner docs and remove old gradle tasks.
add 83bd548 Merge pull request #14290: [BEAM-12022] Touch up Go ValidatesRunner docs and remove old gradle tasks
add 26c7980 Fix nullability issues with BeamZetaSqlCalcRel
add a884c8c Fix nullability issues with BeamZetaSqlCalcRel
add 8ff045d fix nullness
add 0a517bb fix findbugs
add 971a1ba fix findbugs
add 12ed35f Merge pull request #14254: [BEAM-10402] Fix nullability issues with BeamZetaSqlCalcRel
add 48096fc Merge pull request #14309: [BEAM-11648] Resubmit Storage API sink with broken test removed.
add 78721ef [BEAM-11837] Make checker framework opt-out rather than opt-in.
add 88fc2ad Merge pull request #14301 from ibzib/BEAM-11837
add 135974e [BEAM-12042] Check TVF argument list size.
add 7bb91fb Merge pull request #14316 from ibzib/BEAM-12042
add fbbd8e6 Revert "Make beam_fn_api, use_runner_v2, and use_unified_worker all trigger the same behavior in the SDK"
add 361b8d9 Merge pull request #14318: BEAM-12041] Revert "Make beam_fn_api, use_runner_v2, and use_unified_worker all trigger the same behavior in the SDK"
add eb2c792 Switch PULL_REQUEST_TEMPLATE to HTML table; verbose but easier to edit
add 430fd92 Add Dataflow Java11 and V2 tests to pull request template
add 2ccdae9 Add ULR tests to pull request template
add 243128a Merge pull request #14305: Add ULR, Dataflow V2 and Java11 Jenkins statuses to PR template
add 377f4b2 [BEAM-11929] Rely on py3.6+ dictionary ordering in beam.Row (#14156)
add ab1dfa1 Move java executable check to avoid --beam_services localhost:port case (#14315)
add cdc36bf ParDoTest cases that should be ValidatesRunner
add 7ffa532 Merge pull request #14313: ParDoTest cases that should be ValidatesRunner
add 2bb36f8 [BEAM-11934] Add runner determined sharding option for unbounded data to WriteFiles (Java) (#14164)
add 92e7f83 Use beam, rather than typing, list type hint.
add 4d99d92 Merge pull request #14331 from robertwb/list-coder
add 00b1991 Exclude unsupported TestStream tests from SamzaRunner
add 5688739 Exclude unsupported TestStream tests from FlinkRunner
add d25c7fb TestStream tests should be ValidatesRunner
add 9e3e326 Merge pull request #14310: TestStream tests should be ValidatesRunner
add a2da64f [BEAM-11935] Updates Dataflow SDK Harness map to set Environment ID (#14189)
add 54c75db [BEAM-11815] AWS SDK v1, DynamoDBIO should read all data with limits
add ce214c0 [BEAM-11815] AWS SDK v2, DynamoDBIO reads all data with limits
add 75460dc Merge pull request #14314: [BEAM-11815] DynamoDBIO should read all data with limits
add 42e4eeb Remove at most one-line of whitespace after Out[]: prompt
add 8da11dc Merge pull request #14232 from TheNeuralBit/prompt-whitespace
add 7563928 [BEAM-7372] remove codepath and workaround for py2 from io (#14292)
add 26030bf More flexible GBK tests.
add e30c349 Merge pull request #14287 from robertwb/gbk-testing
add 2690888 [BEAM-10761] enable strict dependency checking for sdks/java/testing/watermarks
add 999c74d [BEAM-10961] turn unusedDeclared dependencies into runtimeOnly
add 1a30e11 Merge pull request #14195 from sonam-vend/sdks-java-testing-watermarks
add fe1116d [BEAM-12033] Validate casts from double literals to numeric during ex… (#14332)
add b5ed27f Break up Java PostCommit a bit (#14321)
add 09f40fc [BEAM-12023] Use SamzaPipelineRunner instead of SamzaJobInvocation to run portable pipelines (#14291)
add e229c18 [BEAM-11922] remove broken test (#14288)
add a372096 [BEAM-11948] Add deprecation warning for Flink 1.8/1.9 support removal
add ba4bb1f Merge pull request #14214: [BEAM-11948] Add deprecation warning for Flink 1.8/1.9 support removal
add 3b77ff8 [BEAM-12057] Add missing populateDisplayData methods to ParquetIO
add 91382ad Merge pull request #14344: [BEAM-12057] Add missing populateDisplayData methods to ParquetIO
add 875f63f [BEAM-12054] CassandraIO: flush mutator in @FinishBundle
add 475bbbc Merge pull request #14346: [BEAM-12054] CassandraIO: flush mutator in @FinishBundle
add d1f4808 [BEAM-12050] Sickbay portable Flink tests of event time timers with TestStream
add 776bf54 Merge pull request #14333: [BEAM-12050] Sickbay portable Flink tests of event time timers with TestStream
add a62fe3c [BEAM-12044] JdbcIO read: always force autocommit to false
add 1f73a04 Merge pull request #14349: [BEAM-12044] JdbcIO read: always force autocommit to false
add d6375a2 [BEAM-9547] Raise WontImplement(order-sensitive) for operations that impose order (#14324)
add 0e47ea3 [BEAM-12030] DataFrame read_* raise FileNotFound for non-existent input (#14312)
add f8dd311 Add hooks to allow alternative proto message roots. (#14320)
add 8f71af2 More extensive side input reiteration and pagination tests. (#14298)
add 3c62cc9 [cleanup] Remove unused PayloadFormat enum (#14323)
add cd5bb1c Update BigQueryServices.java
add d5563fb Merge pull request #14358 from apache/ihji-patch-2
add 6f6bd63 Update StorageApiWriteUnshardedRecords.java
add 6ae9d4a Update sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java
add ccf5a54 Merge pull request #14359 from apache/ihji-patch-3
add d130cde Update TableDestination.java
add f8afb0d Merge pull request #14360 from apache/ihji-patch-4
add 1b1d8ad [BEAM-12052] Pick the correct Java container for the executed JDK version in cross-language validate runner tests
add 858f7ea Merge pull request #14337 from ihji/BEAM-12052
add 70c11a2 Appends SDK Harness ID to Java container staging path to prevent dependency conflicts
add 4f596b2 Merge pull request #14343: [BEAM-11935] Appends SDK Harness ID to Java container staging path to prevent dependency conflicts
add 816ff00 fix precommit errors
add f11db21 Merge pull request #14367: Fix Java precommit errors
add 956c7bd [BEAM-12030] Fix pylint breakage (#14357)
add 1a7e213 [BEAM-11628] Add initial support for GroupBy.apply (#13843)
add bae76b0 fix google3 import warnings
add 7e746db Merge pull request #14363 from apache/ihji-patch-5
add e92d184 [BEAM-12069] Make mock a test-only dependency (#14328)
add 5a9260f Refactor Dataflow V2 sickbay lists to prepare for portable job submission
add 20fbcf7 Produce portable pipeline proto before applying Dataflow V1 replacements
add c5546b2 Re-enable portable job submission by default
add a16bbf7 Merge pull request #14339: Roll forward PJS with Pubsub fix
new c6caa46 Re-cut 2.29.0 release branch
new 2105fe8 Set Dataflow container version for 2.29.0 release branch
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (85e93ca)
\
N -- N -- N refs/heads/release-2.29.0 (2105fe8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/PULL_REQUEST_TEMPLATE.md | 259 +-
.test-infra/jenkins/CommonTestProperties.groovy | 4 +-
...ommit_CrossLanguageValidatesRunner_Spark.groovy | 2 +-
...mit_CrossLanguageValidatesRunner_Spark3.groovy} | 10 +-
....groovy => job_PostCommit_Java_Dataflow.groovy} | 17 +-
...roovy => job_PostCommit_Java_DataflowV2.groovy} | 17 +-
... => job_PostCommit_Java_Hadoop_Versions.groovy} | 17 +-
...> job_PostCommit_Java_Jpms_Spark_Java11.groovy} | 18 +-
.../job_PostCommit_Java_Nexmark_Spark.groovy | 8 +-
...Java_PortableValidatesRunner_Spark_Batch.groovy | 3 +-
..._PortableValidatesRunner_Spark_Streaming.groovy | 3 +-
..._Java_ValidatesRunner_Dataflow_streaming.groovy | 2 +-
...ob_PostCommit_Java_ValidatesRunner_Spark.groovy | 3 +-
...ValidatesRunner_SparkStructuredStreaming.groovy | 2 +-
...ommit_Java_ValidatesRunner_Spark_Java11.groovy} | 38 +-
...ommit_Python_ValidatesContainer_Dataflow.groovy | 2 +-
CHANGES.md | 32 +-
README.md | 2 +-
build.gradle.kts | 10 +-
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 36 +-
.../apache/beam/gradle/GrpcVendoring_1_36_0.groovy | 199 +
examples/java/build.gradle | 2 +-
.../examples/cookbook/MapClassIntegrationIT.java | 73 -
examples/kotlin/build.gradle | 2 +-
.../notebooks/tour-of-beam/getting-started.ipynb | 29 +-
.../tour-of-beam/reading-and-writing-data.ipynb | 941 ++
.../pipeline/src/main/proto/beam_runner_api.proto | 4 +
release/build.gradle.kts | 2 +-
.../src/main/scripts/build_release_candidate.sh | 38 +-
release/src/main/scripts/choose_rc_commit.sh | 139 +
release/src/main/scripts/cut_release_branch.sh | 15 +-
release/src/main/scripts/mass_comment.py | 1 +
release/src/main/scripts/run_rc_validation.sh | 2 +-
release/src/main/scripts/set_version.sh | 29 +-
runners/core-construction-java/build.gradle | 3 +
runners/flink/flink_runner.gradle | 8 +
runners/flink/job-server/flink_job_server.gradle | 11 +
.../beam/runners/flink/FlinkPipelineRunner.java | 8 +
.../org/apache/beam/runners/flink/FlinkRunner.java | 7 +
runners/google-cloud-dataflow-java/build.gradle | 11 +-
.../beam/runners/dataflow/DataflowRunner.java | 25 +
.../dataflow/options/DataflowPipelineOptions.java | 2 -
.../dataflow/DataflowPipelineTranslatorTest.java | 147 -
.../dataflow/worker/WindmillStateReader.java | 22 +-
.../dataflow/worker/WindmillStateReaderTest.java | 20 +-
runners/samza/build.gradle | 4 +
.../beam/runners/samza/SamzaJobInvocation.java | 114 -
.../beam/runners/samza/SamzaJobServerDriver.java | 14 +-
runners/spark/2/build.gradle | 35 +
.../job-server/build.gradle} | 23 +-
.../job-server/container/build.gradle} | 15 +-
.../SparkStructuredStreamingPipelineOptions.java | 0
.../SparkStructuredStreamingPipelineResult.java | 0
.../SparkStructuredStreamingRunner.java | 0
.../SparkStructuredStreamingRunnerRegistrar.java | 0
.../aggregators/AggregatorsAccumulator.java | 0
.../aggregators/NamedAggregators.java | 0
.../aggregators/NamedAggregatorsAccumulator.java | 0
.../aggregators/package-info.java | 0
.../structuredstreaming/examples/WordCount.java | 0
.../metrics/AggregatorMetric.java | 0
.../metrics/AggregatorMetricSource.java | 0
.../metrics/CompositeSource.java | 0
.../metrics/MetricsAccumulator.java | 0
.../MetricsContainerStepMapAccumulator.java | 0
.../metrics/SparkBeamMetric.java | 0
.../metrics/SparkBeamMetricSource.java | 0
.../metrics/SparkMetricsContainerStepMap.java | 0
.../metrics/WithMetricsSupport.java | 0
.../structuredstreaming/metrics/package-info.java | 0
.../metrics/sink/CodahaleCsvSink.java | 0
.../metrics/sink/CodahaleGraphiteSink.java | 0
.../metrics/sink/package-info.java | 0
.../spark/structuredstreaming/package-info.java | 0
.../translation/PipelineTranslator.java | 0
.../translation/SchemaHelpers.java | 0
.../translation/SparkTransformOverrides.java | 0
.../translation/TransformTranslator.java | 0
.../translation/TranslationContext.java | 0
.../translation/batch/AggregatorCombiner.java | 0
.../batch/CombinePerKeyTranslatorBatch.java | 0
.../CreatePCollectionViewTranslatorBatch.java | 0
.../translation/batch/DatasetSourceBatch.java | 0
.../translation/batch/DoFnFunction.java | 0
.../translation/batch/DoFnRunnerWithMetrics.java | 0
.../translation/batch/FlattenTranslatorBatch.java | 0
.../batch/GroupByKeyTranslatorBatch.java | 0
.../translation/batch/ImpulseTranslatorBatch.java | 0
.../translation/batch/ParDoTranslatorBatch.java | 0
.../translation/batch/PipelineTranslatorBatch.java | 0
.../translation/batch/ProcessContext.java | 0
.../batch/ReadSourceTranslatorBatch.java | 0
.../batch/ReshuffleTranslatorBatch.java | 0
.../batch/WindowAssignTranslatorBatch.java | 0
.../GroupAlsoByWindowViaOutputBufferFn.java | 0
.../batch/functions/NoOpStepContext.java | 0
.../batch/functions/SparkSideInputReader.java | 0
.../translation/batch/functions/package-info.java | 0
.../translation/batch/package-info.java | 0
.../translation/helpers/CoderHelpers.java | 0
.../translation/helpers/EncoderHelpers.java | 0
.../translation/helpers/KVHelpers.java | 0
.../translation/helpers/MultiOuputCoder.java | 0
.../translation/helpers/RowHelpers.java | 0
.../translation/helpers/SideInputBroadcast.java | 0
.../translation/helpers/WindowingHelpers.java | 0
.../translation/helpers/package-info.java | 0
.../translation/package-info.java | 0
.../streaming/DatasetSourceStreaming.java | 0
.../streaming/PipelineTranslatorStreaming.java | 0
.../streaming/ReadSourceTranslatorStreaming.java | 0
.../translation/streaming/package-info.java | 0
.../translation/utils/CachedSideInputReader.java | 0
.../translation/utils/SideInputStorage.java | 0
.../translation/utils/package-info.java | 0
...parkStructuredStreamingRunnerRegistrarTest.java | 0
.../StructuredStreamingPipelineStateTest.java | 0
.../aggregators/metrics/sink/InMemoryMetrics.java | 0
.../metrics/sink/InMemoryMetricsSinkRule.java | 0
.../metrics/sink/SparkMetricsSinkTest.java | 0
.../metrics/BeamMetricTest.java | 0
.../translation/batch/CombineTest.java | 0
.../translation/batch/ComplexSourceTest.java | 0
.../translation/batch/FlattenTest.java | 0
.../translation/batch/GroupByKeyTest.java | 0
.../translation/batch/ParDoTest.java | 0
.../translation/batch/SimpleSourceTest.java | 0
.../translation/batch/WindowAssignTest.java | 0
.../translation/helpers/EncoderHelpersTest.java | 0
.../translation/streaming/SimpleSourceTest.java | 0
.../utils/SerializationDebugger.java | 0
.../structuredstreaming/utils/package-info.java | 0
runners/spark/3/build.gradle | 35 +
.../job-server/build.gradle} | 23 +-
.../job-server/container/build.gradle} | 15 +-
...ld.gradle => spark_job_server_container.gradle} | 10 +-
.../{build.gradle => spark_job_server.gradle} | 1 +
.../spark/{build.gradle => spark_runner.gradle} | 100 +-
sdks/go/pkg/beam/core/graph/coder/map.go | 30 +-
sdks/go/pkg/beam/core/graph/coder/map_test.go | 27 +-
sdks/go/pkg/beam/core/graph/coder/row_decoder.go | 43 +-
sdks/go/pkg/beam/core/graph/coder/row_encoder.go | 40 +-
sdks/go/pkg/beam/core/graph/coder/row_test.go | 60 +-
sdks/go/pkg/beam/core/runtime/graphx/coder_test.go | 4 +
.../core/runtime/graphx/schema/logicaltypes.go | 3 +
.../pkg/beam/core/runtime/graphx/schema/schema.go | 328 +-
.../beam/core/runtime/graphx/schema/schema_test.go | 315 +-
sdks/go/test/build.gradle | 97 +-
sdks/go/test/run_integration_tests.sh | 239 -
.../src/main/resources/beam/spotbugs-filter.xml | 3 +
sdks/java/container/boot.go | 7 +-
.../container/license_scripts/dep_urls_java.yaml | 2 +
.../license_scripts/pull_licenses_java.py | 2 +-
.../java/org/apache/beam/sdk/io/WriteFiles.java | 216 +-
.../org/apache/beam/sdk/io/WriteFilesTest.java | 78 +-
.../apache/beam/sdk/testing/TestStreamTest.java | 14 +-
.../apache/beam/sdk/transforms/GroupByKeyTest.java | 4 +-
.../org/apache/beam/sdk/transforms/ParDoTest.java | 24 +-
.../extensions/gcp/auth/GcpCredentialFactory.java | 2 +
.../beam/sdk/extensions/sql/SqlTransform.java | 1 -
.../beam/sdk/extensions/sql/impl/BeamSqlEnv.java | 37 +-
.../extensions/sql/impl/CalciteQueryPlanner.java | 15 +
.../extensions/sql/impl/planner/BeamRuleSets.java | 8 +-
.../sql/impl/rule/BeamCalcMergeRule.java | 41 +
.../sql/impl/rule/LogicalCalcMergeRule.java | 41 +
.../meta/provider/kafka/KafkaTableProviderIT.java | 27 +-
.../provider/pubsub/PubsubTableProviderIT.java | 156 +-
.../sql/zetasql/BeamZetaSqlCalcMergeRule.java | 44 +
.../extensions/sql/zetasql/BeamZetaSqlCalcRel.java | 68 +-
.../sql/zetasql/ZetaSQLQueryPlanner.java | 4 +-
.../sql/zetasql/ZetaSqlBeamTranslationUtils.java | 3 +-
.../zetasql/ZetaSqlCalciteTranslationUtils.java | 8 +
.../zetasql/translation/ExpressionConverter.java | 30 +-
.../sql/zetasql/translation/TVFScanConverter.java | 3 +-
.../extensions/sql/zetasql/ZetaSqlJavaUdfTest.java | 15 +-
.../sql/zetasql/ZetaSqlMathFunctionsTest.java | 11 +-
.../sql/zetasql/ZetaSqlNativeUdfTest.java | 21 +
.../sql/zetasql/ZetaSqlNumberTypesTest.java | 84 +
.../extensions/sql/zetasql/ZetaSqlTypesUtils.java | 5 -
.../io/{kinesis => amazon-web-services}/OWNERS | 0
.../beam/sdk/io/aws/dynamodb/DynamoDBIO.java | 17 +-
.../beam/sdk/io/aws/dynamodb/DynamoDBIOTest.java | 34 +
.../io/{kinesis => amazon-web-services2}/OWNERS | 0
sdks/java/io/amazon-web-services2/build.gradle | 10 +-
.../beam/sdk/io/aws2/dynamodb/DynamoDBIO.java | 23 +-
.../beam/sdk/io/aws2/dynamodb/DynamoDBIOTest.java | 41 +
.../apache/beam/sdk/io/cassandra/CassandraIO.java | 14 +-
sdks/java/io/clickhouse/build.gradle | 2 +-
sdks/java/io/google-cloud-platform/build.gradle | 3 +-
.../beam/sdk/io/gcp/bigquery/BatchLoads.java | 9 +-
.../beam/sdk/io/gcp/bigquery/BigQueryIO.java | 130 +-
.../beam/sdk/io/gcp/bigquery/BigQueryOptions.java | 19 +
.../beam/sdk/io/gcp/bigquery/BigQueryServices.java | 49 +
.../sdk/io/gcp/bigquery/BigQueryServicesImpl.java | 142 +-
.../beam/sdk/io/gcp/bigquery/BigQueryUtils.java | 4 +-
.../beam/sdk/io/gcp/bigquery/RetryManager.java | 283 +
.../bigquery/StorageApiDynamicDestinations.java | 78 +
.../StorageApiDynamicDestinationsBeamRow.java | 65 +
.../StorageApiDynamicDestinationsTableRow.java | 71 +
.../gcp/bigquery/StorageApiFinalizeWritesDoFn.java | 171 +
.../bigquery/StorageApiFlushAndFinalizeDoFn.java | 207 +
.../beam/sdk/io/gcp/bigquery/StorageApiLoads.java | 157 +
.../bigquery/StorageApiWriteUnshardedRecords.java | 323 +
.../bigquery/StorageApiWritesShardedRecords.java | 524 +
.../beam/sdk/io/gcp/bigquery/TableDestination.java | 9 +
.../io/gcp/bigquery/TableRowToStorageApiProto.java | 9 +-
.../bigquery/TwoLevelMessageConverterCache.java | 68 +
.../apache/beam/sdk/io/gcp/healthcare/FhirIO.java | 66 +-
.../sdk/io/gcp/healthcare/FhirSearchParameter.java | 14 +
...bsubMessage.java => AddTimestampAttribute.java} | 39 +-
.../beam/sdk/io/gcp/pubsub/NestedRowToMessage.java | 91 +
.../beam/sdk/io/gcp/pubsub/PubsubMessage.java | 55 +-
.../beam/sdk/io/gcp/pubsub/PubsubMessageToRow.java | 50 +-
.../sdk/io/gcp/pubsub/PubsubSchemaIOProvider.java | 92 +-
.../sdk/io/gcp/testing/FakeDatasetService.java | 171 +-
.../apache/beam/sdk/io/gcp/GcpApiSurfaceTest.java | 2 +
.../sdk/io/gcp/bigquery/BigQueryIOWriteTest.java | 279 +-
.../io/gcp/bigquery/BigQueryServicesImplTest.java | 52 +-
.../beam/sdk/io/gcp/bigquery/BigQueryUtilTest.java | 4 +-
.../beam/sdk/io/gcp/bigquery/RetryManagerTest.java | 212 +
.../beam/sdk/io/gcp/healthcare/FhirIOLROIT.java | 2 -
.../beam/sdk/io/gcp/healthcare/FhirIOWriteIT.java | 2 -
.../sdk/io/gcp/pubsub/NestedRowToMessageTest.java | 106 +
.../sdk/io/gcp/pubsub/PubsubMessageToRowTest.java | 58 +-
sdks/java/io/hadoop-format/build.gradle | 2 +-
.../java/org/apache/beam/sdk/io/jdbc/JdbcIO.java | 4 +
.../beam/sdk/io/kafka/KafkaUnboundedSource.java | 7 +-
.../org/apache/beam/sdk/io/kafka/KafkaIOTest.java | 26 +
sdks/java/io/kinesis/build.gradle | 2 +-
.../org/apache/beam/sdk/io/parquet/ParquetIO.java | 306 +-
.../apache/beam/sdk/io/parquet/ParquetIOTest.java | 35 +-
.../src/test/resources/log4j-test.properties | 31 +
sdks/java/maven-archetypes/examples/build.gradle | 6 +-
.../maven-archetypes/gcp-bom-examples/build.gradle | 6 +-
sdks/java/maven-archetypes/starter/build.gradle | 7 +-
sdks/java/testing/jpms-tests/build.gradle | 8 +
sdks/java/testing/load-tests/build.gradle | 2 +-
sdks/java/testing/nexmark/build.gradle | 4 +-
sdks/java/testing/watermarks/build.gradle | 17 +-
sdks/python/apache_beam/coders/coder_impl.py | 4 +-
sdks/python/apache_beam/coders/coders.py | 36 +-
sdks/python/apache_beam/coders/coders_test.py | 4 -
.../apache_beam/coders/coders_test_common.py | 9 +
sdks/python/apache_beam/coders/stream.pyx | 2 +
sdks/python/apache_beam/coders/typecoders.py | 6 +-
sdks/python/apache_beam/coders/typecoders_test.py | 4 -
sdks/python/apache_beam/dataframe/doctests.py | 4 +-
sdks/python/apache_beam/dataframe/expressions.py | 119 +-
.../apache_beam/dataframe/expressions_test.py | 68 +
sdks/python/apache_beam/dataframe/frame_base.py | 18 +-
sdks/python/apache_beam/dataframe/frames.py | 512 +-
sdks/python/apache_beam/dataframe/frames_test.py | 107 +-
sdks/python/apache_beam/dataframe/io.py | 12 +-
sdks/python/apache_beam/dataframe/io_test.py | 5 +
.../apache_beam/dataframe/pandas_doctests_test.py | 130 +-
.../dataframe/pandas_top_level_functions.py | 14 +-
sdks/python/apache_beam/dataframe/partitionings.py | 70 +-
.../apache_beam/dataframe/partitionings_test.py | 16 +-
sdks/python/apache_beam/dataframe/schemas_test.py | 19 +
sdks/python/apache_beam/dataframe/transforms.py | 67 +-
.../apache_beam/dataframe/transforms_test.py | 51 +-
.../apache_beam/examples/snippets/snippets_test.py | 4 +-
sdks/python/apache_beam/internal/metrics/cells.py | 8 -
sdks/python/apache_beam/internal/util.py | 4 -
sdks/python/apache_beam/io/avroio_test.py | 17 +-
sdks/python/apache_beam/io/concat_source_test.py | 4 -
sdks/python/apache_beam/io/filebasedsink.py | 4 -
sdks/python/apache_beam/io/filebasedsink_test.py | 7 -
sdks/python/apache_beam/io/filebasedsource_test.py | 19 -
sdks/python/apache_beam/io/fileio.py | 64 +-
sdks/python/apache_beam/io/fileio_test.py | 6 +-
sdks/python/apache_beam/io/filesystem.py | 4 -
sdks/python/apache_beam/io/filesystems_test.py | 7 -
sdks/python/apache_beam/io/gcp/__init__.py | 29 +-
.../apache_beam/io/gcp/bigquery_avro_tools_test.py | 53 +-
.../apache_beam/io/gcp/bigquery_file_loads_test.py | 3 -
.../apache_beam/io/gcp/bigquery_read_internal.py | 3 +-
sdks/python/apache_beam/io/gcp/bigquery_tools.py | 10 +-
.../apache_beam/io/gcp/dicomio_integration_test.py | 6 +-
sdks/python/apache_beam/io/gcp/gcsio.py | 3 +-
sdks/python/apache_beam/io/gcp/gcsio_test.py | 6 +-
sdks/python/apache_beam/io/gcp/pubsub.py | 4 -
.../apache_beam/io/gcp/tests/bigquery_matcher.py | 6 +-
.../io/gcp/tests/bigquery_matcher_test.py | 9 +-
.../io/gcp/tests/pubsub_matcher_test.py | 7 -
.../python/apache_beam/io/hadoopfilesystem_test.py | 11 -
.../hdfs_integration_test/hdfs_integration_test.sh | 2 +-
sdks/python/apache_beam/io/localfilesystem.py | 4 +-
sdks/python/apache_beam/io/localfilesystem_test.py | 7 -
sdks/python/apache_beam/io/mongodbio.py | 28 +-
sdks/python/apache_beam/io/mongodbio_test.py | 48 +-
sdks/python/apache_beam/io/parquetio_it_test.py | 7 -
sdks/python/apache_beam/io/parquetio_test.py | 7 -
sdks/python/apache_beam/io/restriction_trackers.py | 4 -
.../apache_beam/io/source_test_utils_test.py | 7 -
sdks/python/apache_beam/io/sources_test.py | 7 -
sdks/python/apache_beam/io/textio_test.py | 13 -
sdks/python/apache_beam/io/tfrecordio_test.py | 9 +-
sdks/python/apache_beam/io/vcfio.py | 511 -
sdks/python/apache_beam/io/vcfio_test.py | 635 --
sdks/python/apache_beam/metrics/cells.py | 22 +-
sdks/python/apache_beam/metrics/execution.py | 13 +-
sdks/python/apache_beam/metrics/metricbase.py | 4 -
.../python/apache_beam/options/pipeline_options.py | 7 +-
sdks/python/apache_beam/options/value_provider.py | 4 -
sdks/python/apache_beam/pvalue.py | 22 +-
sdks/python/apache_beam/runners/common.py | 9 +-
.../runners/dataflow/internal/apiclient.py | 61 +-
.../runners/dataflow/internal/apiclient_test.py | 89 +-
.../apache_beam/runners/dataflow/internal/names.py | 4 +-
.../runners/interactive/background_caching_job.py | 1 +
.../runners/interactive/caching/cacheable.py | 78 +
.../runners/interactive/caching/streaming_cache.py | 4 +-
.../interactive/caching/streaming_cache_test.py | 2 +-
.../runners/interactive/interactive_runner.py | 2 +-
.../runners/interactive/pipeline_instrument.py | 119 +-
.../interactive/pipeline_instrument_test.py | 4 +
.../apache_beam/runners/interactive/utils_test.py | 56 +-
.../runners/portability/spark_runner.py | 4 +-
.../runners/portability/spark_runner_test.py | 2 +-
.../portability/spark_uber_jar_job_server.py | 4 +-
.../runners/worker/statesampler_fast.pyx | 1 +
.../testing/benchmarks/chicago_taxi/preprocess.py | 4 +-
.../benchmarks/chicago_taxi/trainer/taxi.py | 5 +-
.../apache_beam/testing/data/vcf/valid-4.0.vcf | 23 -
.../apache_beam/testing/data/vcf/valid-4.0.vcf.bz2 | Bin 781 -> 0 bytes
.../apache_beam/testing/data/vcf/valid-4.0.vcf.gz | Bin 727 -> 0 bytes
.../testing/data/vcf/valid-4.1-large.vcf | 10000 -------------------
.../testing/data/vcf/valid-4.1-large.vcf.gz | Bin 156715 -> 0 bytes
.../apache_beam/testing/data/vcf/valid-4.2.vcf | 42 -
.../apache_beam/testing/data/vcf/valid-4.2.vcf.gz | Bin 1240 -> 0 bytes
sdks/python/apache_beam/testing/test_stream.py | 4 -
sdks/python/apache_beam/testing/util.py | 4 -
sdks/python/apache_beam/transforms/core.py | 8 -
sdks/python/apache_beam/transforms/cy_combiners.py | 4 -
.../cy_dataflow_distribution_counter.pyx | 1 +
sdks/python/apache_beam/transforms/display.py | 4 -
sdks/python/apache_beam/transforms/environments.py | 32 +-
.../apache_beam/transforms/ptransform_test.py | 3 +
.../apache_beam/transforms/sideinputs_test.py | 28 +
sdks/python/apache_beam/transforms/stats.py | 2 +
sdks/python/apache_beam/transforms/trigger.py | 8 -
sdks/python/apache_beam/transforms/window.py | 18 -
sdks/python/apache_beam/typehints/decorators.py | 3 -
sdks/python/apache_beam/typehints/row_type.py | 4 -
.../apache_beam/typehints/trivial_inference.py | 8 -
sdks/python/apache_beam/typehints/typehints.py | 4 -
sdks/python/apache_beam/utils/counters.py | 1 +
sdks/python/apache_beam/utils/proto_utils.py | 2 +
sdks/python/apache_beam/utils/subprocess_server.py | 4 +
sdks/python/apache_beam/utils/timestamp.py | 11 -
sdks/python/apache_beam/utils/windowed_value.py | 14 +-
.../container/license_scripts/dep_urls_py.yaml | 2 +
sdks/python/scripts/generate_pydoc.sh | 1 +
sdks/python/scripts/run_pylint.sh | 1 -
sdks/python/setup.py | 7 +-
sdks/python/test-suites/portable/common.gradle | 10 +-
settings.gradle.kts | 10 +-
vendor/README.md | 46 +-
vendor/grpc-1_36_0/build.gradle | 37 +
website/www/site/content/en/contribute/_index.md | 13 +-
.../site/content/en/contribute/release-guide.md | 173 +-
.../site/content/en/documentation/runners/spark.md | 4 +-
.../en/documentation/runtime/environments.md | 4 +
.../en/documentation/sdks/java/testing/nexmark.md | 4 +-
.../site/content/en/get-started/quickstart-go.md | 2 +-
.../site/content/en/get-started/tour-of-beam.md | 9 +
.../site/content/en/powered-by/open-source/kio.md | 44 +
website/www/site/data/io_matrix.yaml | 6 -
.../www/site/static/images/cut-release-branch.png | Bin 0 -> 34625 bytes
.../site/static/images/logos/powered-by/kio.png | Bin 0 -> 89139 bytes
website/www/site/static/images/tag-rc-commit.png | Bin 0 -> 24964 bytes
372 files changed, 9046 insertions(+), 13938 deletions(-)
copy .test-infra/jenkins/{job_PostCommit_CrossLanguageValidatesRunner_Spark.groovy => job_PostCommit_CrossLanguageValidatesRunner_Spark3.groovy} (83%)
copy .test-infra/jenkins/{job_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.groovy => job_PostCommit_Java_Dataflow.groovy} (70%)
copy .test-infra/jenkins/{job_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.groovy => job_PostCommit_Java_DataflowV2.groovy} (70%)
copy .test-infra/jenkins/{job_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.groovy => job_PostCommit_Java_Hadoop_Versions.groovy} (70%)
copy .test-infra/jenkins/{job_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming.groovy => job_PostCommit_Java_Jpms_Spark_Java11.groovy} (66%)
copy .test-infra/jenkins/{job_PostCommit_Java_ValidatesRunner_Dataflow_streaming.groovy => job_PostCommit_Java_ValidatesRunner_Spark_Java11.groovy} (57%)
create mode 100644 buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_36_0.groovy
delete mode 100644 examples/java/src/test/java/org/apache/beam/examples/cookbook/MapClassIntegrationIT.java
create mode 100644 examples/notebooks/tour-of-beam/reading-and-writing-data.ipynb
create mode 100644 release/src/main/scripts/choose_rc_commit.sh
delete mode 100644 runners/samza/src/main/java/org/apache/beam/runners/samza/SamzaJobInvocation.java
create mode 100644 runners/spark/2/build.gradle
copy runners/spark/{src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/InMemoryMetricsSinkRule.java => 2/job-server/build.gradle} (52%)
copy runners/spark/{src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/package-info.java => 2/job-server/container/build.gradle} (63%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingPipelineOptions.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingPipelineResult.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingRunner.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingRunnerRegistrar.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/AggregatorsAccumulator.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/NamedAggregators.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/NamedAggregatorsAccumulator.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/examples/WordCount.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/AggregatorMetric.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/AggregatorMetricSource.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/CompositeSource.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/MetricsAccumulator.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/MetricsContainerStepMapAccumulator.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkBeamMetric.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkBeamMetricSource.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkMetricsContainerStepMap.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/WithMetricsSupport.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/sink/CodahaleCsvSink.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/sink/CodahaleGraphiteSink.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/sink/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/PipelineTranslator.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/SchemaHelpers.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/SparkTransformOverrides.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/TransformTranslator.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/TranslationContext.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/AggregatorCombiner.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombinePerKeyTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CreatePCollectionViewTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DatasetSourceBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnFunction.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/DoFnRunnerWithMetrics.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/FlattenTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/GroupByKeyTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ImpulseTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ParDoTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/PipelineTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ProcessContext.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ReadSourceTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ReshuffleTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/WindowAssignTranslatorBatch.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/GroupAlsoByWindowViaOutputBufferFn.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/NoOpStepContext.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/SparkSideInputReader.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/functions/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/CoderHelpers.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/EncoderHelpers.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/KVHelpers.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/MultiOuputCoder.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/RowHelpers.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/SideInputBroadcast.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/WindowingHelpers.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/package-info.java (100%)
copy runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/DatasetSourceStreaming.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/PipelineTranslatorStreaming.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/ReadSourceTranslatorStreaming.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/package-info.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/CachedSideInputReader.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/SideInputStorage.java (100%)
rename runners/spark/{ => 2}/src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/utils/package-info.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/SparkStructuredStreamingRunnerRegistrarTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/StructuredStreamingPipelineStateTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/InMemoryMetrics.java (100%)
copy runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/InMemoryMetricsSinkRule.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/SparkMetricsSinkTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/metrics/BeamMetricTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/CombineTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ComplexSourceTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/FlattenTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/GroupByKeyTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/ParDoTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/SimpleSourceTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/batch/WindowAssignTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/helpers/EncoderHelpersTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/translation/streaming/SimpleSourceTest.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/utils/SerializationDebugger.java (100%)
rename runners/spark/{ => 2}/src/test/java/org/apache/beam/runners/spark/structuredstreaming/utils/package-info.java (100%)
create mode 100644 runners/spark/3/build.gradle
rename runners/spark/{src/test/java/org/apache/beam/runners/spark/structuredstreaming/aggregators/metrics/sink/InMemoryMetricsSinkRule.java => 3/job-server/build.gradle} (52%)
rename runners/spark/{src/main/java/org/apache/beam/runners/spark/structuredstreaming/translation/package-info.java => 3/job-server/container/build.gradle} (63%)
rename runners/spark/job-server/container/{build.gradle => spark_job_server_container.gradle} (87%)
rename runners/spark/job-server/{build.gradle => spark_job_server.gradle} (99%)
rename runners/spark/{build.gradle => spark_runner.gradle} (78%)
delete mode 100755 sdks/go/test/run_integration_tests.sh
create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/BeamCalcMergeRule.java
create mode 100644 sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rule/LogicalCalcMergeRule.java
create mode 100644 sdks/java/extensions/sql/zetasql/src/main/java/org/apache/beam/sdk/extensions/sql/zetasql/BeamZetaSqlCalcMergeRule.java
create mode 100644 sdks/java/extensions/sql/zetasql/src/test/java/org/apache/beam/sdk/extensions/sql/zetasql/ZetaSqlNumberTypesTest.java
copy sdks/java/io/{kinesis => amazon-web-services}/OWNERS (100%)
copy sdks/java/io/{kinesis => amazon-web-services2}/OWNERS (100%)
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/RetryManager.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinations.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsBeamRow.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiDynamicDestinationsTableRow.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiFinalizeWritesDoFn.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiFlushAndFinalizeDoFn.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiLoads.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWriteUnshardedRecords.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/StorageApiWritesShardedRecords.java
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TwoLevelMessageConverterCache.java
rename sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/{RowToPubsubMessage.java => AddTimestampAttribute.java} (58%)
create mode 100644 sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/NestedRowToMessage.java
create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigquery/RetryManagerTest.java
create mode 100644 sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/pubsub/NestedRowToMessageTest.java
create mode 100644 sdks/java/io/parquet/src/test/resources/log4j-test.properties
delete mode 100644 sdks/python/apache_beam/io/vcfio.py
delete mode 100644 sdks/python/apache_beam/io/vcfio_test.py
create mode 100644 sdks/python/apache_beam/runners/interactive/caching/cacheable.py
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.0.vcf
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.0.vcf.bz2
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.0.vcf.gz
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.1-large.vcf
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.1-large.vcf.gz
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.2.vcf
delete mode 100644 sdks/python/apache_beam/testing/data/vcf/valid-4.2.vcf.gz
create mode 100644 vendor/grpc-1_36_0/build.gradle
create mode 100644 website/www/site/content/en/powered-by/open-source/kio.md
create mode 100644 website/www/site/static/images/cut-release-branch.png
create mode 100644 website/www/site/static/images/logos/powered-by/kio.png
create mode 100644 website/www/site/static/images/tag-rc-commit.png
[beam] 02/02: Set Dataflow container version for 2.29.0 release
branch
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenn pushed a commit to branch release-2.29.0
in repository https://gitbox.apache.org/repos/asf/beam.git
commit 2105fe8ba041974e097fa5e1d969eeafe210d283
Author: Kenneth Knowles <kl...@google.com>
AuthorDate: Thu Mar 11 19:48:42 2021 -0800
Set Dataflow container version for 2.29.0 release branch
---
runners/google-cloud-dataflow-java/build.gradle | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/runners/google-cloud-dataflow-java/build.gradle b/runners/google-cloud-dataflow-java/build.gradle
index 290ea94..8e8d71d 100644
--- a/runners/google-cloud-dataflow-java/build.gradle
+++ b/runners/google-cloud-dataflow-java/build.gradle
@@ -45,7 +45,7 @@ processResources {
filter org.apache.tools.ant.filters.ReplaceTokens, tokens: [
'dataflow.legacy_environment_major_version' : '8',
'dataflow.fnapi_environment_major_version' : '8',
- 'dataflow.container_version' : 'beam-master-20210310',
+ 'dataflow.container_version' : 'beam-2.29.0',
]
}
[beam] 01/02: Re-cut 2.29.0 release branch
Posted by ke...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
kenn pushed a commit to branch release-2.29.0
in repository https://gitbox.apache.org/repos/asf/beam.git
commit c6caa46c829fc63c704bb8c55023282c9b21b347
Author: Kenneth Knowles <kl...@google.com>
AuthorDate: Tue Mar 30 11:52:33 2021 -0700
Re-cut 2.29.0 release branch
---
.../src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy | 2 +-
gradle.properties | 4 ++--
sdks/go/pkg/beam/core/core.go | 2 +-
sdks/python/apache_beam/version.py | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
index 475a054..fcd8923 100644
--- a/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
+++ b/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
@@ -352,7 +352,7 @@ class BeamModulePlugin implements Plugin<Project> {
// Automatically use the official release version if we are performing a release
// otherwise append '-SNAPSHOT'
- project.version = '2.30.0'
+ project.version = '2.29.0'
if (!isRelease(project)) {
project.version += '-SNAPSHOT'
}
diff --git a/gradle.properties b/gradle.properties
index 364445a..26c4df7 100644
--- a/gradle.properties
+++ b/gradle.properties
@@ -24,8 +24,8 @@ offlineRepositoryRoot=offline-repository
signing.gnupg.executable=gpg
signing.gnupg.useLegacyGpg=true
-version=2.30.0-SNAPSHOT
-sdk_version=2.30.0.dev
+version=2.29.0-SNAPSHOT
+sdk_version=2.29.0.dev
javaVersion=1.8
diff --git a/sdks/go/pkg/beam/core/core.go b/sdks/go/pkg/beam/core/core.go
index baed22a..64feee3 100644
--- a/sdks/go/pkg/beam/core/core.go
+++ b/sdks/go/pkg/beam/core/core.go
@@ -27,5 +27,5 @@ const (
// SdkName is the human readable name of the SDK for UserAgents.
SdkName = "Apache Beam SDK for Go"
// SdkVersion is the current version of the SDK.
- SdkVersion = "2.30.0.dev"
+ SdkVersion = "2.29.0.dev"
)
diff --git a/sdks/python/apache_beam/version.py b/sdks/python/apache_beam/version.py
index 91b417e..4e6b7a0 100644
--- a/sdks/python/apache_beam/version.py
+++ b/sdks/python/apache_beam/version.py
@@ -17,4 +17,4 @@
"""Apache Beam SDK version information and utilities."""
-__version__ = '2.30.0.dev'
+__version__ = '2.29.0.dev'