You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by gi...@apache.org on 2020/07/29 02:46:42 UTC
[beam] tag nightly-master updated (8d460db -> 780cc4f)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to tag nightly-master
in repository https://gitbox.apache.org/repos/asf/beam.git.
*** WARNING: tag nightly-master was modified! ***
from 8d460db (commit)
to 780cc4f (commit)
from 8d460db Implement Numbering functions (#12375)
add 1184987 [BEAM-10306] Add latency measurements to Python Flink ParDo load test
add 716bba4 [BEAM-10306] Add latency measurements to Python Flink GroupByKey load test
add efaa35f Merge pull request #12065: [BEAM-10306] Add latency measurements to Python Flink load test
add 22822b8 [BEAM-9702] Update Java KinesisIO to support AWS SDK v2 (#11318)
add bac13f8 [BEAM-10559] Add some comments and clean up SQL example. (#12355)
add 9458f30 Fix code that was tripping up the linter.
add 57767bd Remove trailing whitespace.
add e39294d Merge pull request #12382 Fix lint error and whitespace for import.
add c8289dc Initiate Go module
add 23e9f9f Rename Introduction-> introduction
add e736175 Fix go.mod vanity URL
add 7d06c5c Rename Core Transforms -> core_transforms
add 221bb49 Rename core_transforms/{Map -> map}
add 0230e0b Rename core_transforms/map/{ParDo -> pardo}
add b446aed Rename core_transforms/map/{ParDo OneToMany -> pardo_onetomany}
add aa32c3d Rename core_transforms/map/{ParDo Struct -> pardo_struct}
add eecfdf0 Rename core_transforms/{GroupByKey/GroupByKey -> groupbykey/groupbykey}
add b751195 Rename core_transforms/{CoGroupByKey/CoGroupByKey -> cogroupbykey/cogroupbykey}
add 8f0b102 Combine/Simple Function => combine/simple_function
add f7d92c1 Rename Flatten -> flatten
add 42bf190 Rename Partition -> partition
add 429f2b0 Rename {Side Input -> side_input}
add c9210f5 Rename {Additional Outputs -> additional_outputs}
add c0322e1 Rename {Branching -> branching}
add dacee73 Rename CombineFn -> combinefn
add c0acd60 Rename {Combine Per Key -> combine_perkey}
add ecb6b3e Rename {GroupByKey -> groupbykey}
add 238356d Update stepik
add 947c9f3 [Beam-10428] Consolidate Go SDK katas as a single module
add 40672a1 removing slack token credentials binding from all CI jobs except the one that uses it
add 6999c9f Merge pull request #12336 from damgad/damgad_jenkins_slack_credentials_cleanup
add 8930694 [BEAM-7390] Add groupintobatches code snippets
add d252745 Merge pull request #12377 from davidcavazos/groupintobatches-docs
add 9cd6b2f [BEAM-7390] Add groupbykey code snippets (#12370)
add 956e4eb [BEAM-10398] Use GitHub Actions in wheels release process for Python (#12150)
add af7fdd1 [BEAM-10552] Update protobuf package to 3.12.2 (#12334)
add d3f902e Fix spotless groovy formatting.
add 8e48821 Merge pull request #12386 from ibzib/spotless
add 747da91e [BEAM-9702] Update CHANGES.md
add a52d6b8 Merge pull request #12379: [BEAM-9702] Add support of AWS SDK v2 for KinesisIO.Read (Java) to CHANGES.md
add 96a9db3 [BEAM-8648] Deprecate OutputHints from Euphoria API.
add 3a3d162 Merge pull request #12388: [BEAM-8648] Deprecate OutputHints from Euphoria API
add fcd1692 [BEAM-10294] using SparkMetricsContainerStepMap for readable metrics presentation in Spark history server UI (#12063)
add 2fbc857 [BEAM-9976] Increase timeout for FlinkSavepointTest
add 44fd321 [BEAM-9976] Add a check to FlinkSavepointTest to ensure no job is running
add 8f2468c [BEAM-9976] Remove legacy workaround to retrieve rest address
add 9e76cc7 Merge pull request #12378: [BEAM-9976] Increase timeout and improve robustness of FlinkSavepointTest
add 57a5b93 Add log4j configuration to sorter extension
add 60d7249 Eliminate null errors from :sdks:java:extensions:sorter and enable checker
add ce2ae52 Merge pull request #12349: [BEAM-10561] Eliminate nullability issues from :sdks:java:extensions:sorter and enable checker
add 5606867 [BEAM-10580] Eliminate nullability errors from :sdks:java:extensions:sql:expansion-service (#12374)
add 87b97c3 [Beam-10563] Eliminate nullability errors from :sdks:java:extensions:kryo (#12373)
add 0ba300f [BEAM-10562]Eliminate nullability errors from :sdks:java:extensions:join-library (#12351)
add 02092dc [BEAM-9865] Cleanup Jenkins WS on successful jobs (#12326)
add dadf190 Build Py3.8 docker containers.
add b0748a5 Correct docs tox task.
add 0aa52d1 Correct the commit hash used in docs generation.
add 0e99293 Exit gracefully when docs PR was created on previous RC.
add 990d632 Add Py38 Postcommits to the list of suites to trigger.
add 2d2a427 Removes unclear instruction.
add 41ec8b1 Fix various issues in release instructions and scripts observed while preparing 2.23.0. #12384
add 19f700d Disable checker plugin for Java 11 jobs due to kelloggm/checkerframework-gradle-plugin#117
add e3f4583 Merge pull request #12396: Disable checker in Java 11 builds
add 87b131a Improve beam CreateList test coverage
add f03fb96 Merge pull request #12392: Improve beam CreateList test coverage
add 780cc4f [BEAM-8494] Add python3.8 to the list of container images to publish. (#12401)
No new revisions were added by this update.
Summary of changes:
.github/workflows/build_wheels.yml | 11 +-
.test-infra/jenkins/CommonJobProperties.groovy | 6 +-
.../job_PreCommit_BeamSQL_ZetaSQL_Java11.groovy | 2 +
..._PreCommit_Java_Examples_Dataflow_Java11.groovy | 4 +-
...job_PreCommit_Java_PortabilityApi_Java11.groovy | 4 +-
.../jenkins/job_PreCommit_SQL_Java11.groovy | 2 +
.../job_beam_PerformanceTests_Analysis.groovy | 5 +
CHANGES.md | 1 +
.../org/apache/beam/gradle/BeamModulePlugin.groovy | 1 +
.../Additional Outputs/task-remote-info.yaml | 2 -
.../Additional Outputs/lesson-remote-info.yaml | 3 -
.../go/Core Transforms/Branching/Branching/go.mod | 24 -
.../go/Core Transforms/Branching/Branching/go.sum | 343 ---------------
.../Branching/Branching/task-remote-info.yaml | 2 -
.../Branching/lesson-remote-info.yaml | 3 -
.../CoGroupByKey/CoGroupByKey/go.mod | 29 --
.../CoGroupByKey/CoGroupByKey/go.sum | 329 --------------
.../CoGroupByKey/task-remote-info.yaml | 2 -
.../CoGroupByKey/lesson-remote-info.yaml | 3 -
.../Core Transforms/Combine/Combine PerKey/go.mod | 30 --
.../Core Transforms/Combine/Combine PerKey/go.sum | 326 --------------
.../Combine/Combine PerKey/task-remote-info.yaml | 2 -
.../go/Core Transforms/Combine/CombineFn/go.mod | 29 --
.../go/Core Transforms/Combine/CombineFn/go.sum | 354 ---------------
.../Combine/CombineFn/task-remote-info.yaml | 2 -
.../Core Transforms/Combine/Simple Function/go.mod | 30 --
.../Core Transforms/Combine/Simple Function/go.sum | 326 --------------
.../Combine/Simple Function/task-remote-info.yaml | 2 -
.../Combine/lesson-remote-info.yaml | 3 -
.../go/Core Transforms/Flatten/Flatten/go.mod | 31 --
.../go/Core Transforms/Flatten/Flatten/go.sum | 145 ------
.../Core Transforms/Flatten/Flatten/task-info.yaml | 31 --
.../Flatten/Flatten/task-remote-info.yaml | 2 -
.../Flatten/lesson-remote-info.yaml | 3 -
.../Core Transforms/GroupByKey/GroupByKey/go.mod | 29 --
.../Core Transforms/GroupByKey/GroupByKey/go.sum | 330 --------------
.../GroupByKey/GroupByKey/task-remote-info.yaml | 2 -
.../GroupByKey/lesson-remote-info.yaml | 3 -
.../go/Core Transforms/Map/ParDo OneToMany/go.mod | 29 --
.../go/Core Transforms/Map/ParDo OneToMany/go.sum | 151 -------
.../Map/ParDo OneToMany/task-remote-info.yaml | 2 -
.../go/Core Transforms/Map/ParDo Struct/go.mod | 29 --
.../go/Core Transforms/Map/ParDo Struct/go.sum | 151 -------
.../Map/ParDo Struct/task-remote-info.yaml | 2 -
learning/katas/go/Core Transforms/Map/ParDo/go.mod | 29 --
learning/katas/go/Core Transforms/Map/ParDo/go.sum | 151 -------
.../Map/ParDo/task-remote-info.yaml | 2 -
.../katas/go/Core Transforms/Map/lesson-info.yaml | 23 -
.../go/Core Transforms/Map/lesson-remote-info.yaml | 3 -
.../go/Core Transforms/Partition/Partition/go.mod | 30 --
.../go/Core Transforms/Partition/Partition/go.sum | 324 --------------
.../Partition/Partition/task-remote-info.yaml | 2 -
.../Partition/lesson-remote-info.yaml | 3 -
.../Core Transforms/Side Input/Side Input/go.mod | 31 --
.../Core Transforms/Side Input/Side Input/go.sum | 326 --------------
.../Side Input/Side Input/task-remote-info.yaml | 2 -
.../go/Core Transforms/Side Input/lesson-info.yaml | 21 -
.../Side Input/lesson-remote-info.yaml | 3 -
.../go/Core Transforms/section-remote-info.yaml | 2 -
.../Introduction/Hello Beam/Hello Beam Test/go.mod | 29 --
.../Introduction/Hello Beam/Hello Beam Test/go.sum | 96 ----
.../Hello Beam Test/task-remote-info.yaml | 2 -
.../go/Introduction/Hello Beam/Hello Beam/go.mod | 30 --
.../go/Introduction/Hello Beam/Hello Beam/go.sum | 151 -------
.../Hello Beam/Hello Beam/task-remote-info.yaml | 2 -
.../go/Introduction/Hello Beam/lesson-info.yaml | 22 -
.../Hello Beam/lesson-remote-info.yaml | 3 -
.../katas/go/Introduction/section-remote-info.yaml | 2 -
.../additional_outputs}/cmd/main.go | 2 +-
.../additional_outputs}/pkg/task/task.go | 0
.../additional_outputs}/task-info.yaml | 4 -
.../additional_outputs/task-remote-info.yaml | 2 +
.../additional_outputs/additional_outputs}/task.md | 0
.../additional_outputs}/test/task_test.go | 2 +-
.../additional_outputs}/lesson-info.yaml | 2 +-
.../additional_outputs/lesson-remote-info.yaml | 3 +
.../branching/branching}/cmd/main.go | 2 +-
.../branching/branching}/pkg/task/task.go | 0
.../branching/branching}/task-info.yaml | 4 -
.../branching/branching/task-remote-info.yaml | 2 +
.../branching/branching}/task.md | 0
.../branching/branching}/test/task_test.go | 2 +-
.../branching}/lesson-info.yaml | 2 +-
.../branching/lesson-remote-info.yaml | 3 +
.../cogroupbykey/cogroupbykey}/cmd/main.go | 2 +-
.../cogroupbykey/cogroupbykey}/pkg/task/task.go | 0
.../cogroupbykey/cogroupbykey}/task-info.yaml | 4 -
.../cogroupbykey/task-remote-info.yaml | 2 +
.../cogroupbykey/cogroupbykey}/task.md | 0
.../cogroupbykey/cogroupbykey}/test/task_test.go | 2 +-
.../cogroupbykey}/lesson-info.yaml | 2 +-
.../cogroupbykey/lesson-remote-info.yaml | 3 +
.../combine/combine_perkey}/cmd/main.go | 2 +-
.../combine/combine_perkey}/pkg/task/task.go | 0
.../combine/combine_perkey}/task-info.yaml | 4 -
.../combine/combine_perkey/task-remote-info.yaml | 2 +
.../combine/combine_perkey}/task.md | 0
.../combine/combine_perkey}/test/task_test.go | 2 +-
.../combine/combinefn}/cmd/main.go | 2 +-
.../combine/combinefn}/pkg/task/task.go | 0
.../combine/combinefn}/task-info.yaml | 4 -
.../combine/combinefn/task-remote-info.yaml | 2 +
.../combine/combinefn}/task.md | 0
.../combine/combinefn}/test/task_test.go | 2 +-
.../combine}/lesson-info.yaml | 6 +-
.../combine/lesson-remote-info.yaml | 3 +
.../combine/simple_function}/cmd/main.go | 2 +-
.../combine/simple_function}/pkg/task/task.go | 0
.../combine/simple_function}/task-info.yaml | 4 -
.../combine/simple_function/task-remote-info.yaml | 2 +
.../combine/simple_function}/task.md | 0
.../combine/simple_function}/test/task_test.go | 2 +-
.../flatten/flatten}/cmd/main.go | 2 +-
.../flatten/flatten}/pkg/task/task.go | 0
.../flatten/flatten}/task-info.yaml | 10 +-
.../flatten/flatten/task-remote-info.yaml | 2 +
.../flatten/flatten}/task.md | 0
.../flatten/flatten}/test/task_test.go | 2 +-
.../flatten}/lesson-info.yaml | 2 +-
.../flatten/lesson-remote-info.yaml | 3 +
.../groupbykey/groupbykey}/cmd/main.go | 2 +-
.../groupbykey/groupbykey}/pkg/task/task.go | 0
.../groupbykey/groupbykey}/task-info.yaml | 4 -
.../groupbykey/groupbykey/task-remote-info.yaml | 2 +
.../groupbykey/groupbykey}/task.md | 0
.../groupbykey/groupbykey}/test/task_test.go | 2 +-
.../groupbykey}/lesson-info.yaml | 2 +-
.../groupbykey/lesson-remote-info.yaml | 3 +
.../map}/lesson-info.yaml | 4 +-
.../go/core_transforms/map/lesson-remote-info.yaml | 3 +
.../map/pardo}/cmd/main.go | 2 +-
.../map/pardo}/pkg/task/task.go | 0
.../map/pardo}/task-info.yaml | 10 +-
.../map/pardo/task-remote-info.yaml | 2 +
.../ParDo => core_transforms/map/pardo}/task.md | 0
.../map/pardo}/test/task_test.go | 2 +-
.../map/pardo_onetomany}/cmd/main.go | 2 +-
.../map/pardo_onetomany}/pkg/task/task.go | 0
.../map/pardo_onetomany}/task-info.yaml | 4 -
.../map/pardo_onetomany/task-remote-info.yaml | 2 +
.../map/pardo_onetomany}/task.md | 0
.../map/pardo_onetomany}/test/task_test.go | 2 +-
.../map/pardo_struct}/cmd/main.go | 2 +-
.../map/pardo_struct}/pkg/task/task.go | 0
.../map/pardo_struct}/task-info.yaml | 4 -
.../map/pardo_struct/task-remote-info.yaml | 2 +
.../map/pardo_struct}/task.md | 0
.../map/pardo_struct}/test/task_test.go | 2 +-
.../partition}/lesson-info.yaml | 2 +-
.../partition/lesson-remote-info.yaml | 3 +
.../partition/partition}/cmd/main.go | 2 +-
.../partition/partition}/pkg/task/task.go | 0
.../partition/partition}/task-info.yaml | 4 -
.../partition/partition/task-remote-info.yaml | 2 +
.../partition/partition}/task.md | 0
.../partition/partition}/test/task_test.go | 2 +-
.../section-info.yaml | 18 +-
.../go/core_transforms/section-remote-info.yaml | 2 +
.../side_input}/lesson-info.yaml | 2 +-
.../side_input/lesson-remote-info.yaml | 3 +
.../side_input/side_input}/cmd/main.go | 2 +-
.../side_input/side_input}/pkg/task/task.go | 0
.../side_input/side_input}/task-info.yaml | 4 -
.../side_input/side_input/task-remote-info.yaml | 2 +
.../side_input/side_input}/task.md | 0
.../side_input/side_input}/test/task_test.go | 2 +-
learning/katas/go/course-info.yaml | 4 +-
learning/katas/go/course-remote-info.yaml | 2 +-
.../Additional Outputs => }/go.mod | 7 +-
.../Additional Outputs => }/go.sum | 10 +-
.../hello_beam/hello_beam}/cmd/main.go | 2 +-
.../hello_beam/hello_beam}/pkg/task/task.go | 0
.../hello_beam/hello_beam}/task-info.yaml | 5 +-
.../hello_beam/hello_beam/task-remote-info.yaml | 2 +
.../hello_beam/hello_beam}/task.md | 0
.../hello_beam/hello_beam}/test/task_test.go | 2 +-
.../hello_beam/hello_beam_test}/pkg/task/task.go | 0
.../hello_beam/hello_beam_test}/task-info.yaml | 11 +-
.../hello_beam_test/task-remote-info.yaml | 2 +
.../hello_beam/hello_beam_test}/task.md | 0
.../hello_beam/hello_beam_test}/test/task_test.go | 2 +-
.../hello_beam}/lesson-info.yaml | 3 +-
.../hello_beam/lesson-remote-info.yaml | 3 +
.../section-info.yaml | 2 +-
.../katas/go/introduction/section-remote-info.yaml | 2 +
.../python-release/python_release_automation.sh | 2 +-
.../src/main/scripts/build_release_candidate.sh | 86 ++--
.../scripts/download_github_actions_artifacts.py | 326 ++++++++++++++
release/src/main/scripts/mass_comment.py | 1 +
release/src/main/scripts/publish_docker_images.sh | 20 +-
.../src/main/scripts/sign_hash_python_wheels.sh | 66 ---
.../beam/runners/flink/FlinkSavepointTest.java | 30 +-
.../runners/spark/metrics/MetricsAccumulator.java | 4 +-
.../MetricsContainerStepMapAccumulator.java | 6 +-
.../metrics/SparkMetricsContainerStepMap.java | 2 +-
.../metrics/MetricsAccumulator.java | 2 +-
.../MetricsContainerStepMapAccumulator.java | 6 +-
.../metrics/SparkMetricsContainerStepMap.java | 2 +-
sdks/go/pkg/beam/create_test.go | 36 +-
.../core/client/operator/AssignEventTime.java | 8 +-
.../euphoria/core/client/operator/CountByKey.java | 3 +-
.../euphoria/core/client/operator/Distinct.java | 3 +-
.../euphoria/core/client/operator/Filter.java | 3 +-
.../euphoria/core/client/operator/FlatMap.java | 3 +-
.../euphoria/core/client/operator/Join.java | 10 +-
.../euphoria/core/client/operator/MapElements.java | 3 +-
.../euphoria/core/client/operator/ReduceByKey.java | 5 +-
.../core/client/operator/ReduceWindow.java | 3 +-
.../euphoria/core/client/operator/SumByKey.java | 3 +-
.../euphoria/core/client/operator/TopPerKey.java | 3 +-
.../euphoria/core/client/operator/Union.java | 3 +-
.../core/client/operator/base/Builders.java | 40 +-
.../core/client/operator/hint/ComputationHint.java | 7 +-
.../core/client/operator/hint/OutputHint.java | 7 +-
.../core/client/operator/hint/SizeHint.java | 7 +-
sdks/java/extensions/join-library/build.gradle | 2 +-
.../sdk/extensions/joinlibrary/InnerJoinTest.java | 8 +-
.../extensions/joinlibrary/OuterFullJoinTest.java | 3 +
.../extensions/joinlibrary/OuterLeftJoinTest.java | 3 +
.../extensions/joinlibrary/OuterRightJoinTest.java | 3 +
sdks/java/extensions/kryo/build.gradle | 3 +-
.../apache/beam/sdk/extensions/kryo/KryoCoder.java | 7 +-
.../apache/beam/sdk/extensions/kryo/KryoState.java | 19 +-
sdks/java/extensions/sorter/build.gradle | 1 -
.../extensions/sorter/BufferedExternalSorter.java | 12 +-
.../extensions/sorter/HadoopExternalSorter.java | 137 +++---
.../beam/sdk/extensions/sorter/InMemorySorter.java | 33 +-
.../extensions/sorter/NativeExternalSorter.java | 21 +-
.../sdk/extensions/sorter/NativeFileSorter.java | 40 +-
.../sorter/src/main}/resources/log4j.properties | 0
.../sorter/BufferedExternalSorterTest.java | 32 +-
.../sdk/extensions/sorter/ExternalSorterTest.java | 28 +-
.../beam/sdk/extensions/sorter/SortValuesTest.java | 5 +-
.../extensions/sql/expansion-service/build.gradle | 2 +-
.../expansion/ExternalSqlTransformRegistrar.java | 4 +-
sdks/java/io/amazon-web-services2/build.gradle | 18 +-
.../sdk/io/aws2/kinesis/AWSClientsProvider.java} | 19 +-
.../sdk/io/aws2/kinesis/BasicKinesisProvider.java | 78 ++++
.../sdk/io/aws2/kinesis/CheckpointGenerator.java} | 14 +-
.../beam/sdk/io/aws2/kinesis/CustomOptional.java | 102 +++++
.../aws2/kinesis/DynamicCheckpointGenerator.java | 74 ++++
.../io/aws2/kinesis/GetKinesisRecordsResult.java | 60 +++
.../kinesis/KinesisClientThrottledException.java} | 14 +-
.../apache/beam/sdk/io/aws2/kinesis/KinesisIO.java | 484 +++++++++++++++++++++
.../beam/sdk/io/aws2/kinesis/KinesisReader.java | 219 ++++++++++
.../io/aws2/kinesis/KinesisReaderCheckpoint.java | 76 ++++
.../beam/sdk/io/aws2/kinesis/KinesisRecord.java | 149 +++++++
.../sdk/io/aws2/kinesis/KinesisRecordCoder.java | 76 ++++
.../aws2/kinesis/KinesisShardClosedException.java} | 13 +-
.../beam/sdk/io/aws2/kinesis/KinesisSource.java | 161 +++++++
.../beam/sdk/io/aws2/kinesis/RateLimitPolicy.java} | 23 +-
.../io/aws2/kinesis/RateLimitPolicyFactory.java | 70 +++
.../beam/sdk/io/aws2/kinesis/RecordFilter.java | 36 +-
.../beam/sdk/io/aws2/kinesis/ShardCheckpoint.java | 177 ++++++++
.../beam/sdk/io/aws2/kinesis/ShardReadersPool.java | 341 +++++++++++++++
.../sdk/io/aws2/kinesis/ShardRecordsIterator.java | 149 +++++++
.../io/aws2/kinesis/SimplifiedKinesisClient.java | 275 ++++++++++++
.../beam/sdk/io/aws2/kinesis/StartingPoint.java | 86 ++++
.../io/aws2/kinesis/StartingPointShardsFinder.java | 198 +++++++++
.../io/aws2/kinesis/StaticCheckpointGenerator.java | 28 +-
.../apache/beam/sdk/io/aws2/kinesis/TimeUtil.java} | 23 +-
.../aws2/kinesis/TransientKinesisException.java} | 15 +-
.../sdk/io/aws2/kinesis/WatermarkParameters.java | 98 +++++
.../beam/sdk/io/aws2/kinesis/WatermarkPolicy.java} | 14 +-
.../io/aws2/kinesis/WatermarkPolicyFactory.java | 152 +++++++
.../beam/sdk/io/aws2/kinesis/package-info.java} | 11 +-
.../sdk/io/aws2/kinesis/AmazonKinesisMock.java | 404 +++++++++++++++++
.../sdk/io/aws2/kinesis/CustomOptionalTest.java | 39 +-
.../kinesis/DynamicCheckpointGeneratorTest.java | 80 ++++
.../beam/sdk/io/aws2/kinesis/KinesisIOIT.java | 155 +++++++
.../sdk/io/aws2/kinesis/KinesisMockReadTest.java | 119 +++++
.../aws2/kinesis/KinesisReaderCheckpointTest.java | 64 +++
.../sdk/io/aws2/kinesis/KinesisReaderTest.java | 184 ++++++++
.../io/aws2/kinesis/KinesisRecordCoderTest.java | 46 ++
.../sdk/io/aws2/kinesis/KinesisServiceMock.java | 84 ++++
.../sdk/io/aws2/kinesis/KinesisTestOptions.java | 62 +++
.../beam/sdk/io/aws2/kinesis/RecordFilterTest.java | 62 +++
.../sdk/io/aws2/kinesis/ShardCheckpointTest.java | 162 +++++++
.../sdk/io/aws2/kinesis/ShardReadersPoolTest.java | 351 +++++++++++++++
.../io/aws2/kinesis/ShardRecordsIteratorTest.java | 186 ++++++++
.../aws2/kinesis/SimplifiedKinesisClientTest.java | 378 ++++++++++++++++
.../kinesis/StartingPointShardsFinderTest.java | 292 +++++++++++++
.../beam/sdk/io/aws2/kinesis/TimeUtilTest.java | 35 +-
.../sdk/io/aws2/kinesis/WatermarkPolicyTest.java | 152 +++++++
.../org/apache/beam/sdk/io/aws2/sns/SnsIOTest.java | 7 +-
.../org.mockito.plugins.MockMaker | 1 -
.../apache_beam/examples/kafkataxi/README.md | 36 +-
.../apache_beam/examples/wordcount_xlang_sql.py | 84 ++--
.../apache-beam-jupyterlab-sidepanel/package.json | 8 +-
.../style/index.css | 2 +-
.../testing/load_tests/group_by_key_test.py | 7 +-
.../testing/load_tests/load_test_metrics_utils.py | 35 ++
.../apache_beam/testing/load_tests/pardo_test.py | 9 +-
.../apache_beam/transforms/ptransform_test.py | 2 +-
sdks/python/container/base_image_requirements.txt | 2 +-
sdks/python/setup.py | 2 +-
.../site/content/en/contribute/release-guide.md | 10 +-
.../transforms/python/aggregation/groupbykey.md | 39 +-
.../python/aggregation/groupintobatches.md | 32 +-
.../en/documentation/transforms/python/overview.md | 2 +-
.../partials/section-menu/en/documentation.html | 1 +
301 files changed, 7086 insertions(+), 4745 deletions(-)
delete mode 100644 learning/katas/go/Core Transforms/Additional Outputs/Additional Outputs/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Additional Outputs/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Branching/Branching/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Branching/Branching/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Branching/Branching/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Branching/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/CoGroupByKey/CoGroupByKey/go.mod
delete mode 100644 learning/katas/go/Core Transforms/CoGroupByKey/CoGroupByKey/go.sum
delete mode 100644 learning/katas/go/Core Transforms/CoGroupByKey/CoGroupByKey/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/CoGroupByKey/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Combine/Combine PerKey/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Combine/Combine PerKey/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Combine/Combine PerKey/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Combine/CombineFn/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Combine/CombineFn/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Combine/CombineFn/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Combine/Simple Function/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Combine/Simple Function/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Combine/Simple Function/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Combine/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Flatten/Flatten/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Flatten/Flatten/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Flatten/Flatten/task-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Flatten/Flatten/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Flatten/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/GroupByKey/GroupByKey/go.mod
delete mode 100644 learning/katas/go/Core Transforms/GroupByKey/GroupByKey/go.sum
delete mode 100644 learning/katas/go/Core Transforms/GroupByKey/GroupByKey/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/GroupByKey/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo OneToMany/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo OneToMany/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo OneToMany/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo Struct/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo Struct/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo Struct/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Map/ParDo/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Map/lesson-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Map/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Partition/Partition/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Partition/Partition/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Partition/Partition/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Partition/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Side Input/Side Input/go.mod
delete mode 100644 learning/katas/go/Core Transforms/Side Input/Side Input/go.sum
delete mode 100644 learning/katas/go/Core Transforms/Side Input/Side Input/task-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Side Input/lesson-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/Side Input/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Core Transforms/section-remote-info.yaml
delete mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam Test/go.mod
delete mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam Test/go.sum
delete mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam Test/task-remote-info.yaml
delete mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam/go.mod
delete mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam/go.sum
delete mode 100644 learning/katas/go/Introduction/Hello Beam/Hello Beam/task-remote-info.yaml
delete mode 100644 learning/katas/go/Introduction/Hello Beam/lesson-info.yaml
delete mode 100644 learning/katas/go/Introduction/Hello Beam/lesson-remote-info.yaml
delete mode 100644 learning/katas/go/Introduction/section-remote-info.yaml
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => core_transforms/additional_outputs/additional_outputs}/cmd/main.go (93%)
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => core_transforms/additional_outputs/additional_outputs}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => core_transforms/additional_outputs/additional_outputs}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/additional_outputs/additional_outputs/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => core_transforms/additional_outputs/additional_outputs}/task.md (100%)
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => core_transforms/additional_outputs/additional_outputs}/test/task_test.go (94%)
rename learning/katas/go/{Core Transforms/Additional Outputs => core_transforms/additional_outputs}/lesson-info.yaml (97%)
create mode 100644 learning/katas/go/core_transforms/additional_outputs/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/Branching/Branching => core_transforms/branching/branching}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Branching/Branching => core_transforms/branching/branching}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Branching/Branching => core_transforms/branching/branching}/task-info.yaml (95%)
create mode 100644 learning/katas/go/core_transforms/branching/branching/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Branching/Branching => core_transforms/branching/branching}/task.md (100%)
rename learning/katas/go/{Core Transforms/Branching/Branching => core_transforms/branching/branching}/test/task_test.go (95%)
copy learning/katas/go/{Core Transforms/Partition => core_transforms/branching}/lesson-info.yaml (98%)
create mode 100644 learning/katas/go/core_transforms/branching/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/CoGroupByKey/CoGroupByKey => core_transforms/cogroupbykey/cogroupbykey}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/CoGroupByKey/CoGroupByKey => core_transforms/cogroupbykey/cogroupbykey}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/CoGroupByKey/CoGroupByKey => core_transforms/cogroupbykey/cogroupbykey}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/cogroupbykey/cogroupbykey/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/CoGroupByKey/CoGroupByKey => core_transforms/cogroupbykey/cogroupbykey}/task.md (100%)
rename learning/katas/go/{Core Transforms/CoGroupByKey/CoGroupByKey => core_transforms/cogroupbykey/cogroupbykey}/test/task_test.go (95%)
copy learning/katas/go/{Core Transforms/CoGroupByKey => core_transforms/cogroupbykey}/lesson-info.yaml (98%)
create mode 100644 learning/katas/go/core_transforms/cogroupbykey/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/Combine/Combine PerKey => core_transforms/combine/combine_perkey}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Combine/Combine PerKey => core_transforms/combine/combine_perkey}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Combine/Combine PerKey => core_transforms/combine/combine_perkey}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/combine/combine_perkey/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Combine/Combine PerKey => core_transforms/combine/combine_perkey}/task.md (100%)
rename learning/katas/go/{Core Transforms/Combine/Combine PerKey => core_transforms/combine/combine_perkey}/test/task_test.go (95%)
rename learning/katas/go/{Core Transforms/Combine/CombineFn => core_transforms/combine/combinefn}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Combine/CombineFn => core_transforms/combine/combinefn}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Combine/CombineFn => core_transforms/combine/combinefn}/task-info.yaml (95%)
create mode 100644 learning/katas/go/core_transforms/combine/combinefn/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Combine/CombineFn => core_transforms/combine/combinefn}/task.md (100%)
rename learning/katas/go/{Core Transforms/Combine/CombineFn => core_transforms/combine/combinefn}/test/task_test.go (94%)
rename learning/katas/go/{Core Transforms/Combine => core_transforms/combine}/lesson-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/combine/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/Combine/Simple Function => core_transforms/combine/simple_function}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Combine/Simple Function => core_transforms/combine/simple_function}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Combine/Simple Function => core_transforms/combine/simple_function}/task-info.yaml (93%)
create mode 100644 learning/katas/go/core_transforms/combine/simple_function/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Combine/Simple Function => core_transforms/combine/simple_function}/task.md (100%)
rename learning/katas/go/{Core Transforms/Combine/Simple Function => core_transforms/combine/simple_function}/test/task_test.go (94%)
rename learning/katas/go/{Core Transforms/Flatten/Flatten => core_transforms/flatten/flatten}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Flatten/Flatten => core_transforms/flatten/flatten}/pkg/task/task.go (100%)
copy learning/katas/go/{Introduction/Hello Beam/Hello Beam => core_transforms/flatten/flatten}/task-info.yaml (91%)
create mode 100644 learning/katas/go/core_transforms/flatten/flatten/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Flatten/Flatten => core_transforms/flatten/flatten}/task.md (100%)
rename learning/katas/go/{Core Transforms/Flatten/Flatten => core_transforms/flatten/flatten}/test/task_test.go (95%)
rename learning/katas/go/{Core Transforms/Flatten => core_transforms/flatten}/lesson-info.yaml (98%)
create mode 100644 learning/katas/go/core_transforms/flatten/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/GroupByKey/GroupByKey => core_transforms/groupbykey/groupbykey}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/GroupByKey/GroupByKey => core_transforms/groupbykey/groupbykey}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/GroupByKey/GroupByKey => core_transforms/groupbykey/groupbykey}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/groupbykey/groupbykey/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/GroupByKey/GroupByKey => core_transforms/groupbykey/groupbykey}/task.md (100%)
rename learning/katas/go/{Core Transforms/GroupByKey/GroupByKey => core_transforms/groupbykey/groupbykey}/test/task_test.go (95%)
rename learning/katas/go/{Core Transforms/GroupByKey => core_transforms/groupbykey}/lesson-info.yaml (98%)
create mode 100644 learning/katas/go/core_transforms/groupbykey/lesson-remote-info.yaml
copy learning/katas/go/{Core Transforms/CoGroupByKey => core_transforms/map}/lesson-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/map/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/Map/ParDo => core_transforms/map/pardo}/cmd/main.go (95%)
rename learning/katas/go/{Core Transforms/Map/ParDo => core_transforms/map/pardo}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Map/ParDo => core_transforms/map/pardo}/task-info.yaml (91%)
create mode 100644 learning/katas/go/core_transforms/map/pardo/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Map/ParDo => core_transforms/map/pardo}/task.md (100%)
rename learning/katas/go/{Core Transforms/Map/ParDo => core_transforms/map/pardo}/test/task_test.go (95%)
rename learning/katas/go/{Core Transforms/Map/ParDo OneToMany => core_transforms/map/pardo_onetomany}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Map/ParDo OneToMany => core_transforms/map/pardo_onetomany}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Map/ParDo OneToMany => core_transforms/map/pardo_onetomany}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/map/pardo_onetomany/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Map/ParDo OneToMany => core_transforms/map/pardo_onetomany}/task.md (100%)
rename learning/katas/go/{Core Transforms/Map/ParDo OneToMany => core_transforms/map/pardo_onetomany}/test/task_test.go (94%)
rename learning/katas/go/{Core Transforms/Map/ParDo Struct => core_transforms/map/pardo_struct}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Map/ParDo Struct => core_transforms/map/pardo_struct}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Map/ParDo Struct => core_transforms/map/pardo_struct}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/map/pardo_struct/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Map/ParDo Struct => core_transforms/map/pardo_struct}/task.md (100%)
rename learning/katas/go/{Core Transforms/Map/ParDo Struct => core_transforms/map/pardo_struct}/test/task_test.go (94%)
rename learning/katas/go/{Core Transforms/Branching => core_transforms/partition}/lesson-info.yaml (98%)
create mode 100644 learning/katas/go/core_transforms/partition/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/Partition/Partition => core_transforms/partition/partition}/cmd/main.go (94%)
rename learning/katas/go/{Core Transforms/Partition/Partition => core_transforms/partition/partition}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Partition/Partition => core_transforms/partition/partition}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/partition/partition/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Partition/Partition => core_transforms/partition/partition}/task.md (100%)
rename learning/katas/go/{Core Transforms/Partition/Partition => core_transforms/partition/partition}/test/task_test.go (94%)
rename learning/katas/go/{Core Transforms => core_transforms}/section-info.yaml (86%)
create mode 100644 learning/katas/go/core_transforms/section-remote-info.yaml
rename learning/katas/go/{Core Transforms/Partition => core_transforms/side_input}/lesson-info.yaml (98%)
create mode 100644 learning/katas/go/core_transforms/side_input/lesson-remote-info.yaml
rename learning/katas/go/{Core Transforms/Side Input/Side Input => core_transforms/side_input/side_input}/cmd/main.go (95%)
rename learning/katas/go/{Core Transforms/Side Input/Side Input => core_transforms/side_input/side_input}/pkg/task/task.go (100%)
rename learning/katas/go/{Core Transforms/Side Input/Side Input => core_transforms/side_input/side_input}/task-info.yaml (94%)
create mode 100644 learning/katas/go/core_transforms/side_input/side_input/task-remote-info.yaml
rename learning/katas/go/{Core Transforms/Side Input/Side Input => core_transforms/side_input/side_input}/task.md (100%)
rename learning/katas/go/{Core Transforms/Side Input/Side Input => core_transforms/side_input/side_input}/test/task_test.go (97%)
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => }/go.mod (84%)
rename learning/katas/go/{Core Transforms/Additional Outputs/Additional Outputs => }/go.sum (98%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam => introduction/hello_beam/hello_beam}/cmd/main.go (94%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam => introduction/hello_beam/hello_beam}/pkg/task/task.go (100%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam => introduction/hello_beam/hello_beam}/task-info.yaml (92%)
create mode 100644 learning/katas/go/introduction/hello_beam/hello_beam/task-remote-info.yaml
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam => introduction/hello_beam/hello_beam}/task.md (100%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam => introduction/hello_beam/hello_beam}/test/task_test.go (94%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam Test => introduction/hello_beam/hello_beam_test}/pkg/task/task.go (100%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam Test => introduction/hello_beam/hello_beam_test}/task-info.yaml (88%)
create mode 100644 learning/katas/go/introduction/hello_beam/hello_beam_test/task-remote-info.yaml
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam Test => introduction/hello_beam/hello_beam_test}/task.md (100%)
rename learning/katas/go/{Introduction/Hello Beam/Hello Beam Test => introduction/hello_beam/hello_beam_test}/test/task_test.go (93%)
rename learning/katas/go/{Core Transforms/CoGroupByKey => introduction/hello_beam}/lesson-info.yaml (96%)
create mode 100644 learning/katas/go/introduction/hello_beam/lesson-remote-info.yaml
rename learning/katas/go/{Introduction => introduction}/section-info.yaml (98%)
create mode 100644 learning/katas/go/introduction/section-remote-info.yaml
create mode 100644 release/src/main/scripts/download_github_actions_artifacts.py
delete mode 100755 release/src/main/scripts/sign_hash_python_wheels.sh
copy sdks/java/{testing/kafka-service/src/test => extensions/sorter/src/main}/resources/log4j.properties (100%)
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/AWSClientsProvider.java} (62%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/BasicKinesisProvider.java
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/CheckpointGenerator.java} (71%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/CustomOptional.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/DynamicCheckpointGenerator.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/GetKinesisRecordsResult.java
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/ComputationHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisClientThrottledException.java} (69%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisIO.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisReader.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisReaderCheckpoint.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisRecord.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisRecordCoder.java
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisShardClosedException.java} (71%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisSource.java
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/SizeHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/RateLimitPolicy.java} (60%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/RateLimitPolicyFactory.java
copy runners/spark/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkMetricsContainerStepMap.java => sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/RecordFilter.java (53%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardCheckpoint.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardReadersPool.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/ShardRecordsIterator.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/SimplifiedKinesisClient.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/StartingPoint.java
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/StartingPointShardsFinder.java
copy runners/spark/src/main/java/org/apache/beam/runners/spark/metrics/SparkMetricsContainerStepMap.java => sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/StaticCheckpointGenerator.java (58%)
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/ComputationHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/TimeUtil.java} (59%)
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/TransientKinesisException.java} (71%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/WatermarkParameters.java
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/WatermarkPolicy.java} (73%)
create mode 100644 sdks/java/io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/WatermarkPolicyFactory.java
copy sdks/java/{extensions/euphoria/src/main/java/org/apache/beam/sdk/extensions/euphoria/core/client/operator/hint/OutputHint.java => io/amazon-web-services2/src/main/java/org/apache/beam/sdk/io/aws2/kinesis/package-info.java} (71%)
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/AmazonKinesisMock.java
copy runners/spark/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkMetricsContainerStepMap.java => sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/CustomOptionalTest.java (50%)
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/DynamicCheckpointGeneratorTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisIOIT.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisMockReadTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisReaderCheckpointTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisReaderTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisRecordCoderTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisServiceMock.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/KinesisTestOptions.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/RecordFilterTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/ShardCheckpointTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/ShardReadersPoolTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/ShardRecordsIteratorTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/SimplifiedKinesisClientTest.java
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/StartingPointShardsFinderTest.java
copy runners/spark/src/main/java/org/apache/beam/runners/spark/structuredstreaming/metrics/SparkMetricsContainerStepMap.java => sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/TimeUtilTest.java (53%)
create mode 100644 sdks/java/io/amazon-web-services2/src/test/java/org/apache/beam/sdk/io/aws2/kinesis/WatermarkPolicyTest.java
delete mode 100644 sdks/java/io/amazon-web-services2/src/test/resources/mockito-extensions/org.mockito.plugins.MockMaker