You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by vi...@apache.org on 2020/10/14 20:40:34 UTC
[hudi] branch rfc-15 updated (e9b1e4e -> 8bdbdf4)
This is an automated email from the ASF dual-hosted git repository.
vinoth pushed a change to branch rfc-15
in repository https://gitbox.apache.org/repos/asf/hudi.git.
discard e9b1e4e [HUDI-842] Fix non-partitioned case and speedup initial creation of metadata table.
discard fe8166c [HUDI-842] Schema evolution strategy for Metadata Table.
discard 550b10f [HUDI-842] Fixed creation of config for read only mode.
omit 93c34af [HUDI-842] Addressed review comments.
omit 026ac1f [HUDI-842] Fixed checkstyle.
omit 65dc72a [HUDI-842] Handled the error case where metadata update succeeds but dataset commit fails.
omit 0bebd43 WIP
omit 8561a3e [HUDI-842] Added unit test for rollback of partial commits (inflight but not completed yet).
omit 864bf73 [HUDI-842] Support for marker based rollbacks.
omit 8ddbb32 [HUDI-842] Support for rollbacks in MOR Table.
omit 9fcaa99 [HUDI-842] Extend rollback metadata to include the files which have been appended to.
omit 3dc02f8 [HUDI-842] Removed HoodieMetadataIndex as we can prep records with the name of the partition and the latest base file id.
omit af6c39b [HUDI-842] Addressed review comments.
omit 8369216 [HUDI-842] Implementation of CLI commands for metadata operations and lookups.
omit 32b8a59 [HUDI-842] Reduce parallelism to speed up the test.
omit 5a65f52 Fix checkstyle and rat issues
omit 76ac0c3 [HUDI-842] Implementation of HUDI RFC-15.
omit 55e8951 [HUDI-842] Implementation of HUDI RFC-15.
add 6537af2 [HUDI-1153] Spark DataSource and Streaming Write must fail when operation type is misconfigured (#2014)
add 83e39e2 [HUDI-781] Add HoodieWriteableTestTable (#2040)
add fe7c9e7 [MINOR] Fix BindException when running tests of shared machines. (#2070)
add 51b16bd [MINOR] fix typo
add e3cf34d Merge pull request #2077 from chuangehh/typofix
add 2fee087 [HUDI-1181] Fix decimal type display issue for record key field (#1953)
add 53d1e55 Test Suite should work with Docker + Unit Tests
add fec7cd3 [HUDI-1130] hudi-test-suite support for schema evolution (can be triggered on any insert/upsert DAG node).
add 063a98f [HUDI-1254] TypedProperties can not get values by initializing an existing properties (#2059)
add a1cff8a [HUDI-1255] Add new Payload(OverwriteNonDefaultsWithLatestAvroPayload) for updating specified fields in storage (#2056)
add 5e61454 [HUDI-802] AWSDmsTransformer does not handle insert and delete of a row in a single batch correctly (#2084)
add 6c84ef2 [HUDI-1282] Check whether the topic exists before deltastrmer consumes Kafka (#2090)
add 581d540 [HUDI-1143] Change timestamp field in HoodieTestDataGenerator from double to long
add 3201665 [HUDI-995] Use HoodieTestTable in more classes (#2079)
add bf65269 [HUDI-1230] Fix for preventing MOR datasource jobs from hanging via spark-submit (#2046)
add 73e5b4c [HUDI-796] Add deduping logic for upserts case (#1558)
add 7c45894 [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable (#2094)
add c8e19e2 [HUDI-801] Adding a way to post process schema after it is fetched (#1524)
add 8087016 [HUDI-1213] Set Default for the bootstrap config : hoodie.bootstrap.full.input.provider (#2087)
add fcc497e [HUDI-1268] fix UpgradeDowngrade fs Rename issue for hdfs and aliyun oss (#2099)
add d37977b [MINOR] Remove useless config for bootstrap integ testing (#2102)
add 83d2e03 [MINOR] Adding scripts to checkout and push to PRs (#2109)
add 6837118 [MINOR] Improve description (#2113)
add 2eaba09 [HUDI-544] Archived commits command code cleanup (#1242)
add 1dd6635 [MINOR] Fix ClassCastException when use QuickstartUtils generate data (#2105)
add ae68b2b [MINOR] fix typos (#2116)
add 1be0b06 [HUDI-995] Migrate HoodieTestUtils APIs to HoodieTestTable (#2112)
add b0f1b73 [MINOR] Fix checkstyle (#2117)
add a86f557 [HUDI-1192] Make create hive database automatically configurable (#1968)
add c0c0095 [MINOR] Reformat prepare_integration_suite script (#2126)
add 20b9b39 [HUDI-1233] Deltastreamer Kafka consumption delay reporting indicators (#2074)
add 32c9cad [HUDI-840] Avoid blank file created by HoodieLogFormatWriter (#1567)
add a99e93b [HUDI-1072] Introduce REPLACE top level action. Implement insert_overwrite operation on top of replace action (#2048)
add 5aaaf8b [MINOR] Change the log level of the dag scheduler for the test suite (#2134)
add 0193a57 [HUDI-842] Implementation of HUDI RFC-15.
add 8d7e1b5 [HUDI-842] Implementation of HUDI RFC-15.
add 0642a80 Fix checkstyle and rat issues
add 65cfc08 [HUDI-842] Reduce parallelism to speed up the test.
add 2f34616 [HUDI-842] Implementation of CLI commands for metadata operations and lookups.
add a3a74a2 [HUDI-842] Addressed review comments.
add bae8a21 [HUDI-842] Extend rollback metadata to include the files which have been appended to.
add 39083c2 [HUDI-842] Support for rollbacks in MOR Table.
add daa3ae2 [HUDI-842] Support for marker based rollbacks.
add cc3c284 [HUDI-842] Added unit test for rollback of partial commits (inflight but not completed yet).
add 69d2939 WIP
add 663a392 [HUDI-842] Handled the error case where metadata update succeeds but dataset commit fails.
add 150b727 [HUDI-842] Fixed checkstyle.
add d33659a [HUDI-842] Addressed review comments.
add 9bb5a2f [HUDI-842] Fixed creation of config for read only mode.
add 2e6e342 [HUDI-842] Schema evolution strategy for Metadata Table.
add 856af2d [HUDI-842] Fix non-partitioned case and speedup initial creation of metadata table.
add e6cac9d [HUDI-842] Rebasing on top of master before the multi-engine changes.
add 623c78a [HUDI-842] Removed HoodieMetadataIndex as we can prep records with the name of the partition and the latest base file id.
add 5c24a9c [HUDI-842] Fixing unit test for non-paritioned case.
add b059869 [HUDI-842] Reworked metrics pusblishing.
add 446bfc5 [HUDI-842] Fixed metrics for metadata size, removed unnecessary code and fixed logs+spellings.
add 8bdbdf4 [HUDI-842] Reader and Writer for Metadata.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (e9b1e4e)
\
N -- N -- N refs/heads/rfc-15 (8bdbdf4)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
docker/demo/config/test-suite/complex-dag-cow.yaml | 16 +-
docker/demo/config/test-suite/target.avsc | 2 +-
.../demo/config/test-suite/test-source.properties | 24 +-
.../hudi/cli/commands/ArchivedCommitsCommand.java | 9 +-
.../apache/hudi/cli/commands/BootstrapCommand.java | 2 +-
.../apache/hudi/cli/commands/MetadataCommand.java | 49 +-
.../apache/hudi/cli/commands/RepairsCommand.java | 10 +-
.../org/apache/hudi/cli/commands/SparkMain.java | 11 +-
.../scala/org/apache/hudi/cli/DeDupeType.scala | 12 +-
.../scala/org/apache/hudi/cli/DedupeSparkJob.scala | 112 +++-
.../cli/commands/TestArchivedCommitsCommand.java | 8 +-
.../hudi/cli/commands/TestCleansCommand.java | 2 +-
.../hudi/cli/commands/TestCommitsCommand.java | 38 +-
.../hudi/cli/commands/TestRollbacksCommand.java | 54 +-
.../apache/hudi/cli/commands/TestStatsCommand.java | 35 +-
.../cli/commands/TestUpgradeDowngradeCommand.java | 60 +-
.../hudi/cli/integ/ITTestBootstrapCommand.java | 2 +-
.../hudi/cli/integ/ITTestCommitsCommand.java | 56 +-
.../cli/integ/ITTestHDFSParquetImportCommand.java | 4 +-
.../hudi/cli/integ/ITTestRepairsCommand.java | 115 ++--
.../HoodieTestCommitMetadataGenerator.java | 67 +--
.../hudi/client/AbstractHoodieWriteClient.java | 51 +-
.../org/apache/hudi/client/HoodieWriteClient.java | 50 +-
.../org/apache/hudi/client/HoodieWriteResult.java | 68 +++
.../apache/hudi/config/HoodieBootstrapConfig.java | 3 +
.../org/apache/hudi/metadata/HoodieMetadata.java | 325 -----------
.../metadata/HoodieMetadataFileSystemView.java | 19 +-
...MetadataImpl.java => HoodieMetadataWriter.java} | 623 +++++---------------
.../org/apache/hudi/metrics/HoodieMetrics.java | 31 -
.../main/java/org/apache/hudi/metrics/Metrics.java | 3 +-
.../apache/hudi/table/HoodieCopyOnWriteTable.java | 7 +
.../java/org/apache/hudi/table/HoodieTable.java | 20 +-
.../hudi/table/HoodieTimelineArchiveLog.java | 51 +-
.../apache/hudi/table/ReplaceArchivalHelper.java | 99 ++++
.../hudi/table/action/HoodieWriteMetadata.java | 14 +-
.../table/action/clean/CleanActionExecutor.java | 7 +-
.../hudi/table/action/clean/CleanPlanner.java | 3 +-
.../action/commit/BaseCommitActionExecutor.java | 55 +-
...va => InsertOverwriteCommitActionExecutor.java} | 44 +-
...ecutor.java => InsertOverwritePartitioner.java} | 35 +-
.../action/restore/BaseRestoreActionExecutor.java | 8 +-
.../restore/CopyOnWriteRestoreActionExecutor.java | 3 +-
.../restore/MergeOnReadRestoreActionExecutor.java | 1 +
.../rollback/BaseRollbackActionExecutor.java | 6 +-
.../hudi/table/upgrade/UpgradeDowngrade.java | 8 +
.../org/apache/hudi/client/TestClientRollback.java | 216 +++----
.../TestHoodieClientOnCopyOnWriteStorage.java | 212 ++++---
.../org/apache/hudi/index/TestHoodieIndex.java | 86 +--
.../hudi/index/bloom/TestHoodieBloomIndex.java | 115 ++--
.../index/bloom/TestHoodieGlobalBloomIndex.java | 117 ++--
.../hudi/io/TestHoodieKeyLocationFetchHandle.java | 64 +--
.../hudi/io/TestHoodieTimelineArchiveLog.java | 204 ++++---
.../apache/hudi/metadata/TestHoodieMetadata.java | 143 ++---
.../java/org/apache/hudi/table/TestCleaner.java | 148 ++---
.../apache/hudi/table/TestConsistencyGuard.java | 26 +-
.../commit/TestCopyOnWriteActionExecutor.java | 18 +-
.../table/action/commit/TestUpsertPartitioner.java | 2 +-
.../TestCopyOnWriteRollbackActionExecutor.java | 97 ++--
.../rollback/TestMarkerBasedRollbackStrategy.java | 10 +-
.../hudi/table/upgrade/TestUpgradeDowngrade.java | 15 +-
.../hudi/testutils/HoodieClientTestHarness.java | 29 +-
.../hudi/testutils/HoodieClientTestUtils.java | 64 ---
.../hudi/testutils/HoodieWriteableTestTable.java | 131 +++++
.../src/test/resources/testDataGeneratorSchema.txt | 3 +-
hudi-common/pom.xml | 2 +
.../src/main/avro/HoodieArchivedMetaEntry.avsc | 8 +
.../{HoodiePath.avsc => HoodieInstantInfo.avsc} | 12 +-
...adata.avsc => HoodieReplaceCommitMetadata.avsc} | 43 +-
.../src/main/avro/HoodieRestoreMetadata.avsc | 10 +
.../src/main/avro/HoodieRollbackMetadata.avsc | 10 +
.../java/org/apache/hudi/avro/HoodieAvroUtils.java | 54 +-
.../bootstrap/index/HFileBootstrapIndex.java | 8 +-
.../apache/hudi/common/config/TypedProperties.java | 23 +-
.../java/org/apache/hudi/common/fs/FSUtils.java | 23 +-
.../org/apache/hudi/common/model/ActionType.java | 3 +-
.../hudi/common/model/BootstrapFileMapping.java | 16 +-
.../hudi/common/model/HoodieCommitMetadata.java | 10 +-
.../common/model/HoodieReplaceCommitMetadata.java | 142 +++++
.../OverwriteNonDefaultsWithLatestAvroPayload.java | 72 +++
.../model/OverwriteWithLatestAvroPayload.java | 9 +-
.../hudi/common/model/WriteOperationType.java | 12 +
.../hudi/common/table/HoodieTableMetaClient.java | 10 +-
.../common/table/log/HoodieLogFormatWriter.java | 121 ++--
.../table/timeline/HoodieActiveTimeline.java | 19 +-
.../table/timeline/HoodieDefaultTimeline.java | 15 +-
.../hudi/common/table/timeline/HoodieInstant.java | 4 +
.../hudi/common/table/timeline/HoodieTimeline.java | 25 +-
.../table/timeline/TimelineMetadataUtils.java | 17 +-
.../table/view/AbstractTableFileSystemView.java | 191 +++++--
.../table/view/FileSystemViewStorageConfig.java | 12 +
.../table/view/HoodieTableFileSystemView.java | 39 +-
.../IncrementalTimelineSyncFileSystemView.java | 49 +-
.../table/view/PriorityBasedFileSystemView.java | 5 +
.../view/RemoteHoodieTableFileSystemView.java | 26 +-
.../table/view/RocksDbBasedFileSystemView.java | 62 +-
.../view/SpillableMapBasedFileSystemView.java | 47 +-
.../common/table/view/TableFileSystemView.java | 5 +
.../org/apache/hudi/common/util/CommitUtils.java | 96 ++++
.../hudi/common/util/RocksDBSchemaHelper.java | 12 +-
.../common/util/queue/BoundedInMemoryQueue.java | 62 +-
.../util/queue/BoundedInMemoryQueueProducer.java | 2 +-
.../apache/hudi/metadata/HoodieMetadataCommon.java | 46 --
.../HoodieMetadataMergedLogRecordScanner.java | 0
.../hudi/metadata/HoodieMetadataPayload.java | 11 +-
.../apache/hudi/metadata/HoodieMetadataReader.java | 627 +++++++++++++++++++++
.../hudi/common/bootstrap/TestBootstrapIndex.java | 2 +-
.../common/functional/TestHoodieLogFormat.java | 9 +-
.../TestHoodieLogFormatAppendFailure.java | 6 +
...OverwriteNonDefaultsWithLatestAvroPayload.java} | 54 +-
.../common/properties/TestTypedProperties.java | 84 +++
.../hudi/common/table/TestTimelineUtils.java | 22 +-
.../table/timeline/TestHoodieActiveTimeline.java | 29 +-
.../table/view/TestHoodieTableFileSystemView.java | 161 +++++-
.../table/view/TestIncrementalFSViewSync.java | 210 +++++--
.../hudi/common/testutils/CompactionTestUtils.java | 52 +-
.../hudi/common/testutils/FileCreateUtils.java | 117 +++-
.../common/testutils/HoodieTestDataGenerator.java | 98 ++--
.../hudi/common/testutils/HoodieTestTable.java | 172 +++++-
.../hudi/common/testutils/HoodieTestUtils.java | 183 +-----
.../testutils/minicluster/HdfsTestService.java | 41 +-
.../apache/hudi/common/util/TestCommitUtils.java | 96 ++++
.../hudi/common/util/TestCompactionUtils.java | 10 +-
.../examples/common/ExampleDataSchemaProvider.java | 2 +-
.../common/HoodieExampleDataGenerator.java | 4 +-
.../examples/spark/HoodieWriteClientExample.java | 6 +-
.../hudi/hadoop/TestHoodieROTablePathFilter.java | 98 ++--
hudi-integ-test/README.md | 278 +++++----
hudi-integ-test/prepare_integration_suite.sh | 46 +-
.../hudi/integ/testsuite/HoodieTestSuiteJob.java | 34 +-
.../integ/testsuite/configuration/DeltaConfig.java | 10 +
.../hudi/integ/testsuite/dag/ExecutionContext.java | 16 +-
.../hudi/integ/testsuite/dag/WriterContext.java | 102 ++++
.../integ/testsuite/dag/nodes/HiveSyncNode.java | 1 -
.../hudi/integ/testsuite/dag/nodes/InsertNode.java | 5 +
.../testsuite/dag/scheduler/DagScheduler.java | 8 +-
.../integ/testsuite/generator/DeltaGenerator.java | 3 +-
.../FlexibleSchemaRecordGenerationIterator.java | 6 +-
.../GenericRecordFullPayloadGenerator.java | 17 +-
.../org/apache/hudi/integ/ITTestHoodieDemo.java | 2 -
.../org/apache/hudi/integ/ITTestHoodieSanity.java | 7 +
.../integ/testsuite/dag/HiveSyncDagGenerator.java | 4 +-
.../hudi/integ/testsuite/dag/TestDagUtils.java | 2 +-
.../testsuite/job/TestHoodieTestSuiteJob.java | 56 +-
.../src/test/resources/unit-test-cow-dag.yaml | 35 +-
.../src/test/resources/unit-test-mor-dag.yaml | 35 +-
.../main/java/org/apache/hudi/DataSourceUtils.java | 127 ++---
.../main/java/org/apache/hudi/QuickstartUtils.java | 10 +-
.../internal/HoodieDataSourceInternalWriter.java | 12 +-
.../org/apache/hudi/payload/AWSDmsAvroPayload.java | 23 +-
.../scala/org/apache/hudi/DataSourceOptions.scala | 16 +-
.../org/apache/hudi/HoodieSparkSqlWriter.scala | 107 ++--
.../java/org/apache/hudi/TestDataSourceUtils.java | 33 +-
.../java/org/apache/hudi/client/TestBootstrap.java | 24 +-
.../apache/hudi/payload/TestAWSDmsAvroPayload.java | 132 +++++
.../functional/HoodieSparkSqlWriterSuite.scala | 138 +++--
.../java/org/apache/hudi/hive/HiveSyncConfig.java | 3 +
.../java/org/apache/hudi/hive/HiveSyncTool.java | 16 +-
.../org/apache/hudi/hive/HoodieHiveClient.java | 25 +-
.../hudi/hive/testutils/HiveTestService.java | 2 +
.../timeline/service/FileSystemViewHandler.java | 8 +
.../service/handlers/FileSliceHandler.java | 8 +-
.../org/apache/hudi/utilities/UtilHelpers.java | 58 +-
.../hudi/utilities/deltastreamer/DeltaSync.java | 14 +-
.../deltastreamer/HoodieDeltaStreamer.java | 3 +-
.../deltastreamer/HoodieDeltaStreamerMetrics.java | 6 +
.../deltastreamer/SourceFormatAdapter.java | 25 +-
.../utilities/schema/DelegatingSchemaProvider.java | 8 +
.../utilities/schema/FilebasedSchemaProvider.java | 5 +-
...chemaProvider.java => SchemaPostProcessor.java} | 34 +-
...r.java => SchemaProviderWithPostProcessor.java} | 33 +-
.../hudi/utilities/sources/AvroKafkaSource.java | 8 +-
.../hudi/utilities/sources/JsonKafkaSource.java | 8 +-
.../apache/hudi/utilities/sources/RowSource.java | 5 +-
.../utilities/sources/helpers/KafkaOffsetGen.java | 29 +-
.../apache/hudi/utilities/DummySchemaProvider.java | 13 +-
.../hudi/utilities/TestSchemaPostProcessor.java | 67 +++
.../functional/TestHDFSParquetImporter.java | 12 +-
.../functional/TestHoodieDeltaStreamer.java | 15 +-
.../hudi/utilities/sources/TestKafkaSource.java | 15 +-
.../utilities/testutils/UtilitiesTestBase.java | 2 +-
.../delta-streamer-config/source-flattened.avsc | 2 +-
.../resources/delta-streamer-config/source.avsc | 2 +-
.../source_short_trip_uber.avsc | 2 +-
.../delta-streamer-config/source_uber.avsc | 2 +-
.../delta-streamer-config/target-flattened.avsc | 2 +-
.../resources/delta-streamer-config/target.avsc | 2 +-
.../target_short_trip_uber.avsc | 2 +-
.../delta-streamer-config/target_uber.avsc | 2 +-
packaging/hudi-hadoop-mr-bundle/pom.xml | 4 +-
packaging/hudi-integ-test-bundle/pom.xml | 70 ++-
packaging/hudi-presto-bundle/pom.xml | 4 +-
packaging/hudi-spark-bundle/pom.xml | 4 +-
packaging/hudi-utilities-bundle/pom.xml | 4 +-
scripts/checkout_pr.sh | 61 ++
scripts/pr_push_command.sh | 60 ++
195 files changed, 5966 insertions(+), 3306 deletions(-)
copy hudi-client/src/main/java/org/apache/hudi/exception/HoodieRestoreException.java => hudi-cli/src/main/scala/org/apache/hudi/cli/DeDupeType.scala (78%)
create mode 100644 hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteResult.java
delete mode 100644 hudi-client/src/main/java/org/apache/hudi/metadata/HoodieMetadata.java
rename hudi-client/src/main/java/org/apache/hudi/metadata/{HoodieMetadataImpl.java => HoodieMetadataWriter.java} (54%)
create mode 100644 hudi-client/src/main/java/org/apache/hudi/table/ReplaceArchivalHelper.java
copy hudi-client/src/main/java/org/apache/hudi/table/action/commit/{InsertCommitActionExecutor.java => InsertOverwriteCommitActionExecutor.java} (50%)
copy hudi-client/src/main/java/org/apache/hudi/table/action/commit/{InsertPreppedCommitActionExecutor.java => InsertOverwritePartitioner.java} (53%)
create mode 100644 hudi-client/src/test/java/org/apache/hudi/testutils/HoodieWriteableTestTable.java
copy hudi-utilities/src/test/resources/delta-streamer-config/source.avsc => hudi-client/src/test/resources/testDataGeneratorSchema.txt (99%)
copy hudi-common/src/main/avro/{HoodiePath.avsc => HoodieInstantInfo.avsc} (83%)
copy hudi-common/src/main/avro/{HoodieBootstrapPartitionMetadata.avsc => HoodieReplaceCommitMetadata.avsc} (59%)
create mode 100644 hudi-common/src/main/java/org/apache/hudi/common/model/HoodieReplaceCommitMetadata.java
create mode 100644 hudi-common/src/main/java/org/apache/hudi/common/model/OverwriteNonDefaultsWithLatestAvroPayload.java
create mode 100644 hudi-common/src/main/java/org/apache/hudi/common/util/CommitUtils.java
delete mode 100644 hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataCommon.java
rename {hudi-client => hudi-common}/src/main/java/org/apache/hudi/metadata/HoodieMetadataMergedLogRecordScanner.java (100%)
create mode 100644 hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataReader.java
copy hudi-common/src/test/java/org/apache/hudi/common/model/{TestOverwriteWithLatestAvroPayload.java => TestOverwriteNonDefaultsWithLatestAvroPayload.java} (63%)
create mode 100644 hudi-common/src/test/java/org/apache/hudi/common/properties/TestTypedProperties.java
create mode 100644 hudi-common/src/test/java/org/apache/hudi/common/util/TestCommitUtils.java
create mode 100644 hudi-integ-test/src/main/java/org/apache/hudi/integ/testsuite/dag/WriterContext.java
copy hudi-cli/src/test/resources/log4j-surefire.properties => hudi-integ-test/src/test/resources/unit-test-cow-dag.yaml (55%)
copy hudi-cli/src/test/resources/log4j-surefire.properties => hudi-integ-test/src/test/resources/unit-test-mor-dag.yaml (55%)
create mode 100644 hudi-spark/src/test/java/org/apache/hudi/payload/TestAWSDmsAvroPayload.java
copy hudi-utilities/src/main/java/org/apache/hudi/utilities/schema/{SchemaProvider.java => SchemaPostProcessor.java} (61%)
copy hudi-utilities/src/main/java/org/apache/hudi/utilities/schema/{DelegatingSchemaProvider.java => SchemaProviderWithPostProcessor.java} (51%)
copy hudi-examples/src/main/java/org/apache/hudi/examples/common/ExampleDataSchemaProvider.java => hudi-utilities/src/test/java/org/apache/hudi/utilities/DummySchemaProvider.java (76%)
create mode 100644 hudi-utilities/src/test/java/org/apache/hudi/utilities/TestSchemaPostProcessor.java
create mode 100755 scripts/checkout_pr.sh
create mode 100755 scripts/pr_push_command.sh