You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by si...@apache.org on 2022/05/13 01:02:03 UTC

[hudi] branch master updated (ecd47e7aae -> 0cec955fa2)

This is an automated email from the ASF dual-hosted git repository.

sivabalan pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git


    from ecd47e7aae [HUDI-3963][Claim RFC number 53] Use Lock-Free Message Queue Improving Hoodie Writing Efficiency. (#5562)
     add 0cec955fa2 [HUDI-4018][HUDI-4027] Adding integ test yamls for immutable use-cases. Added delete partition support to integ tests (#5501)

No new revisions were added by this update.

Summary of changes:
 ...g.yaml => deltastreamer-immutable-dataset.yaml} |  44 ++---
 .../deltastreamer-pure-bulk-inserts.yaml           |  41 ++---
 .../test-suite/deltastreamer-pure-inserts.yaml     |  37 ++--
 .../demo/config/test-suite/insert-overwrite.yaml   |   3 +-
 .../demo/config/test-suite/multi-writer-1-ds.yaml  |   2 +-
 ...i-writer-2-sds.yaml => multi-writer-1-sds.yaml} |   8 +-
 .../demo/config/test-suite/multi-writer-2-sds.yaml |   4 +-
 ...i-writer-2-sds.yaml => multi-writer-3-sds.yaml} |  10 +-
 ...i-writer-2-sds.yaml => multi-writer-4-sds.yaml} |  10 +-
 ....properties => multi-writer-local-3.properties} |   2 +-
 ....properties => multi-writer-local-4.properties} |   2 +-
 ...ark-simple.yaml => spark-delete-partition.yaml} |  43 +++--
 ...lustering.yaml => spark-immutable-dataset.yaml} |  44 ++---
 .../test-suite/spark-non-core-operations.yaml      | 204 +++++++++++++++++++++
 .../config/test-suite/spark-pure-bulk-inserts.yaml |  41 ++---
 .../demo/config/test-suite/spark-pure-inserts.yaml |  33 ++--
 hudi-integ-test/README.md                          |  72 ++++++++
 .../testsuite/HoodieMultiWriterTestSuiteJob.java   |  13 +-
 .../integ/testsuite/configuration/DeltaConfig.java |  10 +
 .../dag/nodes/BaseValidateDatasetNode.java         |   7 +-
 .../integ/testsuite/generator/DeltaGenerator.java  |   2 +-
 .../GenericRecordFullPayloadGenerator.java         |   2 +-
 .../testsuite/dag/nodes/SparkBulkInsertNode.scala  |  39 +---
 ...teNode.scala => SparkDeletePartitionNode.scala} |  43 ++---
 .../org/apache/hudi/HoodieSparkSqlWriter.scala     |   2 +-
 .../org/apache/hudi/TestHoodieSparkSqlWriter.scala |   1 -
 26 files changed, 477 insertions(+), 242 deletions(-)
 copy docker/demo/config/test-suite/{spark-medium-clustering.yaml => deltastreamer-immutable-dataset.yaml} (68%)
 copy hudi-integ-test/src/test/resources/unit-test-cow-dag-spark-datasource.yaml => docker/demo/config/test-suite/deltastreamer-pure-bulk-inserts.yaml (59%)
 copy hudi-integ-test/src/test/resources/unit-test-cow-dag-spark-datasource.yaml => docker/demo/config/test-suite/deltastreamer-pure-inserts.yaml (61%)
 copy docker/demo/config/test-suite/{multi-writer-2-sds.yaml => multi-writer-1-sds.yaml} (93%)
 copy docker/demo/config/test-suite/{multi-writer-2-sds.yaml => multi-writer-3-sds.yaml} (91%)
 copy docker/demo/config/test-suite/{multi-writer-2-sds.yaml => multi-writer-4-sds.yaml} (91%)
 copy docker/demo/config/test-suite/{multi-writer-local-2.properties => multi-writer-local-3.properties} (97%)
 copy docker/demo/config/test-suite/{multi-writer-local-2.properties => multi-writer-local-4.properties} (97%)
 copy docker/demo/config/test-suite/{spark-simple.yaml => spark-delete-partition.yaml} (66%)
 copy docker/demo/config/test-suite/{spark-medium-clustering.yaml => spark-immutable-dataset.yaml} (69%)
 create mode 100644 docker/demo/config/test-suite/spark-non-core-operations.yaml
 copy hudi-integ-test/src/test/resources/unit-test-cow-dag-spark-datasource.yaml => docker/demo/config/test-suite/spark-pure-bulk-inserts.yaml (59%)
 copy hudi-integ-test/src/test/resources/unit-test-cow-dag-spark-datasource.yaml => docker/demo/config/test-suite/spark-pure-inserts.yaml (65%)
 copy hudi-integ-test/src/main/scala/org/apache/hudi/integ/testsuite/dag/nodes/{SparkDeleteNode.scala => SparkDeletePartitionNode.scala} (58%)