You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@iceberg.apache.org by ao...@apache.org on 2023/05/23 00:09:29 UTC

[iceberg] branch master updated (7d13b48896 -> 2502a23663)

This is an automated email from the ASF dual-hosted git repository.

aokolnychyi pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git


    from 7d13b48896 Spark 3.3: Add RewritePositionDeleteFilesSparkAction (#7684)
     add 2502a23663 Spark 3.4: Distribution and ordering enhancements (#7637)

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/iceberg/spark/Spark3Util.java  |    7 +
 .../org/apache/iceberg/spark/SparkConfParser.java  |    9 +-
 .../spark/SparkDistributionAndOrderingUtil.java    |  239 -----
 .../org/apache/iceberg/spark/SparkWriteConf.java   |   72 +-
 .../iceberg/spark/SparkWriteRequirements.java}     |   40 +-
 .../org/apache/iceberg/spark/SparkWriteUtil.java   |  242 +++++
 .../spark/actions/SparkShufflingDataRewriter.java  |    4 +-
 .../spark/source/SparkPositionDeltaWrite.java      |   73 +-
 .../source/SparkPositionDeltaWriteBuilder.java     |   34 +-
 .../apache/iceberg/spark/source/SparkWrite.java    |   14 +-
 .../iceberg/spark/source/SparkWriteBuilder.java    |   63 +-
 .../TestSparkDistributionAndOrderingUtil.java      | 1057 +++++++++++++++++---
 .../apache/iceberg/spark/TestSparkWriteConf.java   |   61 +-
 13 files changed, 1287 insertions(+), 628 deletions(-)
 delete mode 100644 spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/SparkDistributionAndOrderingUtil.java
 copy spark/{v3.2/spark/src/main/java/org/apache/iceberg/spark/SparkTableCache.java => v3.4/spark/src/main/java/org/apache/iceberg/spark/SparkWriteRequirements.java} (51%)
 create mode 100644 spark/v3.4/spark/src/main/java/org/apache/iceberg/spark/SparkWriteUtil.java