You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by ao...@apache.org on 2022/10/07 17:23:54 UTC
[iceberg] branch master updated (09e6f34e1e -> dc5f5c38f8)

This is an automated email from the ASF dual-hosted git repository.

aokolnychyi pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/iceberg.git


    from 09e6f34e1e Core: Deprecate write.manifest-lists.enabled (#5773)
     add dc5f5c38f8 Spark 3.3: Add SparkChangelogTable (#5740)

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/iceberg/ChangelogUtil.java     |  49 ++++
 .../java/org/apache/iceberg/MetadataColumns.java   |  18 ++
 .../main/java/org/apache/iceberg/hadoop/Util.java  |  24 +-
 .../spark/extensions/TestChangelogBatchReads.java  | 248 +++++++++++++++++++++
 .../org/apache/iceberg/spark/SparkCatalog.java     | 126 +++++++----
 .../iceberg/spark/source/ChangelogRowReader.java   |   3 +-
 .../apache/iceberg/spark/source/SparkBatch.java    |  11 +-
 .../{SparkBatch.java => SparkChangelogBatch.java}  | 115 +++++-----
 .../iceberg/spark/source/SparkChangelogScan.java   | 163 ++++++++++++++
 .../iceberg/spark/source/SparkChangelogTable.java  | 102 +++++++++
 .../iceberg/spark/source/SparkInputPartition.java  |  84 +++++++
 .../spark/source/SparkMicroBatchStream.java        |  12 +-
 .../org/apache/iceberg/spark/source/SparkScan.java | 124 ++++-------
 .../iceberg/spark/source/SparkScanBuilder.java     |  37 ++-
 .../apache/iceberg/spark/source/SparkTable.java    |   8 +
 .../actions/TestRemoveOrphanFilesAction3.java      |   6 +-
 .../iceberg/spark/source/TestPathIdentifier.java   |   5 +-
 17 files changed, 925 insertions(+), 210 deletions(-)
 create mode 100644 core/src/main/java/org/apache/iceberg/ChangelogUtil.java
 create mode 100644 spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestChangelogBatchReads.java
 copy spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/{SparkBatch.java => SparkChangelogBatch.java} (50%)
 create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkChangelogScan.java
 create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkChangelogTable.java
 create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/source/SparkInputPartition.java