You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2023/04/16 18:36:23 UTC
[iceberg] branch dependabot/pip/python/pandas-2.0.0 updated (94fe702adb -> 07dd815604)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch dependabot/pip/python/pandas-2.0.0
in repository https://gitbox.apache.org/repos/asf/iceberg.git
discard 94fe702adb Build: Bump pandas from 1.5.3 to 2.0.0 in /python
add 9f3707bc7f AWS: Check commit status after failed commit if AWS client performed retries (#7198)
add 4f49a3772e Core: Fix errorprone warning (#7286)
add 198cfc94a7 Build: Bump Nessie to 0.56.0 (#7283)
add 4be7d49a2e Build: Bump actions/stale from 7.0.0 to 8.0.0 (#7200)
add 1c6ee94025 Build: Bump org.apache.hadoop:hadoop-client from 3.3.4 to 3.3.5 (#7201)
add 92ef136a0e Spark 3.1, 3.2: Broadcast Table instead of FileIO in rewrite manifests (#7263) (#7296)
add 3aaae6b27f Build: `iceberg-delta-lake` Spark version to 3.3.2 (#7199)
add af4ddd2547 Nessie: Use latest hash for catalog APIs (#6789)
add 0123862618 Arrow: Support vectorized read of INT96 timestamp in imported data (#6962)
add cd56cee2f8 Flink: Expose write-parallelism in SQL Hints (#7039)
add fc6bd1e7ab Nessie: Fix testcase failures (#7320)
add 9f9e189ed4 Flink: move the classes from flink.sink.shuffle.statistics pkg to one level up as flink.sink.shuffle pkg (#7322)
add 982e493037 Spark 3.3: Add doc for the changelog view procedure. (#7147)
add df31406b3d Build: Bump Nessie from 0.56.0 to 0.57.0 (#7323)
add 325d282766 Flink 1.15: Port Expose write-parallelism in SQL Hints to 1.15
add 09a1efc0c0 Flink 1.17: Port Expose write-parallelism in SQL Hints to 1.17
add 6e4a6ff0cd Update issue template for 1.2.1 release (#7331)
add e7df8c1a5c Core: Fix SnapshotProducer#targetBranch exception message (#7315)
add 430702bae7 Build: Bump Gradle from 8.0.2 to 8.1 (#7333)
add d29cf6b6e3 Build: Fix flaky checkstyle issue (#7321)
add ecc2492aa6 Infra: Update vote mail sample in source-release.sh (#7330)
add db424ea5d6 Core: Add missing metrics reporters when creating BaseTable (#7341)
add f53c4eeec5 Core, Spark 3.3: Add FileRewriter API (#7175)
add 522480dc4d Spark: Accept an `output-spec-id` to write to a desired partition spec (#7120)
add b756a5da35 ORC, Spark 3.3: Support selected vector with ORC row and batch readers (#7197)
add b78d336163 Flink: use correct scan mode when in TABLE_SCAN_THEN_INCREMENTAL mode (#7338)
add dc05e1f867 Build: Bump rich from 13.3.3 to 13.3.4 in /python (#7354)
add 07dd815604 Build: Bump pandas from 1.5.3 to 2.0.0 in /python
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (94fe702adb)
\
N -- N -- N refs/heads/dependabot/pip/python/pandas-2.0.0 (07dd815604)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/ISSUE_TEMPLATE/iceberg_bug_report.yml | 3 +-
.github/workflows/stale.yml | 2 +-
.../apache/iceberg/io/TestCloseableIterable.java | 5 +-
.../GenericArrowVectorAccessorFactory.java | 30 ++
.../arrow/vectorized/VectorizedArrowReader.java | 17 +
.../parquet/VectorizedColumnIterator.java | 18 +
...orizedDictionaryEncodedParquetValuesReader.java | 17 +
.../vectorized/parquet/VectorizedPageIterator.java | 30 ++
.../VectorizedParquetDefinitionLevelReader.java | 51 +++
.../org/apache/iceberg/aws/glue/GlueTestBase.java | 11 +-
.../aws/glue/TestGlueCatalogCommitFailure.java | 87 ++++-
.../aws/dynamodb/DynamoDbTableOperations.java | 35 +-
.../iceberg/aws/glue/GlueTableOperations.java | 93 +++--
.../org/apache/iceberg/aws/util/RetryDetector.java | 51 +++
.../apache/iceberg/aws/util/TestRetryDetector.java | 156 ++++++++
baseline.gradle | 10 +
build.gradle | 6 +-
.../org/apache/iceberg/BaseMetastoreCatalog.java | 4 +-
.../java/org/apache/iceberg/SnapshotProducer.java | 5 +-
.../apache/iceberg/actions/BinPackStrategy.java | 3 +
.../org/apache/iceberg/actions/FileRewriter.java | 77 ++++
.../apache/iceberg/actions/RewriteStrategy.java | 6 +
.../iceberg/actions/SizeBasedDataRewriter.java | 109 ++++++
.../iceberg/actions/SizeBasedFileRewriter.java | 301 ++++++++++++++++
.../org/apache/iceberg/actions/SortStrategy.java | 3 +
.../apache/iceberg/inmemory/InMemoryInputFile.java | 2 +-
.../apache/iceberg/rest/RESTSessionCatalog.java | 3 +-
.../java/org/apache/iceberg/TestFastAppend.java | 19 +-
.../test/java/org/apache/iceberg/TestMetrics.java | 2 +-
dev/source-release.sh | 6 +-
docs/flink-configuration.md | 1 +
docs/spark-procedures.md | 116 ++++++
.../org/apache/iceberg/flink/FlinkWriteConf.java | 4 +
.../apache/iceberg/flink/FlinkWriteOptions.java | 3 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 9 +-
.../org/apache/iceberg/flink/DataGenerators.java | 5 +-
.../apache/iceberg/flink/TestFlinkTableSink.java | 40 +++
.../org/apache/iceberg/flink/FlinkWriteConf.java | 4 +
.../apache/iceberg/flink/FlinkWriteOptions.java | 3 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 9 +-
.../flink/sink/shuffle}/DataStatistics.java | 4 +-
.../flink/sink/shuffle/DataStatisticsEvent.java | 1 -
.../flink/sink/shuffle}/DataStatisticsFactory.java | 4 +-
.../flink/sink/shuffle/DataStatisticsOperator.java | 2 -
.../flink/sink/shuffle/DataStatisticsOrRecord.java | 3 +-
.../flink/sink/shuffle}/MapDataStatistics.java | 4 +-
.../{statistics => }/MapDataStatisticsFactory.java | 4 +-
.../org/apache/iceberg/flink/DataGenerators.java | 5 +-
.../apache/iceberg/flink/TestFlinkTableSink.java | 40 +++
.../sink/shuffle/TestDataStatisticsOperator.java | 4 -
.../org/apache/iceberg/flink/FlinkWriteConf.java | 4 +
.../apache/iceberg/flink/FlinkWriteOptions.java | 3 +
.../org/apache/iceberg/flink/sink/FlinkSink.java | 9 +-
.../flink/sink/shuffle}/DataStatistics.java | 4 +-
.../flink/sink/shuffle/DataStatisticsEvent.java | 1 -
.../flink/sink/shuffle}/DataStatisticsFactory.java | 4 +-
.../flink/sink/shuffle/DataStatisticsOperator.java | 2 -
.../flink/sink/shuffle/DataStatisticsOrRecord.java | 3 +-
.../flink/sink/shuffle}/MapDataStatistics.java | 4 +-
.../{statistics => }/MapDataStatisticsFactory.java | 4 +-
.../iceberg/flink/source/FlinkSplitPlanner.java | 10 +-
.../enumerator/ContinuousSplitPlannerImpl.java | 4 +-
.../org/apache/iceberg/flink/DataGenerators.java | 5 +-
.../apache/iceberg/flink/TestFlinkTableSink.java | 40 +++
.../sink/shuffle/TestDataStatisticsOperator.java | 4 -
.../flink/source/TestIcebergSourceContinuous.java | 51 +++
gradle/wrapper/gradle-wrapper.properties | 4 +-
hive3/build.gradle | 2 +-
.../apache/iceberg/nessie/NessieIcebergClient.java | 75 ++--
.../iceberg/nessie/NessieTableOperations.java | 44 +++
.../apache/iceberg/nessie/UpdateableReference.java | 15 +-
.../apache/iceberg/nessie/TestMultipleClients.java | 160 +++++++++
.../java/org/apache/iceberg/orc/OrcIterable.java | 6 +-
.../org/apache/iceberg/orc/TestOrcDataReader.java | 170 +++++++++
.../org/apache/iceberg/parquet/ParquetUtil.java | 16 +
.../iceberg/parquet/TestBloomRowGroupFilter.java | 2 +-
python/poetry.lock | 8 +-
python/pyproject.toml | 2 +-
.../iceberg/spark/data/SparkParquetReaders.java | 9 +-
.../java/org/apache/iceberg/spark/SparkUtil.java | 8 +
.../org/apache/iceberg/spark/SparkWriteConf.java | 20 ++
.../apache/iceberg/spark/SparkWriteOptions.java | 2 +
.../actions/BaseRewriteManifestsSparkAction.java | 59 ++-
.../iceberg/spark/data/SparkParquetReaders.java | 9 +-
.../apache/iceberg/spark/source/SparkWrite.java | 15 +-
.../iceberg/spark/sql/TestPartitionedWrites.java | 85 +++++
.../java/org/apache/iceberg/spark/SparkUtil.java | 8 +
.../org/apache/iceberg/spark/SparkWriteConf.java | 20 ++
.../apache/iceberg/spark/SparkWriteOptions.java | 2 +
.../spark/actions/RewriteManifestsSparkAction.java | 32 +-
.../iceberg/spark/data/SparkParquetReaders.java | 9 +-
.../apache/iceberg/spark/source/SparkWrite.java | 8 +-
.../source/TestSparkReaderWithBloomFilter.java | 2 +-
.../iceberg/spark/sql/TestPartitionedWrites.java | 85 +++++
spark/v3.3/build.gradle | 4 +
.../extensions/TestRewriteDataFilesProcedure.java | 6 +-
.../java/org/apache/iceberg/spark/SparkUtil.java | 8 +
.../org/apache/iceberg/spark/SparkWriteConf.java | 20 ++
.../apache/iceberg/spark/SparkWriteOptions.java | 2 +
.../spark/actions/RewriteDataFilesSparkAction.java | 66 ++--
.../spark/actions/SparkBinPackDataRewriter.java | 82 +++++
.../spark/actions/SparkBinPackStrategy.java | 6 +
.../spark/actions/SparkShufflingDataRewriter.java | 138 +++++++
.../spark/actions/SparkSizeBasedDataRewriter.java | 67 ++++
.../spark/actions/SparkSortDataRewriter.java | 60 ++++
.../iceberg/spark/actions/SparkSortStrategy.java | 6 +
.../spark/actions/SparkZOrderDataRewriter.java | 184 ++++++++++
.../iceberg/spark/actions/SparkZOrderStrategy.java | 6 +
.../iceberg/spark/data/SparkParquetReaders.java | 9 +-
.../data/vectorized/VectorizedSparkOrcReaders.java | 87 +++--
.../apache/iceberg/spark/source/SparkWrite.java | 8 +-
.../spark/actions/TestRewriteDataFilesAction.java | 12 +-
.../spark/actions/TestSparkFileRewriter.java | 396 +++++++++++++++++++++
.../spark/data/TestVectorizedOrcDataReader.java | 166 +++++++++
.../source/TestIcebergSourceHadoopTables.java | 5 +
.../spark/source/TestIcebergSourceHiveTables.java | 19 +-
.../spark/source/TestIcebergSourceTablesBase.java | 190 ++++++----
.../source/TestSparkReaderWithBloomFilter.java | 2 +-
.../spark/sql/PartitionedWritesTestBase.java | 96 +++++
.../sql/TestPartitionedWritesToWapBranch.java | 8 +
versions.props | 2 +-
121 files changed, 3708 insertions(+), 403 deletions(-)
create mode 100644 aws/src/main/java/org/apache/iceberg/aws/util/RetryDetector.java
create mode 100644 aws/src/test/java/org/apache/iceberg/aws/util/TestRetryDetector.java
create mode 100644 core/src/main/java/org/apache/iceberg/actions/FileRewriter.java
create mode 100644 core/src/main/java/org/apache/iceberg/actions/SizeBasedDataRewriter.java
create mode 100644 core/src/main/java/org/apache/iceberg/actions/SizeBasedFileRewriter.java
rename flink/{v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/statistics => v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle}/DataStatistics.java (94%)
rename flink/{v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/statistics => v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle}/DataStatisticsFactory.java (91%)
rename flink/{v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/statistics => v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle}/MapDataStatistics.java (94%)
rename flink/v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/{statistics => }/MapDataStatisticsFactory.java (88%)
rename flink/{v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/statistics => v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle}/DataStatistics.java (94%)
rename flink/{v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/statistics => v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle}/DataStatisticsFactory.java (91%)
rename flink/{v1.16/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/statistics => v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle}/MapDataStatistics.java (94%)
rename flink/v1.17/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/{statistics => }/MapDataStatisticsFactory.java (88%)
create mode 100644 nessie/src/test/java/org/apache/iceberg/nessie/TestMultipleClients.java
create mode 100644 orc/src/test/java/org/apache/iceberg/orc/TestOrcDataReader.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/SparkBinPackDataRewriter.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/SparkShufflingDataRewriter.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/SparkSizeBasedDataRewriter.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/SparkSortDataRewriter.java
create mode 100644 spark/v3.3/spark/src/main/java/org/apache/iceberg/spark/actions/SparkZOrderDataRewriter.java
create mode 100644 spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/actions/TestSparkFileRewriter.java
create mode 100644 spark/v3.3/spark/src/test/java/org/apache/iceberg/spark/data/TestVectorizedOrcDataReader.java