You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iceberg.apache.org by gi...@apache.org on 2023/07/27 10:45:41 UTC
[iceberg-docs] branch asf-site updated: deploy: 1be0f0c4af5969ca307b15377b6c267bfd274bd6
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/iceberg-docs.git
The following commit(s) were added to refs/heads/asf-site by this push:
new eab2cabb deploy: 1be0f0c4af5969ca307b15377b6c267bfd274bd6
eab2cabb is described below
commit eab2cabb1d0943e71e3b07d13cc43c4b2832740a
Author: nastra <na...@users.noreply.github.com>
AuthorDate: Thu Jul 27 10:45:36 2023 +0000
deploy: 1be0f0c4af5969ca307b15377b6c267bfd274bd6
---
docs/latest/api/index.html | 10 +++++-----
docs/latest/aws/index.html | 10 +++++-----
docs/latest/branching/index.html | 2 +-
docs/latest/configuration/index.html | 2 +-
docs/latest/custom-catalog/index.html | 2 +-
docs/latest/dell/index.html | 2 +-
docs/latest/delta-lake-migration/index.html | 2 +-
docs/latest/docssearch.json | 2 +-
docs/latest/evolution/index.html | 2 +-
docs/latest/flink-actions/index.html | 4 ++--
docs/latest/flink-configuration/index.html | 2 +-
docs/latest/flink-connector/index.html | 2 +-
docs/latest/flink-ddl/index.html | 2 +-
docs/latest/flink-queries/index.html | 6 +++---
docs/latest/flink-writes/index.html | 2 +-
docs/latest/flink/index.html | 10 +++++-----
docs/latest/getting-started/index.html | 8 ++++----
docs/latest/hive-migration/index.html | 2 +-
docs/latest/hive/index.html | 4 ++--
docs/latest/index.html | 2 +-
docs/latest/index.xml | 2 +-
docs/latest/java-api-quickstart/index.html | 4 ++--
docs/latest/jdbc/index.html | 4 ++--
docs/latest/maintenance/index.html | 12 ++++++------
docs/latest/nessie/index.html | 8 ++++----
docs/latest/partitioning/index.html | 2 +-
docs/latest/performance/index.html | 2 +-
docs/latest/reliability/index.html | 2 +-
docs/latest/schemas/index.html | 2 +-
docs/latest/spark-configuration/index.html | 2 +-
docs/latest/spark-ddl/index.html | 2 +-
docs/latest/spark-procedures/index.html | 10 +++++-----
docs/latest/spark-queries/index.html | 2 +-
docs/latest/spark-structured-streaming/index.html | 2 +-
docs/latest/spark-writes/index.html | 2 +-
docs/latest/table-migration/index.html | 2 +-
36 files changed, 69 insertions(+), 69 deletions(-)
diff --git a/docs/latest/api/index.html b/docs/latest/api/index.html
index ad47e5ee..5a3a2af4 100644
--- a/docs/latest/api/index.html
+++ b/docs/latest/api/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,7 +15,7 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a id=active href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
</span></span></code></pre></div><p>To configure a scan, call <code>filter</code> and <code>select</code> on the <code>TableScan</code> to get a new <code>TableScan</code> with those changes.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>TableScan filteredScan <span style=color:#f92672>=</span> scan<span style=color:#f92672>.</span><span [...]
</span></span></code></pre></div><p>Calls to configuration methods create a new <code>TableScan</code> so that each <code>TableScan</code> is immutable and won’t change unexpectedly if shared across threads.</p><p>When a scan is configured, <code>planFiles</code>, <code>planTasks</code>, and <code>schema</code> are used to return files, tasks, and the read projection.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4; [...]
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span style=color:#a6e22e>equal</span><span style=color:#f92672>(</span><span style=color:#e6db74>"id"</span><span style=color:#f92672>,</span> <span style=color:#ae81ff>5</span><span style=color:#f92672>))</span>
@@ -28,7 +28,7 @@
</span></span></code></pre></div><p>When a scan is configured, call method <code>build</code> to execute scan. <code>build</code> return <code>CloseableIterable<Record></code></p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>CloseableIterable<span style=color:#f92672><</span>Record<span style=color:#f92672>></span> result <span styl [...]
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>where</span><span style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span style=color:#a6e22e>lessThan</span><span style=color:#f92672>(</span><span style=color:#e6db74>"id"</span><span style=color:#f92672>,</span> <span style=color:#ae81ff>5</span><span style=color:#f92672>))</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>build</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>where <code>Record</code> is Iceberg record for iceberg-data module <code>org.apache.iceberg.data.Record</code>.</p><h3 id=update-operations>Update operations</h3><p><code>Table</code> also exposes operations that update the table. These operations use a builder pattern, <a href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/PendingUpdate.html><code>PendingUpdate</code></a>, that commits when <code>PendingUpdate#commit</code> is called.</p><p>For [...]
+</span></span></code></pre></div><p>where <code>Record</code> is Iceberg record for iceberg-data module <code>org.apache.iceberg.data.Record</code>.</p><h3 id=update-operations>Update operations</h3><p><code>Table</code> also exposes operations that update the table. These operations use a builder pattern, <a href=../../../javadoc/1.3.1/index.html?org/apache/iceberg/PendingUpdate.html><code>PendingUpdate</code></a>, that commits when <code>PendingUpdate#commit</code> is called.</p><p>For [...]
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>addColumn</span><span style=color:#f92672>(</span><span style=color:#e6db74>"count"</span><span style=color:#f92672>,</span> Types<span style=color:#f92672>.</span><span style=color:#a6e22e>LongType</span><span style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span style=color:#f92672>())</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>commit</span><span style=color:#f92672>();</span>
</span></span></code></pre></div><p>Available operations to update a table are:</p><ul><li><code>updateSchema</code> – update the table schema</li><li><code>updateProperties</code> – update table properties</li><li><code>updateLocation</code> – update the table’s base location</li><li><code>newAppend</code> – used to append data files</li><li><code>newFastAppend</code> – used to append data files, will not compact metadata</li><li><code>newOverwrite</c [...]
@@ -39,7 +39,7 @@
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>// commit all the changes to the table
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>t<span style=color:#f92672>.</span><span style=color:#a6e22e>commitTransaction</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><h2 id=types>Types</h2><p>Iceberg data types are located in the <a href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/types/package-summary.html><code>org.apache.iceberg.types</code> package</a>.</p><h3 id=primitives>Primitives</h3><p>Primitive type instances are available from static methods in each type class. Types without parameters use <code>get</code>, and types like <code>decimal</code> use factory methods:</p><div class=highlight><pre tabin [...]
+</span></span></code></pre></div><h2 id=types>Types</h2><p>Iceberg data types are located in the <a href=../../../javadoc/1.3.1/index.html?org/apache/iceberg/types/package-summary.html><code>org.apache.iceberg.types</code> package</a>.</p><h3 id=primitives>Primitives</h3><p>Primitive type instances are available from static methods in each type class. Types without parameters use <code>get</code>, and types like <code>decimal</code> use factory methods:</p><div class=highlight><pre tabin [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>Types<span style=color:#f92672>.</span><span style=color:#a6e22e>DoubleType</span><span style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span style=color:#f92672>()</span> <span style=color:#75715e>// double
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>Types<span style=color:#f92672>.</span><span style=color:#a6e22e>DecimalType</span><span style=color:#f92672>.</span><span style=color:#a6e22e>of</span><span style=color:#f92672>(</span><span style=color:#ae81ff>9</span><span style=color:#f92672>,</span> <span style=color:#ae81ff>2</span><span style=color:#f92672>)</span> <span style=color:#75715e>// decimal(9, 2)
</span></span></span></code></pre></div><h3 id=nested-types>Nested types</h3><p>Structs, maps, and lists are created using factory methods in type classes.</p><p>Like struct fields, map keys or values and list elements are tracked as nested fields. Nested fields track <a href=../evolution#correctness>field IDs</a> and nullability.</p><p>Struct fields are created using <code>NestedField.optional</code> or <code>NestedField.required</code>. Map value and list element nullability is set in [...]
@@ -55,7 +55,7 @@
</span></span><span style=display:flex><span> <span style=color:#f92672>)</span>
</span></span></code></pre></div><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span><span style=color:#75715e>// array<1 element: int>
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>ListType list <span style=color:#f92672>=</span> ListType<span style=color:#f92672>.</span><span style=color:#a6e22e>ofRequired</span><span style=color:#f92672>(</span><span style=color:#ae81ff>1</span><span style=color:#f92672>,</span> IntegerType<span style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span style=color:#f92672>());</span>
-</span></span></code></pre></div><h2 id=expressions>Expressions</h2><p>Iceberg’s expressions are used to configure table scans. To create expressions, use the factory methods in <a href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/expressions/Expressions.html><code>Expressions</code></a>.</p><p>Supported predicate expressions are:</p><ul><li><code>isNull</code></li><li><code>notNull</code></li><li><code>equal</code></li><li><code>notEqual</code></li><li><code>lessThan</code [...]
+</span></span></code></pre></div><h2 id=expressions>Expressions</h2><p>Iceberg’s expressions are used to configure table scans. To create expressions, use the factory methods in <a href=../../../javadoc/1.3.1/index.html?org/apache/iceberg/expressions/Expressions.html><code>Expressions</code></a>.</p><p>Supported predicate expressions are:</p><ul><li><code>isNull</code></li><li><code>notNull</code></li><li><code>equal</code></li><li><code>notEqual</code></li><li><code>lessThan</code [...]
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span style=color:#a6e22e>greaterThanOrEqual</span><span style=color:#f92672>(</span><span style=color:#e6db74>"x"</span><span style=color:#f92672>,</span> <span style=color:#ae81ff>5</span><span style=color:#f92672>))</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span style=color:#a6e22e>lessThan</span><span style=color:#f92672>(</span><span style=color:#e6db74>"x"</span><span style=color:#f92672>,</span> <span style=color:#ae81ff>10</span><span style=color:#f92672>))</span>
</span></span></code></pre></div><h2 id=modules>Modules</h2><p>Iceberg table support is organized in library modules:</p><ul><li><code>iceberg-common</code> contains utility classes used in other modules</li><li><code>iceberg-api</code> contains the public Iceberg API, including expressions, types, tables, and operations</li><li><code>iceberg-arrow</code> is an implementation of the Iceberg type system for reading and writing data stored in Iceberg tables using Apache Arrow as the in-mem [...]
diff --git a/docs/latest/aws/index.html b/docs/latest/aws/index.html
index 108d110c..84296476 100644
--- a/docs/latest/aws/index.html
+++ b/docs/latest/aws/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -27,7 +27,7 @@ To choose a different HTTP client library such as <a href=https://mvnrepository.
see the section <a href=#aws-client-customization>client customization</a> for more details.</p><p>All the AWS module features can be loaded through custom catalog properties,
you can go to the documentations of each engine to see how to load a custom catalog.
Here are some examples.</p><h3 id=spark>Spark</h3><p>For example, to use AWS features with Spark 3.3 (with scala 2.12) and AWS clients version 2.20.18, you can start the Spark SQL shell with:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=display:flex><span><span style=color:#75715e># add Iceberg dependency</span>
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.0
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.1
</span></span><span style=display:flex><span>DEPENDENCIES<span style=color:#f92672>=</span><span style=color:#e6db74>"org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:</span>$ICEBERG_VERSION<span style=color:#e6db74>"</span>
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e># add AWS dependency</span>
@@ -48,7 +48,7 @@ Here are some examples.</p><h3 id=spark>Spark</h3><p>For example, to use AWS fea
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog.catalog-impl<span style=color:#f92672>=</span>org.apache.iceberg.aws.glue.GlueCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog.io-impl<span style=color:#f92672>=</span>org.apache.iceberg.aws.s3.S3FileIO
</span></span></code></pre></div><p>As you can see, In the shell command, we use <code>--packages</code> to specify the additional AWS bundle and HTTP client dependencies with their version as <code>2.20.18</code>.</p><h3 id=flink>Flink</h3><p>To use AWS module with Flink, you can download the necessary dependencies and specify them when starting the Flink SQL client:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-si [...]
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.0
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.1
</span></span><span style=display:flex><span>MAVEN_URL<span style=color:#f92672>=</span>https://repo1.maven.org/maven2
</span></span><span style=display:flex><span>ICEBERG_MAVEN_URL<span style=color:#f92672>=</span>$MAVEN_URL/org/apache/iceberg
</span></span><span style=display:flex><span>wget $ICEBERG_MAVEN_URL/iceberg-flink-runtime/$ICEBERG_VERSION/iceberg-flink-runtime-$ICEBERG_VERSION.jar
@@ -231,7 +231,7 @@ In this case, a <a href=https://docs.aws.amazon.com/IAM/latest/UserGuide/id_role
Iceberg provides an AWS client factory <code>AssumeRoleAwsClientFactory</code> to support this common use case.
This also serves as an example for users who would like to implement their own AWS client factory.</p><p>This client factory has the following configurable catalog properties:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>client.assume-role.arn</td><td>null, requires user input</td><td>ARN of the role to assume, e.g. arn:aws:iam::123456789:role/myRoleToAssume</td></tr><tr><td>client.assume-role.region</td><td>null, requires user inp [...]
The Glue, S3 and DynamoDB clients are then initialized with the assume-role credential and region to access resources.
-Here is an example to start Spark shell with this client factory:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-shell data-lang=shell><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime:1.3.0,software.amazon.awssdk:bundle:2.20.18 <span style=color:#ae81ff>\
+Here is an example to start Spark shell with this client factory:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-shell data-lang=shell><span style=display:flex><span>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime:1.3.1,software.amazon.awssdk:bundle:2.20.18 <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog.warehouse<span style=color:#f92672>=</span>s3://my-bucket/my/key/prefix <span style=color:#ae81ff>\ </span>
</span></span><span style=display:flex><span> --conf spark.sql.catalog.my_catalog.catalog-impl<span style=color:#f92672>=</span>org.apache.iceberg.aws.glue.GlueCatalog <span style=color:#ae81ff>\
@@ -251,7 +251,7 @@ More details could be found <a href=https://docs.aws.amazon.com/athena/latest/ug
Please refer to the <a href=https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-iceberg-use-cluster.html>official documentation</a> on how to create a cluster with Iceberg installed.</p><p>For versions before 6.5.0, you can use a <a href=https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-bootstrap.html>bootstrap action</a> similar to the following to pre-install all necessary dependencies:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#2728 [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>
</span></span><span style=display:flex><span>AWS_SDK_VERSION<span style=color:#f92672>=</span>2.20.18
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.0
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.1
</span></span><span style=display:flex><span>MAVEN_URL<span style=color:#f92672>=</span>https://repo1.maven.org/maven2
</span></span><span style=display:flex><span>ICEBERG_MAVEN_URL<span style=color:#f92672>=</span>$MAVEN_URL/org/apache/iceberg
</span></span><span style=display:flex><span>AWS_MAVEN_URL<span style=color:#f92672>=</span>$MAVEN_URL/software/amazon/awssdk
diff --git a/docs/latest/branching/index.html b/docs/latest/branching/index.html
index 58f5b5aa..ab6d1600 100644
--- a/docs/latest/branching/index.html
+++ b/docs/latest/branching/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a id=active href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/configuration/index.html b/docs/latest/configuration/index.html
index 7df19372..529865ca 100644
--- a/docs/latest/configuration/index.html
+++ b/docs/latest/configuration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a id=active href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/custom-catalog/index.html b/docs/latest/custom-catalog/index.html
index 2337a336..3dd05781 100644
--- a/docs/latest/custom-catalog/index.html
+++ b/docs/latest/custom-catalog/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/dell/index.html b/docs/latest/dell/index.html
index 8a3a0c39..0b447e5c 100644
--- a/docs/latest/dell/index.html
+++ b/docs/latest/dell/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/delta-lake-migration/index.html b/docs/latest/delta-lake-migration/index.html
index 9375f446..5ad600e2 100644
--- a/docs/latest/delta-lake-migration/index.html
+++ b/docs/latest/delta-lake-migration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/docssearch.json b/docs/latest/docssearch.json
index cb2588c8..a728eb0e 100644
--- a/docs/latest/docssearch.json
+++ b/docs/latest/docssearch.json
@@ -1 +1 @@
-[{"categories":null,"content":" Getting Started The latest version of Iceberg is 1.3.0.\nSpark is currently the most feature-rich compute engine for Iceberg operations. We recommend you to get started with Spark to understand Iceberg concepts and features with examples. You can also view documentations of using Iceberg with other compute engine under the Multi-Engine Support page.\nUsing Iceberg in Spark 3 To use Iceberg in a Spark shell, use the --packages option:\nspark-shell --package [...]
\ No newline at end of file
+[{"categories":null,"content":" Getting Started The latest version of Iceberg is 1.3.1.\nSpark is currently the most feature-rich compute engine for Iceberg operations. We recommend you to get started with Spark to understand Iceberg concepts and features with examples. You can also view documentations of using Iceberg with other compute engine under the Multi-Engine Support page.\nUsing Iceberg in Spark 3 To use Iceberg in a Spark shell, use the --packages option:\nspark-shell --package [...]
\ No newline at end of file
diff --git a/docs/latest/evolution/index.html b/docs/latest/evolution/index.html
index fe32077e..b69d9de7 100644
--- a/docs/latest/evolution/index.html
+++ b/docs/latest/evolution/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a id=active href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-actions/index.html b/docs/latest/flink-actions/index.html
index ba75450a..b4b422f9 100644
--- a/docs/latest/flink-actions/index.html
+++ b/docs/latest/flink-actions/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -22,7 +22,7 @@
</span></span><span style=display:flex><span>RewriteDataFilesActionResult result <span style=color:#f92672>=</span> Actions<span style=color:#f92672>.</span><span style=color:#a6e22e>forTable</span><span style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>rewriteDataFiles</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>For more details of the rewrite files action, please refer to <a href=../../../javadoc/1.3.0/org/apache/iceberg/flink/actions/RewriteDataFilesAction.html>RewriteDataFilesAction</a></p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#rewrite-files-action>Rewrite files action.</a></li></ul></nav></div></div></div></div></section></body><script src=https://iceberg.apache.org/docs/latest//js/jquery-1.11.0.js></script>
+</span></span></code></pre></div><p>For more details of the rewrite files action, please refer to <a href=../../../javadoc/1.3.1/org/apache/iceberg/flink/actions/RewriteDataFilesAction.html>RewriteDataFilesAction</a></p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#rewrite-files-action>Rewrite files action.</a></li></ul></nav></div></div></div></div></section></body><script src=https://iceberg.apache.org/docs/latest//js/jquery-1.11.0.js></script>
<script src=https://iceberg.apache.org/docs/latest//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/latest//js/search.js></script>
<script src=https://iceberg.apache.org/docs/latest//js/bootstrap.min.js></script>
diff --git a/docs/latest/flink-configuration/index.html b/docs/latest/flink-configuration/index.html
index 69b12779..b90ba8b7 100644
--- a/docs/latest/flink-configuration/index.html
+++ b/docs/latest/flink-configuration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-connector/index.html b/docs/latest/flink-connector/index.html
index 67bb1225..08841d72 100644
--- a/docs/latest/flink-connector/index.html
+++ b/docs/latest/flink-connector/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-ddl/index.html b/docs/latest/flink-ddl/index.html
index 8b1f8486..2c84dede 100644
--- a/docs/latest/flink-ddl/index.html
+++ b/docs/latest/flink-ddl/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink-queries/index.html b/docs/latest/flink-queries/index.html
index ddaf0cd9..bd11acaf 100644
--- a/docs/latest/flink-queries/index.html
+++ b/docs/latest/flink-queries/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -72,7 +72,7 @@ refer to <a href=../flink-configuration/#read-options>Flink Configuration</a></p
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e>// Submit and execute this streaming read job.
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>env<span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>(</span><span style=color:#e6db74>"Test Iceberg Streaming Read"</span><span style=color:#f92672>);</span>
-</span></span></code></pre></div><p>There are other options that can be set, please see the <a href=../../../javadoc/1.3.0/org/apache/iceberg/flink/source/FlinkSource.html>FlinkSource#Builder</a>.</p><h2 id=reading-with-datastream-flip-27-source>Reading with DataStream (FLIP-27 source)</h2><p><a href=https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface>FLIP-27 source interface</a>
+</span></span></code></pre></div><p>There are other options that can be set, please see the <a href=../../../javadoc/1.3.1/org/apache/iceberg/flink/source/FlinkSource.html>FlinkSource#Builder</a>.</p><h2 id=reading-with-datastream-flip-27-source>Reading with DataStream (FLIP-27 source)</h2><p><a href=https://cwiki.apache.org/confluence/display/FLINK/FLIP-27%3A+Refactor+Source+Interface>FLIP-27 source interface</a>
was introduced in Flink 1.12. It aims to solve several shortcomings of the old <code>SourceFunction</code>
streaming source interface. It also unifies the source interfaces for both batch and streaming executions.
Most source connectors (like Kafka, file) in Flink repo have migrated to the FLIP-27 interface.
@@ -120,7 +120,7 @@ CDC read is not supported yet.</p><div class=highlight><pre tabindex=0 style=col
</span></span><span style=display:flex><span><span style=color:#75715e>// Submit and execute this streaming read job.
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>env<span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>(</span><span style=color:#e6db74>"Test Iceberg Streaming Read"</span><span style=color:#f92672>);</span>
</span></span></code></pre></div><p>There are other options that could be set by Java API, please see the
-<a href=../../../javadoc/1.3.0/org/apache/iceberg/flink/source/IcebergSource.html>IcebergSource#Builder</a>.</p><h3 id=reading-branches-and-tags-with-datastream>Reading branches and tags with DataStream</h3><p>Branches and tags can also be read via the DataStream API</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>StreamExecutionEnvironment [...]
+<a href=../../../javadoc/1.3.1/org/apache/iceberg/flink/source/IcebergSource.html>IcebergSource#Builder</a>.</p><h3 id=reading-branches-and-tags-with-datastream>Reading branches and tags with DataStream</h3><p>Branches and tags can also be read via the DataStream API</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>StreamExecutionEnvironment [...]
</span></span><span style=display:flex><span>TableLoader tableLoader <span style=color:#f92672>=</span> TableLoader<span style=color:#f92672>.</span><span style=color:#a6e22e>fromHadoopTable</span><span style=color:#f92672>(</span><span style=color:#e6db74>"hdfs://nn:8020/warehouse/path"</span><span style=color:#f92672>);</span>
</span></span><span style=display:flex><span><span style=color:#75715e>// Read from branch
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>DataStream<span style=color:#f92672><</span>RowData<span style=color:#f92672>></span> batch <span style=color:#f92672>=</span> FlinkSource<span style=color:#f92672>.</span><span style=color:#a6e22e>forRowData</span><span style=color:#f92672>()</span>
diff --git a/docs/latest/flink-writes/index.html b/docs/latest/flink-writes/index.html
index 2fc536cc..c7af29e2 100644
--- a/docs/latest/flink-writes/index.html
+++ b/docs/latest/flink-writes/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/flink/index.html b/docs/latest/flink/index.html
index 5839c570..f867af5e 100644
--- a/docs/latest/flink/index.html
+++ b/docs/latest/flink/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -31,14 +31,14 @@
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span><span style=color:#75715e># Start the flink standalone cluster</span>
</span></span><span style=display:flex><span>./bin/start-cluster.sh
-</span></span></code></pre></div><p>Start the Flink SQL client. There is a separate <code>flink-runtime</code> module in the Iceberg project to generate a bundled jar, which could be loaded by Flink SQL client directly. To build the <code>flink-runtime</code> bundled jar manually, build the <code>iceberg</code> project, and it will generate the jar under <code><iceberg-root-dir>/flink-runtime/build/libs</code>. Or download the <code>flink-runtime</code> jar from the <a href=https://re [...]
+</span></span></code></pre></div><p>Start the Flink SQL client. There is a separate <code>flink-runtime</code> module in the Iceberg project to generate a bundled jar, which could be loaded by Flink SQL client directly. To build the <code>flink-runtime</code> bundled jar manually, build the <code>iceberg</code> project, and it will generate the jar under <code><iceberg-root-dir>/flink-runtime/build/libs</code>. Or download the <code>flink-runtime</code> jar from the <a href=https://re [...]
</span></span><span style=display:flex><span>export HADOOP_CLASSPATH<span style=color:#f92672>=</span><span style=color:#e6db74>`</span>$HADOOP_HOME/bin/hadoop classpath<span style=color:#e6db74>`</span>
</span></span><span style=display:flex><span>
-</span></span><span style=display:flex><span>./bin/sql-client.sh embedded -j <flink-runtime-directory>/iceberg-flink-runtime-1.16-1.3.0.jar shell
+</span></span><span style=display:flex><span>./bin/sql-client.sh embedded -j <flink-runtime-directory>/iceberg-flink-runtime-1.16-1.3.1.jar shell
</span></span></code></pre></div><p>By default, Iceberg ships with Hadoop jars for Hadoop catalog. To use Hive catalog, load the Hive jars when opening the Flink SQL client. Fortunately, Flink has provided a <a href=https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-hive-2.3.9_2.12/1.16.1/flink-sql-connector-hive-2.3.9_2.12-1.16.1.jar>bundled hive jar</a> for the SQL client. An example on how to download the dependencies and get started:</p><div class=highlight><pr [...]
</span></span><span style=display:flex><span>export HADOOP_CLASSPATH<span style=color:#f92672>=</span><span style=color:#e6db74>`</span>$HADOOP_HOME/bin/hadoop classpath<span style=color:#e6db74>`</span>
</span></span><span style=display:flex><span>
-</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.0
+</span></span><span style=display:flex><span>ICEBERG_VERSION<span style=color:#f92672>=</span>1.3.1
</span></span><span style=display:flex><span>MAVEN_URL<span style=color:#f92672>=</span>https://repo1.maven.org/maven2
</span></span><span style=display:flex><span>ICEBERG_MAVEN_URL<span style=color:#f92672>=</span><span style=color:#e6db74>${</span>MAVEN_URL<span style=color:#e6db74>}</span>/org/apache/iceberg
</span></span><span style=display:flex><span>ICEBERG_PACKAGE<span style=color:#f92672>=</span>iceberg-flink-runtime
@@ -58,7 +58,7 @@
</span></span><span style=display:flex><span><span style=color:#f92672>from</span> pyflink.datastream <span style=color:#f92672>import</span> StreamExecutionEnvironment
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env <span style=color:#f92672>=</span> StreamExecutionEnvironment<span style=color:#f92672>.</span>get_execution_environment()
-</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(), <span style=color:#e6db74>"iceberg-flink-runtime-1.16-1.3.0.jar"</span>)
+</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(), <span style=color:#e6db74>"iceberg-flink-runtime-1.16-1.3.1.jar"</span>)
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env<span style=color:#f92672>.</span>add_jars(<span style=color:#e6db74>"file://</span><span style=color:#e6db74>{}</span><span style=color:#e6db74>"</span><span style=color:#f92672>.</span>format(iceberg_flink_runtime_jar))
</span></span></code></pre></div><p>Next, create a <code>StreamTableEnvironment</code> and execute Flink SQL statements. The below example shows how to create a custom catalog via the Python Table API:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-python data-lang=python><span style=display:flex><span><span style=color:#f92672>from</span> pyflink.table <span style=color:#f92672>import</sp [...]
diff --git a/docs/latest/getting-started/index.html b/docs/latest/getting-started/index.html
index f5eb16cb..a28c892f 100644
--- a/docs/latest/getting-started/index.html
+++ b/docs/latest/getting-started/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,10 +15,10 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
We recommend you to get started with Spark to understand Iceberg concepts and features with examples.
-You can also view documentations of using Iceberg with other compute engine under the <a href=https://iceberg.apache.org/multi-engine-support>Multi-Engine Support</a> page.</p><h2 id=using-iceberg-in-spark-3>Using Iceberg in Spark 3</h2><p>To use Iceberg in a Spark shell, use the <code>--packages</code> option:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=disp [...]
-</span></span></code></pre></div><div class=info>If you want to include Iceberg in your Spark installation, add the <a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/1.3.0/iceberg-spark-runtime-3.2_2.12-1.3.0.jar"><code>iceberg-spark-runtime-3.2_2.12</code> Jar</a> to Spark’s <code>jars</code> folder.</div><h3 id=adding-catalogs>Adding catalogs</h3><p>Iceberg comes with <a href=../spark-configuration#catalogs>catalogs</a> tha [...]
+You can also view documentations of using Iceberg with other compute engine under the <a href=https://iceberg.apache.org/multi-engine-support>Multi-Engine Support</a> page.</p><h2 id=using-iceberg-in-spark-3>Using Iceberg in Spark 3</h2><p>To use Iceberg in a Spark shell, use the <code>--packages</code> option:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sh data-lang=sh><span style=disp [...]
+</span></span></code></pre></div><div class=info>If you want to include Iceberg in your Spark installation, add the <a href="https://search.maven.org/remotecontent?filepath=org/apache/iceberg/iceberg-spark-runtime-3.2_2.12/1.3.1/iceberg-spark-runtime-3.2_2.12-1.3.1.jar"><code>iceberg-spark-runtime-3.2_2.12</code> Jar</a> to Spark’s <code>jars</code> folder.</div><h3 id=adding-catalogs>Adding catalogs</h3><p>Iceberg comes with <a href=../spark-configuration#catalogs>catalogs</a> tha [...]
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.extensions<span style=color:#f92672>=</span>org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkSessionCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.spark_catalog.type<span style=color:#f92672>=</span>hive <span style=color:#ae81ff>\
diff --git a/docs/latest/hive-migration/index.html b/docs/latest/hive-migration/index.html
index e8114365..c748dee6 100644
--- a/docs/latest/hive-migration/index.html
+++ b/docs/latest/hive-migration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/hive/index.html b/docs/latest/hive/index.html
index 5dbc8bcb..35b73bd2 100644
--- a/docs/latest/hive/index.html
+++ b/docs/latest/hive/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -71,7 +71,7 @@ The data is inserted / committed when the query finishes. So for a transient per
</span></span><span style=display:flex><span> <span style=color:#66d9ef>SELECT</span> <span style=color:#f92672>*</span> <span style=color:#66d9ef>FROM</span> <span style=color:#66d9ef>source</span>;
</span></span></code></pre></div><h3 id=create-table-like-table>CREATE TABLE LIKE TABLE</h3><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CREATE</span> <span style=color:#66d9ef>TABLE</span> target <span style=color:#66d9ef>LIKE</span> <span style=color:#66d9ef>source</span> STORED <span style=color:#66d9ef>BY</span> [...]
</span></span></code></pre></div><h3 id=create-external-table-overlaying-an-existing-iceberg-table>CREATE EXTERNAL TABLE overlaying an existing Iceberg table</h3><p>The <code>CREATE EXTERNAL TABLE</code> command is used to overlay a Hive table “on top of” an existing Iceberg table. Iceberg
-tables are created using either a <a href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/catalog/Catalog.html><code>Catalog</code></a>, or an implementation of the <a href=../../../javadoc/1.3.0/index.html?org/apache/iceberg/Tables.html><code>Tables</code></a> interface, and Hive needs to be configured accordingly to
+tables are created using either a <a href=../../../javadoc/1.3.1/index.html?org/apache/iceberg/catalog/Catalog.html><code>Catalog</code></a>, or an implementation of the <a href=../../../javadoc/1.3.1/index.html?org/apache/iceberg/Tables.html><code>Tables</code></a> interface, and Hive needs to be configured accordingly to
operate on these different types of table.</p><h4 id=hive-catalog-tables>Hive catalog tables</h4><p>As described before, tables created by the <code>HiveCatalog</code> with Hive engine feature enabled are directly visible by the
Hive engine, so there is no need to create an overlay.</p><h4 id=custom-catalog-tables>Custom catalog tables</h4><p>For a table in a registered catalog, specify the catalog name in the statement using table property <code>iceberg.catalog</code>.
For example, the SQL below creates an overlay for a table in a <code>hadoop</code> type catalog named <code>hadoop_cat</code>:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>SET</span>
diff --git a/docs/latest/index.html b/docs/latest/index.html
index 713337a0..8384fa6d 100644
--- a/docs/latest/index.html
+++ b/docs/latest/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=./branching/>Branching and Tagging</a></li><li><a href=./configuration/>Configuration</a></li><li><a href=./evolution/>Evolution</a></li><li><a href=./maintenance/>Maintenance</a></li><li><a href=./partitioning/>Partitioning</a></li><li><a href=./performance/>Performance</a></li><li><a href=./reliability/>Reliability</a></li><li><a href=./schemas/>Schemas</a></li></ul></div><li><a clas [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/index.xml b/docs/latest/index.xml
index 9b6c22a8..a979fd30 100644
--- a/docs/latest/index.xml
+++ b/docs/latest/index.xml
@@ -1,4 +1,4 @@
-<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Introduction on Apache Iceberg</title><link>https://iceberg.apache.org/docs/latest/</link><description>Recent content in Introduction on Apache Iceberg</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><atom:link href="https://iceberg.apache.org/docs/latest/index.xml" rel="self" type="application/rss+xml"/><item><title>Getting Sta [...]
+<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Introduction on Apache Iceberg</title><link>https://iceberg.apache.org/docs/latest/</link><description>Recent content in Introduction on Apache Iceberg</description><generator>Hugo -- gohugo.io</generator><language>en-us</language><atom:link href="https://iceberg.apache.org/docs/latest/index.xml" rel="self" type="application/rss+xml"/><item><title>Getting Sta [...]
Spark is currently the most feature-rich compute engine for Iceberg operations. We recommend you to get started with Spark to understand Iceberg concepts and features with examples. You can also view documentations of using Iceberg with other compute engine under the Multi-Engine Support page.
Using Iceberg in Spark 3 To use Iceberg in a Spark shell, use the --packages option:</description></item><item><title>Hive</title><link>https://iceberg.apache.org/docs/latest/hive/</link><pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate><guid>https://iceberg.apache.org/docs/latest/hive/</guid><description>Hive Iceberg supports reading and writing Iceberg tables through Hive by using a StorageHandler.
Feature support Iceberg compatibility with Hive 2.x and Hive 3.1.2/3 supports the following features:
diff --git a/docs/latest/java-api-quickstart/index.html b/docs/latest/java-api-quickstart/index.html
index 4607002e..c88f5efc 100644
--- a/docs/latest/java-api-quickstart/index.html
+++ b/docs/latest/java-api-quickstart/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,7 +15,7 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class="collapse in"><ul class=sub-menu><li><a id=active href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
You can initialize a Hive catalog with a name and some properties.
(see: <a href=../configuration/#catalog-properties>Catalog properties</a>)</p><p><strong>Note:</strong> Currently, <code>setConf</code> is always required for hive catalogs, but this will change in the future.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span><span style=color:#f92672>import</span> org.apache.iceberg.hive.HiveCatalog<span sty [...]
</span></span><span style=display:flex><span>
diff --git a/docs/latest/jdbc/index.html b/docs/latest/jdbc/index.html
index 3aac90e0..1c465a43 100644
--- a/docs/latest/jdbc/index.html
+++ b/docs/latest/jdbc/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -18,7 +18,7 @@
<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
The database that JDBC connects to must support atomic transaction to allow the JDBC catalog implementation to
properly support atomic Iceberg table commits and read serializable isolation.</p><h3 id=configurations>Configurations</h3><p>Because each database and database service provider might require different configurations,
-the JDBC catalog allows arbitrary configurations through:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>uri</td><td></td><td>the JDBC connection string</td></tr><tr><td>jdbc.<property_key></td><td></td><td>any key value pairs to configure the JDBC connection</td></tr></tbody></table><h3 id=examples>Examples</h3><h4 id=spark>Spark</h4><p>You can start a Spark session with a MySQL JDBC connection using the following configurations: [...]
+the JDBC catalog allows arbitrary configurations through:</p><table><thead><tr><th>Property</th><th>Default</th><th>Description</th></tr></thead><tbody><tr><td>uri</td><td></td><td>the JDBC connection string</td></tr><tr><td>jdbc.<property_key></td><td></td><td>any key value pairs to configure the JDBC connection</td></tr></tbody></table><h3 id=examples>Examples</h3><h4 id=spark>Spark</h4><p>You can start a Spark session with a MySQL JDBC connection using the following configurations: [...]
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog<span style=color:#f92672>=</span>org.apache.iceberg.spark.SparkCatalog <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog.warehouse<span style=color:#f92672>=</span>s3://my-bucket/my/key/prefix <span style=color:#ae81ff>\
</span></span></span><span style=display:flex><span><span style=color:#ae81ff></span> --conf spark.sql.catalog.my_catalog.catalog-impl<span style=color:#f92672>=</span>org.apache.iceberg.jdbc.JdbcCatalog <span style=color:#ae81ff>\
diff --git a/docs/latest/maintenance/index.html b/docs/latest/maintenance/index.html
index eddfb6c0..3d85363b 100644
--- a/docs/latest/maintenance/index.html
+++ b/docs/latest/maintenance/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a id=active href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
@@ -15,12 +15,12 @@
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=API class=collapse><ul class=sub-menu><li><a href=../java-api-quickstart/>Java Quickstart</a></li><li><a href=../api/>Java API</a></li><li><a href=../custom-catalog/>Java Custom Catalog</a></li></ul></div><li><a class="chevron-toggle collapsed" data-toggle=collapse data-parent=full href=#Migration><span>Migration</span>
<i class="fa fa-chevron-right"></i>
-<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
+<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
</span></span><span style=display:flex><span><span style=color:#66d9ef>long</span> tsToExpire <span style=color:#f92672>=</span> System<span style=color:#f92672>.</span><span style=color:#a6e22e>currentTimeMillis</span><span style=color:#f92672>()</span> <span style=color:#f92672>-</span> <span style=color:#f92672>(</span><span style=color:#ae81ff>1000</span> <span style=color:#f92672>*</span> <span style=color:#ae81ff>60</span> <span style=color:#f92672>*</span> <span style=color:#ae81f [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span>table<span style=color:#f92672>.</span><span style=color:#a6e22e>expireSnapshots</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>expireOlderThan</span><span style=color:#f92672>(</span>tsToExpire<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>commit</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>See the <a href=../../../javadoc/1.3.0/org/apache/iceberg/ExpireSnapshots.html><code>ExpireSnapshots</code> Javadoc</a> to see more configuration options.</p><p>There is also a Spark action that can run table expiration in parallel for large tables:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>Table ta [...]
+</span></span></code></pre></div><p>See the <a href=../../../javadoc/1.3.1/org/apache/iceberg/ExpireSnapshots.html><code>ExpireSnapshots</code> Javadoc</a> to see more configuration options.</p><p>There is also a Spark action that can run table expiration in parallel for large tables:</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-java data-lang=java><span style=display:flex><span>Table ta [...]
</span></span><span style=display:flex><span>SparkActions
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>expireSnapshots</span><span style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
@@ -34,7 +34,7 @@ Configuring <code>write.metadata.delete-after-commit.enabled=true</code> and <co
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>deleteOrphanFiles</span><span style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>See the <a href=../../../javadoc/1.3.0/org/apache/iceberg/actions/DeleteOrphanFiles.html>DeleteOrphanFiles Javadoc</a> to see more configuration options.</p><p>This action may take a long time to finish if you have lots of files in data and metadata directories. It is recommended to execute this periodically, but you may not need to execute this often.</p><div class=info>It is dangerous to remove orphan files with a retention interval shorter than the [...]
+</span></span></code></pre></div><p>See the <a href=../../../javadoc/1.3.1/org/apache/iceberg/actions/DeleteOrphanFiles.html>DeleteOrphanFiles Javadoc</a> to see more configuration options.</p><p>This action may take a long time to finish if you have lots of files in data and metadata directories. It is recommended to execute this periodically, but you may not need to execute this often.</p><div class=info>It is dangerous to remove orphan files with a retention interval shorter than the [...]
might corrupt the table if in-progress files are considered orphaned and are deleted. The default interval is 3 days.</div><div class=info>Iceberg uses the string representations of paths when determining which files need to be removed. On some file systems,
the path can change over time, but it still represents the same file. For example, if you change authorities for an HDFS cluster,
none of the old path urls used during creation will match those that appear in a current listing. <em>This will lead to data loss when
@@ -46,13 +46,13 @@ FileSystem API to avoid unintentional deletion.</div><h2 id=optional-maintenance
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>filter</span><span style=color:#f92672>(</span>Expressions<span style=color:#f92672>.</span><span style=color:#a6e22e>equal</span><span style=color:#f92672>(</span><span style=color:#e6db74>"date"</span><span style=color:#f92672>,</span> <span style=color:#e6db74>"2020-08-18"</span><span style=color:#f92672>))</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>option</span><span style=color:#f92672>(</span><span style=color:#e6db74>"target-file-size-bytes"</span><span style=color:#f92672>,</span> Long<span style=color:#f92672>.</span><span style=color:#a6e22e>toString</span><span style=color:#f92672>(</span><span style=color:#ae81ff>500</span> <span style=color:#f92672>*</span> <span style=color:#ae81ff>1024</span> <span style=c [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span> <span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>The <code>files</code> metadata table is useful for inspecting data file sizes and determining when to compact partitions.</p><p>See the <a href=../../../javadoc/1.3.0/org/apache/iceberg/actions/RewriteDataFiles.html><code>RewriteDataFiles</code> Javadoc</a> to see more configuration options.</p><h3 id=rewrite-manifests>Rewrite manifests</h3><p>Iceberg uses metadata in its manifest list and manifest files speed up query planning and to prune unnecessar [...]
+</span></span></code></pre></div><p>The <code>files</code> metadata table is useful for inspecting data file sizes and determining when to compact partitions.</p><p>See the <a href=../../../javadoc/1.3.1/org/apache/iceberg/actions/RewriteDataFiles.html><code>RewriteDataFiles</code> Javadoc</a> to see more configuration options.</p><h3 id=rewrite-manifests>Rewrite manifests</h3><p>Iceberg uses metadata in its manifest list and manifest files speed up query planning and to prune unnecessar [...]
</span></span><span style=display:flex><span>SparkActions
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>get</span><span style=color:#f92672>()</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>rewriteManifests</span><span style=color:#f92672>(</span>table<span style=color:#f92672>)</span>
</span></span><span style=display:flex><span> <span style=color:#f92672>.</span><span style=color:#a6e22e>rewriteIf</span><span style=color:#f92672>(</span>file <span style=color:#f92672>-></span> file<span style=color:#f92672>.</span><span style=color:#a6e22e>length</span><span style=color:#f92672>()</span> <span style=color:#f92672><</span> <span style=color:#ae81ff>10</span> <span style=color:#f92672>*</span> <span style=color:#ae81ff>1024</span> <span style=color:#f92672>*</ [...]
</span></span></span><span style=display:flex><span><span style=color:#75715e></span> <span style=color:#f92672>.</span><span style=color:#a6e22e>execute</span><span style=color:#f92672>();</span>
-</span></span></code></pre></div><p>See the <a href=../../../javadoc/1.3.0/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code> Javadoc</a> to see more configuration options.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#recommended-maintenance>Recommended Maintenance</a><ul><li><a href=#expire-snapshots>Expire Snapshots</a></li><li><a href=#remove-old-metadata-files>Remove old metadata files</a></li><li><a href=# [...]
+</span></span></code></pre></div><p>See the <a href=../../../javadoc/1.3.1/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code> Javadoc</a> to see more configuration options.</p></div><div id=toc class=markdown-body><div id=full><nav id=TableOfContents><ul><li><a href=#recommended-maintenance>Recommended Maintenance</a><ul><li><a href=#expire-snapshots>Expire Snapshots</a></li><li><a href=#remove-old-metadata-files>Remove old metadata files</a></li><li><a href=# [...]
<script src=https://iceberg.apache.org/docs/latest//js/jquery.easing.min.js></script>
<script type=text/javascript src=https://iceberg.apache.org/docs/latest//js/search.js></script>
<script src=https://iceberg.apache.org/docs/latest//js/bootstrap.min.js></script>
diff --git a/docs/latest/nessie/index.html b/docs/latest/nessie/index.html
index 496c947c..7a55d415 100644
--- a/docs/latest/nessie/index.html
+++ b/docs/latest/nessie/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -18,9 +18,9 @@
<i class="fa fa-chevron-down"></i></a></li><div id=Migration class=collapse><ul class=sub-menu><li><a href=../table-migration/>Overview</a></li><li><a href=../hive-migration/>Hive Migration</a></li><li><a href=../delta-lake-migration/>Delta Lake Migration</a></li></ul></div><li><a href=https://iceberg.apache.org/docs/latest/../../javadoc/latest><span>Javadoc</span></a></li><li><a target=_blank href=https://py.iceberg.apache.org/><span>PyIceberg</span></a></li></div></div><div id=content [...]
This section describes how to use Iceberg with Nessie. Nessie provides several key features on top of Iceberg:</p><ul><li>multi-table transactions</li><li>git-like operations (eg branches, tags, commits)</li><li>hive-like metastore capabilities</li></ul><p>See <a href=https://projectnessie.org>Project Nessie</a> for more information on Nessie. Nessie requires a server to run, see
<a href=https://projectnessie.org/try/>Getting Started</a> to start a Nessie server.</p><h2 id=enabling-nessie-catalog>Enabling Nessie Catalog</h2><p>The <code>iceberg-nessie</code> module is bundled with Spark and Flink runtimes for all versions from <code>0.11.0</code>. To get started
-with Nessie (with spark-3.3) and Iceberg simply add the Iceberg runtime to your process. Eg: <code>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.3.0</code>.</p><h2 id=spark-sql-extensions>Spark SQL Extensions</h2><p>From Spark 3.1 and above, Nessie SQL extensions can be used to manage the Nessie repo as shown below.
+with Nessie (with spark-3.3) and Iceberg simply add the Iceberg runtime to your process. Eg: <code>spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.3.1</code>.</p><h2 id=spark-sql-extensions>Spark SQL Extensions</h2><p>From Spark 3.1 and above, Nessie SQL extensions can be used to manage the Nessie repo as shown below.
Example for Spark 3.3 with scala 2.12:</p><pre tabindex=0><code>bin/spark-sql
- --packages "org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.3.0,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.3_2.12:0.59.0"
+ --packages "org.apache.iceberg:iceberg-spark-runtime-3.3_2.12:1.3.1,org.projectnessie.nessie-integrations:nessie-spark-extensions-3.3_2.12:0.59.0"
--conf spark.sql.extensions="org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSparkSessionExtensions"
--conf <other settings>
</code></pre><p>Please refer <a href=https://projectnessie.org/tools/sql/>Nessie SQL extension document</a> to learn more about it.</p><h2 id=nessie-catalog>Nessie Catalog</h2><p>One major feature introduced in release <code>0.11.0</code> is the ability to easily interact with a <a href=../custom-catalog>Custom
@@ -41,7 +41,7 @@ and <a href=../flink#custom-catalog>Flink Configuration</a> for instructions for
</span></span><span style=display:flex><span><span style=color:#f92672>from</span> pyflink.table <span style=color:#f92672>import</span> StreamTableEnvironment
</span></span><span style=display:flex><span>
</span></span><span style=display:flex><span>env <span style=color:#f92672>=</span> StreamExecutionEnvironment<span style=color:#f92672>.</span>get_execution_environment()
-</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(), <span style=color:#e6db74>"iceberg-flink-runtime-1.3.0.jar"</span>)
+</span></span><span style=display:flex><span>iceberg_flink_runtime_jar <span style=color:#f92672>=</span> os<span style=color:#f92672>.</span>path<span style=color:#f92672>.</span>join(os<span style=color:#f92672>.</span>getcwd(), <span style=color:#e6db74>"iceberg-flink-runtime-1.3.1.jar"</span>)
</span></span><span style=display:flex><span>env<span style=color:#f92672>.</span>add_jars(<span style=color:#e6db74>"file://</span><span style=color:#e6db74>{}</span><span style=color:#e6db74>"</span><span style=color:#f92672>.</span>format(iceberg_flink_runtime_jar))
</span></span><span style=display:flex><span>table_env <span style=color:#f92672>=</span> StreamTableEnvironment<span style=color:#f92672>.</span>create(env)
</span></span><span style=display:flex><span>
diff --git a/docs/latest/partitioning/index.html b/docs/latest/partitioning/index.html
index 4a228a69..34b6b492 100644
--- a/docs/latest/partitioning/index.html
+++ b/docs/latest/partitioning/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a id=active href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/performance/index.html b/docs/latest/performance/index.html
index 6187596d..1fca843d 100644
--- a/docs/latest/performance/index.html
+++ b/docs/latest/performance/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a id=active href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/reliability/index.html b/docs/latest/reliability/index.html
index e92e3c39..b7b02df5 100644
--- a/docs/latest/reliability/index.html
+++ b/docs/latest/reliability/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a id=active href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/schemas/index.html b/docs/latest/schemas/index.html
index b2462d35..f3dccf48 100644
--- a/docs/latest/schemas/index.html
+++ b/docs/latest/schemas/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class="collapse in"><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a id=active href=../schemas/>Schemas</a></li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-configuration/index.html b/docs/latest/spark-configuration/index.html
index 8427c2d9..3eddbe23 100644
--- a/docs/latest/spark-configuration/index.html
+++ b/docs/latest/spark-configuration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-ddl/index.html b/docs/latest/spark-ddl/index.html
index 84838562..5a0eb594 100644
--- a/docs/latest/spark-ddl/index.html
+++ b/docs/latest/spark-ddl/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-procedures/index.html b/docs/latest/spark-procedures/index.html
index fc6ac716..824a0e4f 100644
--- a/docs/latest/spark-procedures/index.html
+++ b/docs/latest/spark-procedures/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
@@ -29,8 +29,8 @@ the <code>expire_snapshots</code> procedure will never remove files which are st
</span></span></code></pre></div><p>Remove snapshots with snapshot ID <code>123</code> (note that this snapshot ID should not be the current snapshot):</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> hive_prod.<span style=color:#66d9ef>system</span>.expire_snapshots(<span style=color:#66d9ef>table</span> [...]
</span></span></code></pre></div><h3 id=remove_orphan_files><code>remove_orphan_files</code></h3><p>Used to remove files which are not referenced in any metadata files of an Iceberg table and can thus be considered “orphaned”.</p><h4 id=usage-6>Usage</h4><table><thead><tr><th>Argument Name</th><th>Required?</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>table</code></td><td>✔️</td><td>string</td><td>Name of the table to clean</td></tr><tr><td><code>old [...]
</span></span></code></pre></div><p>Remove any files in the <code>tablelocation/data</code> folder which are not known to the table <code>db.sample</code>.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef>system</span>.remove_orphan_fi [...]
-</span></span></code></pre></div><h3 id=rewrite_data_files><code>rewrite_data_files</code></h3><p>Iceberg tracks each data file in a table. More data files leads to more metadata stored in manifest files, and small data files causes an unnecessary amount of metadata and less efficient queries from file open costs.</p><p>Iceberg can compact data files in parallel using Spark with the <code>rewriteDataFiles</code> action. This will combine small files into larger files to reduce metadata o [...]
-and <a href=../../../javadoc/1.3.0/org/apache/iceberg/actions/SortStrategy.html#field.summary><code>SortStrategy</code> Javadoc</a>
+</span></span></code></pre></div><h3 id=rewrite_data_files><code>rewrite_data_files</code></h3><p>Iceberg tracks each data file in a table. More data files leads to more metadata stored in manifest files, and small data files causes an unnecessary amount of metadata and less efficient queries from file open costs.</p><p>Iceberg can compact data files in parallel using Spark with the <code>rewriteDataFiles</code> action. This will combine small files into larger files to reduce metadata o [...]
+and <a href=../../../javadoc/1.3.1/org/apache/iceberg/actions/SortStrategy.html#field.summary><code>SortStrategy</code> Javadoc</a>
for list of all the supported options for this action.</p><h4 id=output-6>Output</h4><table><thead><tr><th>Output Name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>rewritten_data_files_count</code></td><td>int</td><td>Number of data which were re-written by this command</td></tr><tr><td><code>added_data_files_count</code></td><td>int</td><td>Number of new data files which were written by this command</td></tr></tbody></table><h4 id=examples-3>Examples</h4><p>Rew [...]
and also split large files according to the default write size of the table.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef>system</span>.rewrite_data_files(<span style=color:#e6db74>'db.sample'</span>)
</span></span></code></pre></div><p>Rewrite the data files in table <code>db.sample</code> by sorting all the data on id and name
@@ -39,10 +39,10 @@ using the same defaults as bin-pack to determine which files to rewrite.</p><div
Using the same defaults as bin-pack to determine which files to rewrite.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef>system</span>.rewrite_data_files(<span style=color:#66d9ef>table</span> <span style=color:#f92672>=></span> <s [...]
</span></span></code></pre></div><p>Rewrite the data files in table <code>db.sample</code> using bin-pack strategy in any partition where more than 2 or more files need to be rewritten.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef> [...]
</span></span></code></pre></div><p>Rewrite the data files in table <code>db.sample</code> and select the files that may contain data matching the filter (id = 3 and name = “foo”) to be rewritten.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<sp [...]
-</span></span></code></pre></div><h3 id=rewrite_manifests><code>rewrite_manifests</code></h3><p>Rewrite manifests for a table to optimize scan planning.</p><p>Data files in manifests are sorted by fields in the partition spec. This procedure runs in parallel using a Spark job.</p><p>See the <a href=../../../javadoc/1.3.0/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code> Javadoc</a>
+</span></span></code></pre></div><h3 id=rewrite_manifests><code>rewrite_manifests</code></h3><p>Rewrite manifests for a table to optimize scan planning.</p><p>Data files in manifests are sorted by fields in the partition spec. This procedure runs in parallel using a Spark job.</p><p>See the <a href=../../../javadoc/1.3.1/org/apache/iceberg/actions/RewriteManifests.html><code>RewriteManifests</code> Javadoc</a>
to see more configuration options.</p><div class=info>This procedure invalidates all cached Spark plans that reference the affected table.</div><h4 id=usage-8>Usage</h4><table><thead><tr><th>Argument Name</th><th>Required?</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>table</code></td><td>✔️</td><td>string</td><td>Name of the table to update</td></tr><tr><td><code>use_caching</code></td><td>️</td><td>boolean</td><td>Use Spark caching during operation (defaults to [...]
</span></span></code></pre></div><p>Rewrite the manifests in table <code>db.sample</code> and disable the use of Spark caching. This could be done to avoid memory issues on executors.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</span>.<span style=color:#66d9ef>sy [...]
-</span></span></code></pre></div><h3 id=rewrite_position_delete_files><code>rewrite_position_delete_files</code></h3><p>Iceberg can rewrite position delete files, which serves two purposes:</p><ul><li>Minor Compaction: Compact small position delete files into larger ones. This reduces the size of metadata stored in manifest files and overhead of opening small delete files.</li><li>Remove Dangling Deletes: Filter out position delete records that refer to data files that are no longer live [...]
+</span></span></code></pre></div><h3 id=rewrite_position_delete_files><code>rewrite_position_delete_files</code></h3><p>Iceberg can rewrite position delete files, which serves two purposes:</p><ul><li>Minor Compaction: Compact small position delete files into larger ones. This reduces the size of metadata stored in manifest files and overhead of opening small delete files.</li><li>Remove Dangling Deletes: Filter out position delete records that refer to data files that are no longer live [...]
for list of all the supported options for this procedure.</p><p>Dangling deletes are always filtered out during rewriting.</p><h4 id=output-8>Output</h4><table><thead><tr><th>Output Name</th><th>Type</th><th>Description</th></tr></thead><tbody><tr><td><code>rewritten_delete_files_count</code></td><td>int</td><td>Number of delete files which were removed by this command</td></tr><tr><td><code>added_delete_files_count</code></td><td>int</td><td>Number of delete files which were added by th [...]
</span></span></code></pre></div><p>Rewrite all position delete files in table <code>db.sample</code>, writing new files <code>target-file-size-bytes</code>. Dangling deletes are removed from rewritten delete files.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span style=color:#66d9ef>CALL</span> <span style=color:#66d9ef>catalog_name</sp [...]
</span></span></code></pre></div><p>Rewrite position delete files in table <code>db.sample</code>. This selects position delete files in partitions where 2 or more position delete files need to be rewritten based on size criteria. Dangling deletes are removed from rewritten delete files.</p><div class=highlight><pre tabindex=0 style=color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4><code class=language-sql data-lang=sql><span style=display:flex><span><span s [...]
diff --git a/docs/latest/spark-queries/index.html b/docs/latest/spark-queries/index.html
index 0e8cb320..7380e7d8 100644
--- a/docs/latest/spark-queries/index.html
+++ b/docs/latest/spark-queries/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-structured-streaming/index.html b/docs/latest/spark-structured-streaming/index.html
index 29eadb8a..618ab88a 100644
--- a/docs/latest/spark-structured-streaming/index.html
+++ b/docs/latest/spark-structured-streaming/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/spark-writes/index.html b/docs/latest/spark-writes/index.html
index 813d7b6a..05af81ba 100644
--- a/docs/latest/spark-writes/index.html
+++ b/docs/latest/spark-writes/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>
diff --git a/docs/latest/table-migration/index.html b/docs/latest/table-migration/index.html
index f880bfdf..910d199a 100644
--- a/docs/latest/table-migration/index.html
+++ b/docs/latest/table-migration/index.html
@@ -3,7 +3,7 @@
<span class=icon-bar></span>
<span class=icon-bar></span>
<span class=icon-bar></span></button>
-<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.0</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
+<a class="page-scroll navbar-brand" href=https://iceberg.apache.org/><img class=top-navbar-logo src=https://iceberg.apache.org/docs/latest//img/iceberg-logo-icon.png> Apache Iceberg</a></div><div><input type=search class=form-control id=search-input placeholder=Search... maxlength=64 data-hotkeys=s/></div><div class=versions-dropdown><span>1.3.1</span> <i class="fa fa-chevron-down"></i><div class=versions-dropdown-content><ul><li class=versions-dropdown-selection><a href=https://iceberg. [...]
<i class="fa fa-chevron-right"></i>
<i class="fa fa-chevron-down"></i></a></li><div id=Tables class=collapse><ul class=sub-menu><li><a href=../branching/>Branching and Tagging</a></li><li><a href=../configuration/>Configuration</a></li><li><a href=../evolution/>Evolution</a></li><li><a href=../maintenance/>Maintenance</a></li><li><a href=../partitioning/>Partitioning</a></li><li><a href=../performance/>Performance</a></li><li><a href=../reliability/>Reliability</a></li><li><a href=../schemas/>Schemas</a></li></ul></div><li [...]
<i class="fa fa-chevron-right"></i>